Menu
Home
People
Places
Arts
History
Plants & Animals
Science
Life & Culture
Technology
Reference.org
Analysis of algorithms
open-in-new
<h2 id="cost-models">Cost models</h2> <p>Time efficiency estimates depend on what we define to be a step. For the analysis to correspond usefully to the actual run-time, the time required to perform a step must be guaranteed to be bounded above by a constant. One must be careful here; for instance, some analyses count an addition of two numbers as one step. This assumption may not be warranted in certain contexts. For example, if the numbers involved in a computation may be arbitrarily large, the time required by a single addition can no longer be assumed to be constant. </p><p>Two cost models are generally used:<a class="footnote-ref" id="fnref:3" href="#fn:3"><sup>3</sup></a><a class="footnote-ref" id="fnref:4" href="#fn:4"><sup>4</sup></a><a class="footnote-ref" id="fnref:5" href="#fn:5"><sup>5</sup></a><a class="footnote-ref" id="fnref:6" href="#fn:6"><sup>6</sup></a><a class="footnote-ref" id="fnref:7" href="#fn:7"><sup>7</sup></a> </p> <ul><li>the uniform cost model, also called unit-cost model (and similar variations), assigns a constant cost to every machine operation, regardless of the size of the numbers involved</li> <li>the logarithmic cost model, also called logarithmic-cost measurement (and similar variations), assigns a cost to every machine operation proportional to the number of bits involved</li></ul> <p>The latter is more cumbersome to use, so it is only employed when necessary, for example in the analysis of <a href="/facts/Arbitrary-precision_arithmetic/cXxc8yB2">arbitrary-precision arithmetic</a> algorithms, like those used in <a href="/facts/Cryptography/e94PEVau">cryptography</a>. </p><p>A key point which is often overlooked is that published lower bounds for problems are often given for a model of computation that is more restricted than the set of operations that you could use in practice and therefore there are algorithms that are faster than what would naively be thought possible.<a class="footnote-ref" id="fnref:8" href="#fn:8"><sup>8</sup></a> </p> <h2 id="run-time-analysis">Run-time analysis</h2> <p>Run-time analysis is a theoretical classification that estimates and anticipates the increase in <i><a href="/facts/DTIME/oyVAQfHk">running time</a></i> (or run-time or execution time) of an <a href="/facts/Algorithm/fnl5NmRt">algorithm</a> as its <i><a href="/facts/Information/3LdrzXtJ">input size</a></i> (usually denoted as <i>n</i>) increases. Run-time efficiency is a topic of great interest in <a href="/facts/Computer_science/lN3pEyPz">computer science</a>: A <a href="/facts/Computer_program/e38jH9N9">program</a> can take seconds, hours, or even years to finish executing, depending on which algorithm it implements. While <a href="/facts/Software_profiling/JbSlDHPP">software profiling</a> techniques can be used to measure an algorithm's run-time in practice, they cannot provide timing data for all infinitely many possible inputs; the latter can only be achieved by the theoretical methods of run-time analysis. </p> <h3>Shortcomings of empirical metrics</h3> <p>Since algorithms are <a href="/facts/Platform-independent/16zqdvNR">platform-independent</a> (i.e. a given algorithm can be implemented in an arbitrary <a href="/facts/Programming_language/cWTYbgWa">programming language</a> on an arbitrary <a href="/facts/Computer/SQe1aK05">computer</a> running an arbitrary <a href="/facts/Operating_system/8XTuvMh2">operating system</a>), there are additional significant drawbacks to using an <a href="/facts/Empirical/JUaYSGJc">empirical</a> approach to gauge the comparative performance of a given set of algorithms. </p><p>Take as an example a program that looks up a specific entry in a <a href="/facts/Collation/cdLx5Cum">sorted</a> <a href="/facts/List_(computing)/9tzhAH6M">list</a> of size <i>n</i>. Suppose this program were implemented on Computer A, a state-of-the-art machine, using a <a href="/facts/Linear_search/cBh6w2sq">linear search</a> algorithm, and on Computer B, a much slower machine, using a <a href="/facts/Binary_search_algorithm/FFmQzunc">binary search algorithm</a>. <a href="/facts/Benchmark_(computing)/XSS42cRe">Benchmark testing</a> on the two computers running their respective programs might look something like the following: </p> <table><tbody><tr><th><i>n</i> (list size)</th><th>Computer A run-time(in <a href="/facts/Nanosecond/gARLTp4c">nanoseconds</a>)</th><th>Computer B run-time(in <a href="/facts/Nanosecond/gARLTp4c">nanoseconds</a>)</th></tr><tr><td>16</td><td>8</td><td>100,000</td></tr><tr><td>63</td><td>32</td><td>150,000</td></tr><tr><td>250</td><td>125</td><td>200,000</td></tr><tr><td>1,000</td><td>500</td><td>250,000</td></tr></tbody></table> <p>Based on these metrics, it would be easy to jump to the conclusion that <i>Computer A</i> is running an algorithm that is far superior in efficiency to that of <i>Computer B</i>. However, if the size of the input-list is increased to a sufficient number, that conclusion is dramatically demonstrated to be in error: </p> <table><tbody><tr><th><i>n</i> (list size)</th><th>Computer A run-time(in <a href="/facts/Nanosecond/gARLTp4c">nanoseconds</a>)</th><th>Computer B run-time(in <a href="/facts/Nanosecond/gARLTp4c">nanoseconds</a>)</th></tr><tr><td>16</td><td>8</td><td>100,000</td></tr><tr><td>63</td><td>32</td><td>150,000</td></tr><tr><td>250</td><td>125</td><td>200,000</td></tr><tr><td>1,000</td><td>500</td><td>250,000</td></tr><tr><td>...</td><td>...</td><td>...</td></tr><tr><td>1,000,000</td><td>500,000</td><td>500,000</td></tr><tr><td>4,000,000</td><td>2,000,000</td><td>550,000</td></tr><tr><td>16,000,000</td><td>8,000,000</td><td>600,000</td></tr><tr><td>...</td><td>...</td><td>...</td></tr><tr><td>63,072 × 1012</td><td>31,536 × 1012 ns,or 1 year</td><td>1,375,000 ns,or 1.375 milliseconds</td></tr></tbody></table> <p>Computer A, running the linear search program, exhibits a <a href="/facts/Linear/fChtg3KC">linear</a> growth rate. The program's run-time is directly proportional to its input size. Doubling the input size doubles the run-time, quadrupling the input size quadruples the run-time, and so forth. On the other hand, Computer B, running the binary search program, exhibits a <a href="/facts/Logarithm/QeXaV97q">logarithmic</a> growth rate. Quadrupling the input size only increases the run-time by a constant amount (in this example, 50,000 ns). Even though Computer A is ostensibly a faster machine, Computer B will inevitably surpass Computer A in run-time because it is running an algorithm with a much slower growth rate. </p> <h3>Orders of growth</h3> <p class="note">Main article: <a href="/facts/Big_O_notation/weFFjSWg">Big O notation</a></p> <p>Informally, an algorithm can be said to exhibit a growth rate on the order of a <a href="/facts/Function_(mathematics)/CdReEKCh">mathematical function</a> if beyond a certain input size <i>n</i>, the function <i>f</i>(<i>n</i>) times a positive constant provides an <a href="/facts/Asymptotic_analysis/kt87qLpu">upper bound or limit</a> for the run-time of that algorithm. In other words, for a given input size <i>n</i> greater than some <i>n</i>0 and a constant <i>c</i>, the run-time of that algorithm will never be larger than <i>c</i> × <i>f</i>(<i>n</i>). This concept is frequently expressed using Big O notation. For example, since the run-time of <a href="/facts/Insertion_sort/N5yWe8Co">insertion sort</a> <a href="/facts/Quadratic_growth/5oILdKRt">grows quadratically</a> as its input size increases, insertion sort can be said to be of order <i>O</i>(<i>n</i>2). </p><p>Big O notation is a convenient way to express the <a href="/facts/Best%2c_worst_and_average_case/Ed9ekH30">worst-case scenario</a> for a given algorithm, although it can also be used to express the average-case — for example, the worst-case scenario for <a href="/facts/Quicksort/1BeqhLAH">quicksort</a> is <i>O</i>(<i>n</i>2), but the average-case run-time is <i>O</i>(<i>n</i> log <i>n</i>). </p> <h3>Empirical orders of growth</h3> <p>Assuming the run-time follows power rule, <i>t</i> ≈ <i>kn</i><i>a</i>, the coefficient <i>a</i> can be found <a class="footnote-ref" id="fnref:9" href="#fn:9"><sup>9</sup></a> by taking empirical measurements of run-time {<i>t</i>1, <i>t</i>2} at some problem-size points {<i>n</i>1, <i>n</i>2}, and calculating <i>t</i>2/<i>t</i>1 = (<i>n</i>2/<i>n</i>1)<i>a</i> so that <i>a</i> = log(<i>t</i>2/<i>t</i>1)/log(<i>n</i>2/<i>n</i>1). In other words, this measures the slope of the empirical line on the <a href="/facts/Log%25E2%2580%2593log_plot/VES46X8k">log–log plot</a> of run-time vs. input size, at some size point. If the order of growth indeed follows the power rule (and so the line on the log–log plot is indeed a straight line), the empirical value of will stay constant at different ranges, and if not, it will change (and the line is a curved line)—but still could serve for comparison of any two given algorithms as to their <i>empirical local orders of growth</i> behaviour. Applied to the above table: </p> <table><tbody><tr><th><i>n</i> (list size)</th><th>Computer A run-time(in <a href="/facts/Nanosecond/gARLTp4c">nanoseconds</a>)</th><th>Local order of growth(n^_)</th><th>Computer B run-time(in <a href="/facts/Nanosecond/gARLTp4c">nanoseconds</a>)</th><th>Local order of growth(n^_)</th></tr><tr><td>15</td><td>7</td><td></td><td>100,000</td><td></td></tr><tr><td>65</td><td>32</td><td>1.04</td><td>150,000</td><td>0.28</td></tr><tr><td>250</td><td>125</td><td>1.01</td><td>200,000</td><td>0.21</td></tr><tr><td>1,000</td><td>500</td><td>1.00</td><td>250,000</td><td>0.16</td></tr><tr><td>...</td><td>...</td><td></td><td>...</td><td></td></tr><tr><td>1,000,000</td><td>500,000</td><td>1.00</td><td>500,000</td><td>0.10</td></tr><tr><td>4,000,000</td><td>2,000,000</td><td>1.00</td><td>550,000</td><td>0.07</td></tr><tr><td>16,000,000</td><td>8,000,000</td><td>1.00</td><td>600,000</td><td>0.06</td></tr><tr><td>...</td><td>...</td><td></td><td>...</td><td></td></tr></tbody></table> <p>It is clearly seen that the first algorithm exhibits a linear order of growth indeed following the power rule. The empirical values for the second one are diminishing rapidly, suggesting it follows another rule of growth and in any case has much lower local orders of growth (and improving further still), empirically, than the first one. </p> <h3>Evaluating run-time complexity</h3> <p>The run-time complexity for the worst-case scenario of a given algorithm can sometimes be evaluated by examining the structure of the algorithm and making some simplifying assumptions. Consider the following <a href="/facts/Pseudocode/XgN19duK">pseudocode</a>: </p> 1 <i>get a positive integer n from input</i> 2 if n > 10 3 print "This might take a while..." 4 for i = 1 to n 5 for j = 1 to i 6 print i * j 7 print "Done!" <p>A given computer will take a <a href="/facts/DTIME/oyVAQfHk">discrete amount of time</a> to execute each of the <a href="/facts/Instruction_(computer_science)/8p8vwUeE">instructions</a> involved with carrying out this algorithm. Say that the actions carried out in step 1 are considered to consume time at most <i>T</i>1, step 2 uses time at most <i>T</i>2, and so forth. </p><p>In the algorithm above, steps 1, 2 and 7 will only be run once. For a worst-case evaluation, it should be assumed that step 3 will be run as well. Thus the total amount of time to run steps 1–3 and step 7 is: </p> T 1 + T 2 + T 3 + T 7 . {\displaystyle T_{1}+T_{2}+T_{3}+T_{7}.\,} <p>The <a href="/facts/Program_loop/2XTDfErC">loops</a> in steps 4, 5 and 6 are trickier to evaluate. The outer loop test in step 4 will execute ( <i>n</i> + 1 ) times,<a class="footnote-ref" id="fnref:10" href="#fn:10"><sup>10</sup></a> which will consume <i>T</i>4( <i>n</i> + 1 ) time. The inner loop, on the other hand, is governed by the value of j, which <a href="/facts/Iteration/AMc1p7BL">iterates</a> from 1 to <i>i</i>. On the first pass through the outer loop, j iterates from 1 to 1: The inner loop makes one pass, so running the inner loop body (step 6) consumes <i>T</i>6 time, and the inner loop test (step 5) consumes 2<i>T</i>5 time. During the next pass through the outer loop, j iterates from 1 to 2: the inner loop makes two passes, so running the inner loop body (step 6) consumes 2<i>T</i>6 time, and the inner loop test (step 5) consumes 3<i>T</i>5 time. </p><p>Altogether, the total time required to run the inner loop <i>body</i> can be expressed as an <a href="/facts/Arithmetic_progression/c4ew9Xoq">arithmetic progression</a>: </p> T 6 + 2 T 6 + 3 T 6 + ⋯ + ( n − 1 ) T 6 + n T 6 {\displaystyle T_{6}+2T_{6}+3T_{6}+\cdots +(n-1)T_{6}+nT_{6}} <p>which can be <a href="/facts/Factorization/tXrBXFh1">factored</a><a class="footnote-ref" id="fnref:11" href="#fn:11"><sup>11</sup></a> as </p> [ 1 + 2 + 3 + ⋯ + ( n − 1 ) + n ] T 6 = [ 1 2 ( n 2 + n ) ] T 6 {\displaystyle \left[1+2+3+\cdots +(n-1)+n\right]T_{6}=\left[{\frac {1}{2}}(n^{2}+n)\right]T_{6}} <p>The total time required to run the inner loop <i>test</i> can be evaluated similarly: </p> 2 T 5 + 3 T 5 + 4 T 5 + ⋯ + ( n − 1 ) T 5 + n T 5 + ( n + 1 ) T 5 = T 5 + 2 T 5 + 3 T 5 + 4 T 5 + ⋯ + ( n − 1 ) T 5 + n T 5 + ( n + 1 ) T 5 − T 5 {\displaystyle {\begin{aligned}&2T_{5}+3T_{5}+4T_{5}+\cdots +(n-1)T_{5}+nT_{5}+(n+1)T_{5}\\={}&T_{5}+2T_{5}+3T_{5}+4T_{5}+\cdots +(n-1)T_{5}+nT_{5}+(n+1)T_{5}-T_{5}\end{aligned}}} <p>which can be factored as </p> T 5 [ 1 + 2 + 3 + ⋯ + ( n − 1 ) + n + ( n + 1 ) ] − T 5 = [ 1 2 ( n 2 + n ) ] T 5 + ( n + 1 ) T 5 − T 5 = [ 1 2 ( n 2 + n ) ] T 5 + n T 5 = [ 1 2 ( n 2 + 3 n ) ] T 5 {\displaystyle {\begin{aligned}&T_{5}\left[1+2+3+\cdots +(n-1)+n+(n+1)\right]-T_{5}\\={}&\left[{\frac {1}{2}}(n^{2}+n)\right]T_{5}+(n+1)T_{5}-T_{5}\\={}&\left[{\frac {1}{2}}(n^{2}+n)\right]T_{5}+nT_{5}\\={}&\left[{\frac {1}{2}}(n^{2}+3n)\right]T_{5}\end{aligned}}} <p>Therefore, the total run-time for this algorithm is: </p> f ( n ) = T 1 + T 2 + T 3 + T 7 + ( n + 1 ) T 4 + [ 1 2 ( n 2 + n ) ] T 6 + [ 1 2 ( n 2 + 3 n ) ] T 5 {\displaystyle f(n)=T_{1}+T_{2}+T_{3}+T_{7}+(n+1)T_{4}+\left[{\frac {1}{2}}(n^{2}+n)\right]T_{6}+\left[{\frac {1}{2}}(n^{2}+3n)\right]T_{5}} <p>which reduces to </p> f ( n ) = [ 1 2 ( n 2 + n ) ] T 6 + [ 1 2 ( n 2 + 3 n ) ] T 5 + ( n + 1 ) T 4 + T 1 + T 2 + T 3 + T 7 {\displaystyle f(n)=\left[{\frac {1}{2}}(n^{2}+n)\right]T_{6}+\left[{\frac {1}{2}}(n^{2}+3n)\right]T_{5}+(n+1)T_{4}+T_{1}+T_{2}+T_{3}+T_{7}} <p>As a <a href="/facts/Rule-of-thumb/RopfGsJ0">rule-of-thumb</a>, one can assume that the highest-order term in any given function dominates its rate of growth and thus defines its run-time order. In this example, n2 is the highest-order term, so one can conclude that <i>f</i>(<i>n</i>) = <i>O</i>(<i>n</i>2). Formally this can be proven as follows: </p> <blockquote><p>Prove that [ 1 2 ( n 2 + n ) ] T 6 + [ 1 2 ( n 2 + 3 n ) ] T 5 + ( n + 1 ) T 4 + T 1 + T 2 + T 3 + T 7 ≤ c n 2 , n ≥ n 0 {\displaystyle \left[{\frac {1}{2}}(n^{2}+n)\right]T_{6}+\left[{\frac {1}{2}}(n^{2}+3n)\right]T_{5}+(n+1)T_{4}+T_{1}+T_{2}+T_{3}+T_{7}\leq cn^{2},\ n\geq n_{0}} </p> [ 1 2 ( n 2 + n ) ] T 6 + [ 1 2 ( n 2 + 3 n ) ] T 5 + ( n + 1 ) T 4 + T 1 + T 2 + T 3 + T 7 ≤ ( n 2 + n ) T 6 + ( n 2 + 3 n ) T 5 + ( n + 1 ) T 4 + T 1 + T 2 + T 3 + T 7 ( for n ≥ 0 ) {\displaystyle {\begin{aligned}&\left[{\frac {1}{2}}(n^{2}+n)\right]T_{6}+\left[{\frac {1}{2}}(n^{2}+3n)\right]T_{5}+(n+1)T_{4}+T_{1}+T_{2}+T_{3}+T_{7}\\\leq {}&(n^{2}+n)T_{6}+(n^{2}+3n)T_{5}+(n+1)T_{4}+T_{1}+T_{2}+T_{3}+T_{7}\ ({\text{for }}n\geq 0)\end{aligned}}} <p>Let <i>k</i> be a constant greater than or equal to [<i>T</i>1..<i>T</i>7] T 6 ( n 2 + n ) + T 5 ( n 2 + 3 n ) + ( n + 1 ) T 4 + T 1 + T 2 + T 3 + T 7 ≤ k ( n 2 + n ) + k ( n 2 + 3 n ) + k n + 5 k = 2 k n 2 + 5 k n + 5 k ≤ 2 k n 2 + 5 k n 2 + 5 k n 2 ( for n ≥ 1 ) = 12 k n 2 {\displaystyle {\begin{aligned}&T_{6}(n^{2}+n)+T_{5}(n^{2}+3n)+(n+1)T_{4}+T_{1}+T_{2}+T_{3}+T_{7}\leq k(n^{2}+n)+k(n^{2}+3n)+kn+5k\\={}&2kn^{2}+5kn+5k\leq 2kn^{2}+5kn^{2}+5kn^{2}\ ({\text{for }}n\geq 1)=12kn^{2}\end{aligned}}} Therefore [ 1 2 ( n 2 + n ) ] T 6 + [ 1 2 ( n 2 + 3 n ) ] T 5 + ( n + 1 ) T 4 + T 1 + T 2 + T 3 + T 7 ≤ c n 2 , n ≥ n 0 for c = 12 k , n 0 = 1 {\displaystyle \left[{\frac {1}{2}}(n^{2}+n)\right]T_{6}+\left[{\frac {1}{2}}(n^{2}+3n)\right]T_{5}+(n+1)T_{4}+T_{1}+T_{2}+T_{3}+T_{7}\leq cn^{2},n\geq n_{0}{\text{ for }}c=12k,n_{0}=1} </p> </blockquote> <p>A more <a href="/facts/Elegance/NrPbHNjc">elegant</a> approach to analyzing this algorithm would be to declare that [<i>T</i>1..<i>T</i>7] are all equal to one unit of time, in a system of units chosen so that one unit is greater than or equal to the actual times for these steps. This would mean that the algorithm's run-time breaks down as follows:<a class="footnote-ref" id="fnref:12" href="#fn:12"><sup>12</sup></a> </p> <blockquote><p> 4 + ∑ i = 1 n i ≤ 4 + ∑ i = 1 n n = 4 + n 2 ≤ 5 n 2 ( for n ≥ 1 ) = O ( n 2 ) . {\displaystyle 4+\sum _{i=1}^{n}i\leq 4+\sum _{i=1}^{n}n=4+n^{2}\leq 5n^{2}\ ({\text{for }}n\geq 1)=O(n^{2}).} </p></blockquote> <h3>Growth rate analysis of other resources</h3> <p>The methodology of run-time analysis can also be utilized for predicting other growth rates, such as consumption of <a href="/facts/DSPACE/co9rzRQd">memory space</a>. As an example, consider the following pseudocode which manages and reallocates memory usage by a program based on the size of a <a href="/facts/Computer_file/FjFH3Ufm">file</a> which that program manages: </p> while <i>file is still open:</i> let n = <i>size of file</i> for <i>every 100,000 <a href="/facts/Kilobyte/jIe6t4yv">kilobytes</a> of increase in file size</i> <i>double the amount of memory reserved</i> <p>In this instance, as the file size n increases, memory will be consumed at an <a href="/facts/Exponential_growth/m4yEqKJ7">exponential growth</a> rate, which is order <i>O</i>(2<i>n</i>). This is an extremely rapid and most likely unmanageable growth rate for consumption of memory <a href="/facts/Resource_(computer_science)/bX958SEx">resources</a>. </p> <h2 id="relevance">Relevance</h2> <p>Algorithm analysis is important in practice because the accidental or unintentional use of an inefficient algorithm can significantly impact system performance. In time-sensitive applications, an algorithm taking too long to run can render its results outdated or useless. An inefficient algorithm can also end up requiring an uneconomical amount of computing power or storage in order to run, again rendering it practically useless. </p> <h2 id="constant-factors">Constant factors</h2> <p>Analysis of algorithms typically focuses on the asymptotic performance, particularly at the elementary level, but in practical applications constant factors are important, and real-world data is in practice always limited in size. The limit is typically the size of addressable memory, so on 32-bit machines 232 = 4 GiB (greater if <a href="/facts/Segmented_memory/QCUGw5v5">segmented memory</a> is used) and on 64-bit machines 264 = 16 EiB. Thus given a limited size, an order of growth (time or space) can be replaced by a constant factor, and in this sense all practical algorithms are <i>O</i>(1) for a large enough constant, or for small enough data. </p><p>This interpretation is primarily useful for functions that grow extremely slowly: (binary) <a href="/facts/Iterated_logarithm/bEY9cfh8">iterated logarithm</a> (log*) is less than 5 for all practical data (265536 bits); (binary) log-log (log log <i>n</i>) is less than 6 for virtually all practical data (264 bits); and binary log (log <i>n</i>) is less than 64 for virtually all practical data (264 bits). An algorithm with non-constant complexity may nonetheless be more efficient than an algorithm with constant complexity on practical data if the overhead of the constant time algorithm results in a larger constant factor, e.g., one may have K > k log log n {\displaystyle K>k\log \log n} so long as K / k > 6 {\displaystyle K/k>6} and n < 2 2 6 = 2 64 {\displaystyle n<2^{2^{6}}=2^{64}} . </p><p>For large data linear or quadratic factors cannot be ignored, but for small data an asymptotically inefficient algorithm may be more efficient. This is particularly used in <a href="/facts/Hybrid_algorithm/mgL7jc0c">hybrid algorithms</a>, like <a href="/facts/Timsort/llcssLEF">Timsort</a>, which use an asymptotically efficient algorithm (here <a href="/facts/Merge_sort/A5jLYfzS">merge sort</a>, with time complexity n log n {\displaystyle n\log n} ), but switch to an asymptotically inefficient algorithm (here <a href="/facts/Insertion_sort/N5yWe8Co">insertion sort</a>, with time complexity n 2 {\displaystyle n^{2}} ) for small data, as the simpler algorithm is faster on small data. </p> <h2 id="see-also">See also</h2> <ul><li><a href="/facts/Amortized_analysis/9fFYenJz">Amortized analysis</a></li> <li><a href="/facts/Analysis_of_parallel_algorithms/GC8l28HW">Analysis of parallel algorithms</a></li> <li><a href="/facts/Asymptotic_computational_complexity/fMuA7mEP">Asymptotic computational complexity</a></li> <li><a href="/facts/Information-based_complexity/plCCweHD">Information-based complexity</a></li> <li><a href="/facts/Master_theorem_(analysis_of_algorithms)/hknsWQks">Master theorem (analysis of algorithms)</a></li> <li><a href="/facts/NP-complete/NFPv56DU">NP-complete</a></li> <li><a href="/facts/Numerical_analysis/QT9cYa9z">Numerical analysis</a></li> <li><a href="/facts/Polynomial_time/77T62gmf">Polynomial time</a></li> <li><a href="/facts/Program_optimization/nXz6RpRM">Program optimization</a></li> <li><a href="/facts/Scalability/jh7JomPa">Scalability</a></li> <li><a href="/facts/Smoothed_analysis/aRpTTGDF">Smoothed analysis</a></li> <li><a href="/facts/Termination_analysis/T740cWY7">Termination analysis</a> — the subproblem of checking whether a program will terminate at all</li></ul> <h2 id="notes">Notes</h2> <ul><li><a href="/facts/Robert_Sedgewick_(computer_scientist)/f1lY2MFL">Sedgewick, Robert</a>; <a href="/facts/Philippe_Flajolet/8C9NqUlD">Flajolet, Philippe</a> (2013). <i>An Introduction to the Analysis of Algorithms</i> (2nd ed.). Addison-Wesley. <a href="/facts/ISBN_(identifier)/15AdSPa9">ISBN</a> 978-0-321-90575-8.</li> <li>Greene, Daniel A.; <a href="/facts/Donald_Knuth/GozNzDM6">Knuth, Donald E.</a> (1982). <i>Mathematics for the Analysis of Algorithms</i> (Second ed.). Birkhäuser. <a href="/facts/ISBN_(identifier)/15AdSPa9">ISBN</a> 3-7643-3102-X.</li> <li><a href="/facts/Thomas_H._Cormen/qJ2Ys15T">Cormen, Thomas H.</a>; <a href="/facts/Charles_E._Leiserson/6Uf19apB">Leiserson, Charles E.</a>; <a href="/facts/Ronald_L._Rivest/nQsIcAKa">Rivest, Ronald L.</a> & <a href="/facts/Clifford_Stein/5ajkVbCm">Stein, Clifford</a> (2001). <i><a href="/facts/Introduction_to_Algorithms/o9n7KDsg">Introduction to Algorithms</a></i>. Chapter 1: Foundations (Second ed.). Cambridge, MA: MIT Press and McGraw-Hill. pp. 3–122. <a href="/facts/ISBN_(identifier)/15AdSPa9">ISBN</a> 0-262-03293-7.</li> <li><a href="/facts/Robert_Sedgewick_(computer_scientist)/f1lY2MFL">Sedgewick, Robert</a> (1998). <a href="https://archive.org/details/algorithmsinc00sedg"><i>Algorithms in C, Parts 1-4: Fundamentals, Data Structures, Sorting, Searching</i></a> (3rd ed.). Reading, MA: Addison-Wesley Professional. <a href="/facts/ISBN_(identifier)/15AdSPa9">ISBN</a> 978-0-201-31452-6.</li> <li><a href="/facts/Donald_Knuth/GozNzDM6">Knuth, Donald</a>. <i><a href="/facts/The_Art_of_Computer_Programming/MR1bMD9e">The Art of Computer Programming</a></i>. Addison-Wesley.</li> <li><a href="/facts/Oded_Goldreich/Ufn1bqW7">Goldreich, Oded</a> (2010). <i>Computational Complexity: A Conceptual Perspective</i>. <a href="/facts/Cambridge_University_Press/fgEBSSRq">Cambridge University Press</a>. <a href="/facts/ISBN_(identifier)/15AdSPa9">ISBN</a> 978-0-521-88473-0.</li></ul> <h2 id="external-links">External links</h2> <ul><li> Media related to Analysis of algorithms at Wikimedia Commons</li></ul> <h2 id="references">References</h2> <ol> <li id="fn:1"><p>"Knuth: Recent News". 28 August 2016. Archived from the original on 28 August 2016. <a href="https://web.archive.org/web/20160828152021/http://www-cs-faculty.stanford.edu/~uno/news.html" target="_blank">https://web.archive.org/web/20160828152021/http://www-cs-faculty.stanford.edu/~uno/news.html</a> <a href="#fnref:1" class="footnote-back-ref">↩</a></p></li> <li id="fn:2"><p>Cormen, Thomas H., ed. (2009). Introduction to algorithms (3rd ed.). Cambridge, Mass: MIT Press. pp. 44–52. ISBN 978-0-262-03384-8. OCLC 311310321. <a href="978-0-262-03384-8" target="_blank">978-0-262-03384-8</a> <a href="#fnref:2" class="footnote-back-ref">↩</a></p></li> <li id="fn:3"><p>Alfred V. Aho; John E. Hopcroft; Jeffrey D. Ullman (1974). The design and analysis of computer algorithms. Addison-Wesley Pub. Co. ISBN 9780201000290., section 1.3 <a href="9780201000290" target="_blank">9780201000290</a> <a href="#fnref:3" class="footnote-back-ref">↩</a></p></li> <li id="fn:4"><p>Juraj Hromkovič (2004). Theoretical computer science: introduction to Automata, computability, complexity, algorithmics, randomization, communication, and cryptography. Springer. pp. 177–178. ISBN 978-3-540-14015-3. <a href="978-3-540-14015-3" target="_blank">978-3-540-14015-3</a> <a href="#fnref:4" class="footnote-back-ref">↩</a></p></li> <li id="fn:5"><p>Giorgio Ausiello (1999). Complexity and approximation: combinatorial optimization problems and their approximability properties. Springer. pp. 3–8. ISBN 978-3-540-65431-5. <a href="978-3-540-65431-5" target="_blank">978-3-540-65431-5</a> <a href="#fnref:5" class="footnote-back-ref">↩</a></p></li> <li id="fn:6"><p>Wegener, Ingo (2005), Complexity theory: exploring the limits of efficient algorithms, Berlin, New York: Springer-Verlag, p. 20, ISBN 978-3-540-21045-0 <a href="978-3-540-21045-0" target="_blank">978-3-540-21045-0</a> <a href="#fnref:6" class="footnote-back-ref">↩</a></p></li> <li id="fn:7"><p>Robert Endre Tarjan (1983). Data structures and network algorithms. SIAM. pp. 3–7. ISBN 978-0-89871-187-5. <a href="978-0-89871-187-5" target="_blank">978-0-89871-187-5</a> <a href="#fnref:7" class="footnote-back-ref">↩</a></p></li> <li id="fn:8"><p>Examples of the price of abstraction?, cstheory.stackexchange.com <a href="https://cstheory.stackexchange.com/q/608" target="_blank">https://cstheory.stackexchange.com/q/608</a> <a href="#fnref:8" class="footnote-back-ref">↩</a></p></li> <li id="fn:9"><p>How To Avoid O-Abuse and Bribes Archived 2017-03-08 at the Wayback Machine, at the blog "Gödel's Lost Letter and P=NP" by R. J. Lipton, professor of Computer Science at Georgia Tech, recounting idea by Robert Sedgewick <a href="http://rjlipton.wordpress.com/2009/07/24/how-to-avoid-o-abuse-and-bribes/" target="_blank">http://rjlipton.wordpress.com/2009/07/24/how-to-avoid-o-abuse-and-bribes/</a> <a href="#fnref:9" class="footnote-back-ref">↩</a></p></li> <li id="fn:10"><p>an extra step is required to terminate the for loop, hence n + 1 and not n executions <a href="#fnref:10" class="footnote-back-ref">↩</a></p></li> <li id="fn:11"><p>It can be proven by induction that 1 + 2 + 3 + ⋯ + ( n − 1 ) + n = n ( n + 1 ) 2 {\displaystyle 1+2+3+\cdots +(n-1)+n={\frac {n(n+1)}{2}}} <a href="/wiki/Mathematical_induction" target="_blank">/wiki/Mathematical_induction</a> <a href="#fnref:11" class="footnote-back-ref">↩</a></p></li> <li id="fn:12"><p>This approach, unlike the above approach, neglects the constant time consumed by the loop tests which terminate their respective loops, but it is trivial to prove that such omission does not affect the final result <a href="/wiki/Trivial_(mathematics)" target="_blank">/wiki/Trivial_(mathematics)</a> <a href="#fnref:12" class="footnote-back-ref">↩</a></p></li> </ol>