Algorithms & Data Structures
1. Complexity Analysis
Efficiency of an algorithm, measured by 2 elements:
Time complexity - amount of time it takes for a given algorithm to execute
Space complexity - mount of space a given algorithm uses at most during
execution
Empirical time analysis
Analysis method for time complexity - roughly equivalent to a brute force approach
when programming or a proof by exhaustion for Reasoning and Logic: execute and
measure the time:
long startTime = System.currentTimeMillis(); // record the
starting time
/* (run the algorithm) */
long endTime = System.currentTimeMillis(); // record the ending
time
long elapsed = endTime - startTime; // calculate the
elapsed time
Problems:
Results may differ when different hardware / compiler / OS / etc. are used
Experiments are restricted to a limited set of inputs
Requires a full implementation of the algorithm
A compiler may optimize your code or require a warm up time.
Theoretical complexity analysis for time
Way to analyse the efficiency of an algorithm without having to run additional code,
don’t need to execute your algorithm to analyse it.
Primitive operations are operations that have a run time of 1:
- Assigning a value to a variable
- Performing an arithmetic operation
- Comparing two numbers (and no more than two)
- Accessing a single element of an array by index
- Calling and returning from a method
Assign a mathematical function to the algorithm that describes the running time.
T algorithm ( n )=number of primitive operations performed for an input size n
,To set up T algorithm ( n ):
1. State the size of the problem n - length of an array.
2. Simplify the program to only using primitive operations (optional).
3. Express the running time by counting operations.
! Note: when expressing the running time, we consider the worst-case of our problem
n.
! Note: a + only represents an arithmetic operation when applied to numbers.
Theoretical complexity analysis for space
Space used can go down, while time cannot. Count the number of stack frames
used.
Salgorithm ( n )=maximum amount of memory needed at any point in the algorithm for an input size n
A stack frame is added or pushed onto the stack when a method is called and is only
removed or popped off the stack when you return from the method. With recursive
method it is possible to have multiple stack frames on the stack at once.
Big O Proof
f ( n ) is O ( g ( n ) ) if f ( n ) ≤ c∗g ( n ) , c ≥ 0 , n0 ≥n
2. Asymptotic growth comparison
The following functions are from ‘best’ asymptotic growth to ’worst’:
1. Constant: f ( n )=c
2. Logarithmic: f ( n )=log (n)
3. Linear: f ( n )=n
4. Linearithmic (n-log-n): f ( n )=n log (n)
5. Quadratic: f ( n )=n2
6. Cubic: f ( n )=n3
7. Polynomials: f ( n )=n a
8. Factorial: f ( n )=n !
9. Exponential: f ( n )=e n
10. Worst ever: f ( n )=n n
, 3. Often used tricks
Gauss's sum identity:
n
n(n+1)
∑ i=1+ 2+ 3+…+n= 2
i=1
Geometric sum:
a1 ( 1−r )
m m
∑ ai=1+a+ a2 +…+ an= 1−r
, r ≠1
i=0
where:
o m is the number of terms, in this case n
o a 1 is the first term, in this case 1
o r is the constant that each term is multiplied by to get the next
term, in this case a
Particular geometric series:
n
∑ 2i =2n+1−1
i=0
4. Finding time complexity of a recursive algorithm
1. State / define the size of the input n
2. State the recurrence equation by counting operations. The exact amount of
operations shouldn’t be counted, that depends on the hardware, compiler, etc.
It is more about counting loops, etc. Specific operations can be replaced by a
constant, often called c xwhere x is a number
3. Create a form of this recurrence equation where there is no T(n) in the
equation anymore. There are 2 methods to do so:
a. Making an educated guess and then proving this (by induction)
b. Unfolding the equation and then create a final equation based on that
4. Determine the complexity in Big O notation
5. Prove the complexity in Big O notation
5. Arrays
1. Complexity Analysis
Efficiency of an algorithm, measured by 2 elements:
Time complexity - amount of time it takes for a given algorithm to execute
Space complexity - mount of space a given algorithm uses at most during
execution
Empirical time analysis
Analysis method for time complexity - roughly equivalent to a brute force approach
when programming or a proof by exhaustion for Reasoning and Logic: execute and
measure the time:
long startTime = System.currentTimeMillis(); // record the
starting time
/* (run the algorithm) */
long endTime = System.currentTimeMillis(); // record the ending
time
long elapsed = endTime - startTime; // calculate the
elapsed time
Problems:
Results may differ when different hardware / compiler / OS / etc. are used
Experiments are restricted to a limited set of inputs
Requires a full implementation of the algorithm
A compiler may optimize your code or require a warm up time.
Theoretical complexity analysis for time
Way to analyse the efficiency of an algorithm without having to run additional code,
don’t need to execute your algorithm to analyse it.
Primitive operations are operations that have a run time of 1:
- Assigning a value to a variable
- Performing an arithmetic operation
- Comparing two numbers (and no more than two)
- Accessing a single element of an array by index
- Calling and returning from a method
Assign a mathematical function to the algorithm that describes the running time.
T algorithm ( n )=number of primitive operations performed for an input size n
,To set up T algorithm ( n ):
1. State the size of the problem n - length of an array.
2. Simplify the program to only using primitive operations (optional).
3. Express the running time by counting operations.
! Note: when expressing the running time, we consider the worst-case of our problem
n.
! Note: a + only represents an arithmetic operation when applied to numbers.
Theoretical complexity analysis for space
Space used can go down, while time cannot. Count the number of stack frames
used.
Salgorithm ( n )=maximum amount of memory needed at any point in the algorithm for an input size n
A stack frame is added or pushed onto the stack when a method is called and is only
removed or popped off the stack when you return from the method. With recursive
method it is possible to have multiple stack frames on the stack at once.
Big O Proof
f ( n ) is O ( g ( n ) ) if f ( n ) ≤ c∗g ( n ) , c ≥ 0 , n0 ≥n
2. Asymptotic growth comparison
The following functions are from ‘best’ asymptotic growth to ’worst’:
1. Constant: f ( n )=c
2. Logarithmic: f ( n )=log (n)
3. Linear: f ( n )=n
4. Linearithmic (n-log-n): f ( n )=n log (n)
5. Quadratic: f ( n )=n2
6. Cubic: f ( n )=n3
7. Polynomials: f ( n )=n a
8. Factorial: f ( n )=n !
9. Exponential: f ( n )=e n
10. Worst ever: f ( n )=n n
, 3. Often used tricks
Gauss's sum identity:
n
n(n+1)
∑ i=1+ 2+ 3+…+n= 2
i=1
Geometric sum:
a1 ( 1−r )
m m
∑ ai=1+a+ a2 +…+ an= 1−r
, r ≠1
i=0
where:
o m is the number of terms, in this case n
o a 1 is the first term, in this case 1
o r is the constant that each term is multiplied by to get the next
term, in this case a
Particular geometric series:
n
∑ 2i =2n+1−1
i=0
4. Finding time complexity of a recursive algorithm
1. State / define the size of the input n
2. State the recurrence equation by counting operations. The exact amount of
operations shouldn’t be counted, that depends on the hardware, compiler, etc.
It is more about counting loops, etc. Specific operations can be replaced by a
constant, often called c xwhere x is a number
3. Create a form of this recurrence equation where there is no T(n) in the
equation anymore. There are 2 methods to do so:
a. Making an educated guess and then proving this (by induction)
b. Unfolding the equation and then create a final equation based on that
4. Determine the complexity in Big O notation
5. Prove the complexity in Big O notation
5. Arrays
