DSA Notes
HubAlgorithm vs Data Structure — Key Differences, Definitions, and How They Work TogetherApplications of DSA — How Data Structures and Algorithms Power Real-World SoftwareTime and Space Complexity — A Complete BeginnerWhat is DSA? — A Complete BeginnerWhy Learn DSA? — Career Benefits, Real-World Impact, and Long-Term ValueAsymptotic Analysis — Complete Framework with Recurrence Relations and Master TheoremBest, Average, and Worst Case Analysis — A Practical Guide for Every AlgorithmBig O Notation — Complete Guide with Formal Definition, Rules, and ExamplesBig Omega Notation — Lower Bounds, Best Case, and Formal DefinitionBig Theta (Θ) Notation — Tight Bounds and Exact Growth RatesSpace Complexity — Memory Analysis, Auxiliary Space, and Time-Space TradeoffsTime Complexity Examples — 25 Worked Problems from Basic to AdvancedArray Interview Problems — 15 Classic Problems with Complete SolutionsArray Operations — Every Operation Explained with Code, Complexity, and Edge CasesArray Rotation — Algorithms, Techniques, and Interview ProblemsArray Searching — Linear Search, Binary Search, and Advanced TechniquesArrays — The Complete Introduction to the Most Fundamental Data StructureDynamic Arrays — How Python Lists and ArrayLists Work InternallyMultidimensional Arrays — 2D Arrays, Matrices, and Memory LayoutSliding Window Technique — Complete Guide with 10 Classic ProblemsAnagram Problems — Detection, Grouping, and Sliding Window VariationsPalindrome Problems — Check, Longest, Count, and Expand Around CenterPattern Searching — KMP, Rabin-Karp, and Boyer-Moore AlgorithmsString Interview Problems — 10 Classic Problems with Complete SolutionsString Matching — Longest Common Subsequence, Edit Distance, and Z-AlgorithmString Operations — Complete Reference with Complexity and ImplementationStrings — Complete Introduction to String Data Structure and OperationsCircular Linked List — Structure, Operations, and ApplicationsDoubly Linked List — Complete Implementation with Bidirectional OperationsLinked List Interview Problems — 12 Classic Problems with Complete SolutionsLinked Lists — A Complete Introduction with Memory Model and TypesLinked List Operations — Every Technique with Code and ComplexityReverse Linked List — All Approaches from Basic to AdvancedSingly Linked List — Complete Implementation with All OperationsBalanced Parentheses — Stack-Based Validation with All VariantsInfix, Prefix, and Postfix Expressions — Stack-Based Conversion and EvaluationStack Implementation — Three Complete Implementations with Edge CasesStack Interview Problems — 10 Classic Problems with Complete SolutionsStack — Complete Introduction to LIFO Data StructureStack Using Arrays — Memory Layout, Capacity Management, and PerformanceStack Using Linked List — Implementation, Analysis, and When to Use ItCircular Queue — The Ring Buffer Explained with ImplementationDeque — Double-Ended Queue with Complete Implementation and ApplicationsPriority Queue — Complete Guide from Concept to ImplementationQueue Applications — BFS, Level-Order Traversal, 01-BFS, and MoreQueue Implementation — Array, Linked List, and Two-Stack ApproachesQueue Interview Problems — 10 Classic Problems with Complete SolutionsQueue — Complete Introduction to the FIFO Data StructureAVL Tree - Self-Balancing Binary Search TreeBinary Search Tree (BST)Binary Tree - Structure and OperationsFenwick Tree (Binary Indexed Tree)Red-Black Tree - Another Balanced BSTSegment Tree - Range Query Data StructureTree Interview Problems CollectionTree Data Structure FundamentalsTree Traversals - Inorder, Preorder, Postorder, Level OrderHeap — Complete Introduction to the Priority-Based Tree Data StructureHeap Operations — Complete Reference with Complexity AnalysisHeap Problems — 8 Classic Interview Problems with Complete SolutionsMax Heap — Complete Implementation and ApplicationsMin Heap — Complete Implementation with All OperationsPriority Queue Using Heap — Implementation, Applications, and PatternsBellman-Ford Algorithm — Shortest Path with Negative Weights and Cycle DetectionBFS Algorithm — Breadth-First Search with All Applications and VariantsDFS Algorithm — Depth-First Search with Cycle Detection and All ApplicationsDijkstraFloyd-Warshall Algorithm — All-Pairs Shortest Paths with Complete AnalysisGraph Interview Problems — 12 Classic Problems with Complete SolutionsGraph — Complete Introduction to the Most Powerful Data StructureGraph Representation — Adjacency List, Matrix, Edge List with Complete AnalysisKruskalMinimum Spanning Tree — Definition, Properties, and ApplicationsPrimShortest Path Algorithms — Complete Comparison and Selection GuideTopological Sorting — DFS and BFS (KahnUnion-Find (DSU) in Graphs — Connected Components and Dynamic ConnectivitySeparate Chaining — Complete Implementation and AnalysisCollision Handling — Chaining vs Open Addressing Complete ComparisonHash Functions — Design, Properties, and Common ImplementationsHashing — Complete Introduction to Hash Tables and Hash FunctionsHashing Problems — 10 Classic Interview Problems with Complete SolutionsOpen Addressing — Linear Probing, Quadratic Probing, and Double HashingBubble Sort — Complete Analysis with Optimizations and When to Use ItCounting Sort — O(n+k) Sorting That Breaks the Comparison Lower BoundHeap Sort — Guaranteed O(n log n) with O(1) SpaceInsertion Sort — The O(n) Best Case Sort That Powers PythonMerge Sort — Complete Guide to the Guaranteed O(n log n) Stable SortQuick Sort — The Fastest Practical Sorting Algorithm with Complete AnalysisRadix Sort — O(nk) Digit-by-Digit Sorting for Large Integer RangesSelection Sort — Complete Analysis with Swaps Minimization PropertySorting Algorithm Comparison — The Complete Decision GuideSorting — Complete Introduction to Sorting AlgorithmsBinary Search — Complete Guide with All Variants and Proof of CorrectnessInterpolation Search — O(log log n) Search for Uniformly Distributed DataLinear Search — Complete Guide with All Variants and ApplicationsSearch Algorithm Comparison — Choosing the Right Search for Every SituationSearching Interview Problems — 10 Classic Problems with Complete SolutionsTernary Search — Finding Maximum/Minimum of Unimodal FunctionsBacktracking Introduction — The Recursive Exploration with UndoRecursion Basics — The Complete BeginnerRecursion Problems — 12 Classic Problems from Basic to AdvancedRecursion Tree — Visualizing Recursion for Complexity AnalysisRecursion vs Iteration — When to Use Each and How to ConvertTail Recursion — Definition, Optimization, and Converting to IterationDP Interview Problems — 12 Classic Problems with Pattern Recognition GuideDynamic Programming — Complete Introduction with Properties and Problem IdentificationFibonacci DP — From Exponential to Linear with Memoization and Matrix ExponentiationKnapsack Problem — 0/1, Unbounded, and Fractional with Complete AnalysisLongest Common Subsequence — Complete Guide with DP Table and String VariantsLongest Increasing Subsequence — O(n log n) with Patience SortingMatrix Chain Multiplication — Interval DP with Complete AnalysisMemoization — Top-Down Dynamic Programming with CachingTabulation — Bottom-Up Dynamic Programming with Complete Table BuildingActivity Selection Problem — Greedy Scheduling with Proof of CorrectnessFractional Knapsack — Greedy by Value Density with ProofGreedy Interview Problems — 10 Classic Problems with Complete SolutionsGreedy Algorithms — Complete Introduction with Proof TechniquesHuffman Coding — Optimal Prefix-Free Data CompressionJob Sequencing Problem — Greedy Scheduling with DeadlinesBacktracking — The Complete Guide to Recursive Exploration with UndoBacktracking Problems — 12 Classic Interview Problems with Complete SolutionsN-Queens Problem — Complete Solution with Backtracking and OptimizationRat in a Maze — Backtracking Path Finding with All SolutionsSubset Generation — All Subsets, Combinations, and Partitions with BacktrackingSudoku Solver — Backtracking with Constraint PropagationCodeforces Guide — Complete Strategy to Reach 1800+ RatingCommon CP Techniques — Essential Algorithms Every Competitive Programmer Must KnowContest Strategies — How to Perform Your Best in Competitive Programming ContestsCompetitive Programming — Complete Introduction for BeginnersFast Input/Output — Why I/O Speed Matters and Complete Templates for C++ and PythonLeetCode Guide — Complete Strategy to Crack FAANG InterviewsBit Manipulation — Complete Guide with All Tricks and Interview ProblemsBloom Filter — Space-Efficient Probabilistic Membership TestingComputational Geometry — Points, Lines, Convex Hull, and Geometric AlgorithmsDisjoint Set Union (DSU) — Union-Find with Path Compression and Union by RankRandomized Algorithms — Las Vegas, Monte Carlo, and Probabilistic TechniquesSkip List — The Probabilistic Data Structure that Competes with Balanced BSTsSuffix Array — Construction, LCP Array, and String Search ApplicationsTrie Data Structure — Complete Guide with Implementation and ApplicationsAutocomplete System Using TrieLibrary Management System Using DSARoute Finder Using GraphsMini Search Engine ProjectSocial Network Analysis Using GraphsEssential Coding Patterns for InterviewsDSA Cheatsheet: Quick ReferenceEssential DSA Interview QuestionsFAANG DSA Sheet: Company-Specific Patterns and ProblemsProblem Solving Strategies for Coding InterviewsComplexity CheatsheetImportant Algorithms SummaryBest Practice Platforms for DSARecommended Books for DSA
Master randomized algorithms — the difference between Las Vegas and Monte Carlo algorithms, randomized quicksort with expected O(n log n), reservoir sampling for streaming data, randomized selection for kth element, Miller-Rabin primality test, and why randomization defeats adversarial inputs.
What Are Randomized Algorithms?
A randomized algorithm is an algorithm that makes random choices during its execution. Instead of following a fixed deterministic path, it uses random numbers to guide decisions. This randomness provides powerful benefits:
- Break adversarial patterns: An attacker cannot craft a worst-case input because the algorithm's behavior is unpredictable
- Simplify algorithms: Complex deterministic algorithms often have elegant randomized equivalents
- Expected efficiency: Many problems that are hard to solve deterministically in O(n log n) have simple randomized solutions with the same expected complexity
Randomized QuickSort — Las Vegas Algorithm
Why Randomize?
Standard QuickSort chooses a fixed pivot (e.g., last element). An adversary who knows this can create sorted or reverse-sorted input that causes O(n²) performance every time.
Randomized QuickSort picks a random pivot, making adversarial inputs impossible to construct.
Example
import random
def randomized_quicksort(arr, low=None, high=None):
"""
Randomized QuickSort — Las Vegas algorithm.
Expected O(n log n) for ANY input, not just random inputs.
No adversary can force O(n²) because pivot is random.
"""
if low is None:
low, high = 0, len(arr) - 1
if low < high:
# Random pivot selection — KEY CHANGE from standard quicksort
pivot_idx = random.randint(low, high)
arr[pivot_idx], arr[high] = arr[high], arr[pivot_idx] # Move to end
pivot_pos = partition(arr, low, high)
randomized_quicksort(arr, low, pivot_pos - 1)
randomized_quicksort(arr, pivot_pos + 1, high)
return arr
def partition(arr, low, high):
pivot = arr[high]
i = low - 1
for j in range(low, high):
if arr[j] <= pivot:
i += 1
arr[i], arr[j] = arr[j], arr[i]
arr[i+1], arr[high] = arr[high], arr[i+1]
return i + 1
# Test on adversarial input (sorted — worst case for deterministic quicksort)
arr = list(range(1000)) # Already sorted — O(n²) for fixed pivot!
result = randomized_quicksort(arr.copy())
print(result[:5], "...", result[-5:]) # Works in O(n log n) expected
Expected Complexity Proof (Informal)
A random pivot splits the array into left and right portions. Consider "good splits": where each portion has between n/4 and 3n/4 elements.
- A random pivot gives a good split with probability ≥ 1/2
- With a good split, we make at least 1/4 progress per level
- Expected levels needed: O(log n)
- Total expected work: O(n log n)
Randomized Selection (QuickSelect) — Las Vegas
Problem: Find the kth smallest element in an unsorted array.
Naive approach: Sort then index — O(n log n) QuickSelect: O(n) expected, O(n²) worst case, but randomization makes worst case astronomically unlikely.
Example
def quickselect(arr, k):
"""
Find kth smallest element (0-indexed k).
Expected O(n) time. Las Vegas — always returns correct answer.
"""
if len(arr) == 1:
return arr[0]
# Random pivot
pivot = random.choice(arr)
# Partition into three groups
lower = [x for x in arr if x < pivot]
equal = [x for x in arr if x == pivot]
upper = [x for x in arr if x > pivot]
if k < len(lower):
return quickselect(lower, k)
elif k < len(lower) + len(equal):
return pivot # kth element is the pivot itself
else:
return quickselect(upper, k - len(lower) - len(equal))
arr = [3, 1, 4, 1, 5, 9, 2, 6, 5, 3, 5]
print(quickselect(arr, 0)) # 1 (minimum)
print(quickselect(arr, 5)) # 4 (median-ish)
print(quickselect(arr, 10)) # 9 (maximum)
Expected O(n) proof: On average, a random pivot eliminates at least half the elements. Total work: n + n/2 + n/4 + ... = 2n = O(n).
Reservoir Sampling — Uniformly Sample k Items from a Stream
Problem: Given a stream of unknown length n, select k items uniformly at random. You cannot store all n items (n might be huge or unknown).
Example
import random
def reservoir_sampling(stream, k):
"""
Select k items from stream uniformly at random.
Each item has exactly k/n probability of being selected.
Time: O(n), Space: O(k)
"""
reservoir = []
for i, item in enumerate(stream):
if i < k:
reservoir.append(item) # Fill reservoir with first k items
else:
# Random index: include new item with probability k/(i+1)
j = random.randint(0, i)
if j < k:
reservoir[j] = item # Replace random existing item
return reservoir
# Test: select 3 from 1000 items — each should have 3/1000 = 0.3% chance
stream = range(1000)
sample = reservoir_sampling(stream, 3)
print(f"Sample of 3 from 1000: {sample}")
# Verify uniformity with many trials:
from collections import Counter
selections = Counter()
for _ in range(100000):
sample = reservoir_sampling(range(10), 3)
for item in sample: selections[item] += 1
print({k: v/100000 for k, v in sorted(selections.items())})
# Should be approximately {0: 0.3, 1: 0.3, ..., 9: 0.3}
Why it works: After seeing i+1 elements, each element has k/(i+1) probability of being in the reservoir. Mathematical induction proves this invariant holds at every step.
Applications: Online analytics, A/B testing, sampling social media feeds, distributed computing where you can't hold all data.
Miller-Rabin Primality Test — Monte Carlo
Problem: Is the number n prime?
Naive: Trial division up to √n — O(√n) — too slow for large n (cryptographic primes are 2048 bits = hundreds of digits).
Miller-Rabin: Probabilistic test — O(k log² n) for k rounds. After k rounds, probability of a composite being declared prime is at most 4^(-k).
Example
def miller_rabin(n, k=10):
"""
Monte Carlo primality test.
k rounds: probability of error ≤ 4^(-k)
k=10: error probability < 1 in 1 million
k=40: error probability < 1 in 10^24
Returns:
True → n is PROBABLY prime (composite with probability ≤ 4^(-k))
False → n is DEFINITELY composite
"""
if n < 2: return False
if n == 2 or n == 3: return True
if n % 2 == 0: return False
# Write n-1 as 2^r * d (factor out powers of 2)
r, d = 0, n - 1
while d % 2 == 0:
r += 1
d //= 2
# k rounds of testing
for _ in range(k):
a = random.randrange(2, n - 1) # Random witness
x = pow(a, d, n) # a^d mod n
if x == 1 or x == n - 1:
continue # Passed this round
for _ in range(r - 1):
x = pow(x, 2, n) # Square x
if x == n - 1:
break
else:
return False # n is composite
return True # n is probably prime
# Test
primes = [2, 3, 5, 7, 11, 13, 17, 19, 23, 29, 97, 997, 9973]
composites = [4, 9, 15, 25, 100, 1000, 9999]
print("Primes:", [miller_rabin(p) for p in primes])
# [True, True, True, True, True, True, True, True, True, True, True, True, True]
print("Composites:", [miller_rabin(c) for c in composites])
# [False, False, False, False, False, False, False]
# Large prime (used in RSA)
large_prime = 2**127 - 1 # Mersenne prime
print(f"2^127 - 1 is prime: {miller_rabin(large_prime)}") # True
Random Walk Algorithms
Example
def random_walk_2d(steps):
"""
Random walk on a 2D grid.
Expected distance from origin after n steps: O(√n)
"""
x, y = 0, 0
directions = [(0,1),(0,-1),(1,0),(-1,0)]
for _ in range(steps):
dx, dy = random.choice(directions)
x += dx
y += dy
distance = (x**2 + y**2) ** 0.5
return x, y, distance
x, y, dist = random_walk_2d(10000)
print(f"Final position: ({x}, {y}), distance from origin: {dist:.1f}")
# Expected ~100 (√10000 = 100)
Applications: Search algorithms, physics simulations, financial modeling, network analysis.
Randomized Hashing to Prevent DoS
Example
import hashlib
import os
class SecureHashMap:
"""
Hash map with random salt to prevent hash collision attacks.
Without randomization: adversary can craft inputs causing O(n²) lookups.
"""
def __init__(self):
self.salt = os.urandom(16) # Random secret salt
self._data = {}
def _hash(self, key: str) -> str:
salted = self.salt + key.encode()
return hashlib.sha256(salted).hexdigest()
def set(self, key: str, value) -> None:
h = self._hash(key)
self._data[h] = (key, value)
def get(self, key: str):
h = self._hash(key)
result = self._data.get(h)
return result[1] if result else None
Python's built-in dict uses hash randomization (PYTHONHASHSEED) for exactly this reason — to prevent adversarial inputs from degrading hash map performance.
When to Use Randomized Algorithms
USE randomized algorithms when
✅ Adversarial inputs are possible (use randomized quicksort)
✅ Streaming data of unknown/large size (reservoir sampling)
✅ Exact computation is too slow (Miller-Rabin vs trial division)
✅ Approximate answers are acceptable (Monte Carlo integration)
✅ Deterministic alternatives are extremely complex (skip lists)
DON'T use when
❌ Output must be deterministically correct every time
❌ Reproducing exact results is required (debugging, testing)
❌ Worst-case guarantees are mandatory (use deterministic alternatives)
Summary
| Algorithm | Type | Expected Time | Correctness |
|---|---|---|---|
| Randomized QuickSort | Las Vegas | O(n log n) | Always correct |
| QuickSelect | Las Vegas | O(n) | Always correct |
| Reservoir Sampling | Las Vegas | O(n) | Always correct |
| Miller-Rabin (k rounds) | Monte Carlo | O(k log² n) | Error ≤ 4^(-k) |
| Random Walk | — | O(n) | — |
Randomized algorithms are a fundamental tool in a programmer's toolkit. QuickSelect, randomized hashing, and reservoir sampling appear in production systems every day. Understanding the distinction between Las Vegas (always correct, random time) and Monte Carlo (bounded time, probabilistic correctness) is essential for choosing the right approach.
*Next: Computational Geometry*
Continue the lesson path
Study next in this module
01Advanced TopicsDisjoint Set Union (DSU) — Union-Find with Path Compression and Union by RankMaster Disjoint Set Union (Union-Find) completely — what it is, naive implementation, path compression optimization, union by rank, combined O(α(n)) amortized complexity, Kruskal02Advanced TopicsSkip List — The Probabilistic Data Structure that Competes with Balanced BSTsMaster skip lists — what they are, how probabilistic layering achieves O(log n) expected search, complete implementation with insert/search/delete, why Redis and LevelDB use skip lists instead of balanced trees, and comparison with AVL trees and hash maps.03Advanced TopicsComputational Geometry — Points, Lines, Convex Hull, and Geometric AlgorithmsMaster computational geometry fundamentals — cross product for orientation testing, point-in-polygon detection, convex hull algorithms (Graham scan and Jarvis march), closest pair of points, line segment intersection, and applications in graphics and competitive programming.04Advanced TopicsSuffix Array — Construction, LCP Array, and String Search ApplicationsMaster suffix arrays — what a suffix array is, how to build it using simple and O(n log n) algorithms, the LCP (Longest Common Prefix) array, how to use suffix arrays for pattern searching in O(m log n), finding the longest repeated substring, and counting distinct substrings.