Search Tech Journey

Find topics, journeys and posts

6-month learning plan23 / 130
back to blog
pythonbeginner 50m read

S023 · Arrays & Strings — Indexing, Slicing, Two-Pointer

The two data structures every algorithm question secretly starts from — and the two-pointer template that solves half of them in O(n).

🧩DSAM03 · Data Structures & Algorithms· Session 023 of 130 90 min

🎯 Own arrays and strings so completely that you reach for the two-pointer / sliding-window template before you reach for a for-loop.

Why this session exists

Arrays are the substrate of computing. Strings are arrays of characters. Together they show up in every single interview, every hot code path in every backend, and every log-parsing pipeline you'll ever write. This session isn't about the syntax (you already know a[3]) — it's about the two-pointer and sliding-window patterns that turn painful O(n²) loops into elegant O(n) solutions. Once you see these patterns, you can't un-see them.

You will be able to
  • Explain in one breath why arrays give O(1) index access and O(n) middle-insert.
  • Write a two-pointer solution for reverse-in-place, two-sum-on-sorted, and remove-duplicates.
  • Write a sliding-window solution for longest-substring-without-repeats in one sitting.
  • Explain Python string immutability and why ''.join(list) beats + in a loop.
  • Recognise which real production bug (log parser, tokeniser, path normaliser) is secretly one of these patterns.

Prerequisites

  • S005–S007 · Python data types (list, str, tuple).
  • S021 · Probability (you'll compute expected O(n)/O(n²) counts).
  • No formal Big-O background required — we'll build it here.


(a) Intuition · 5 min

Arrays are shelves; strings are shelves of letters
🌍 Real world

An array is a row of numbered lockers. Locker #7 is exactly 7 lockers from the start — you can walk there in one step. Adding a new locker at #7 means shifting every locker after it one to the right — a lot of moving.

A two-pointer approach is two people walking down the row from opposite ends, meeting in the middle. They together examine every locker exactly once — O(n) — but never twice.

💻 Code world

Under the hood, a Python list is a contiguous block of pointer slots in memory. a[i] is arithmetic: base_address + i * pointer_size — one memory read, O(1).

Two-pointer and sliding-window turn many O(n²) brute-force loops into O(n) by exploiting the fact that when a pointer moves forward, some prior work can be reused. That single realisation is the entire pattern.

The three ideas that unlock this whole family

What you're really learning
  • Random access — arrays give O(1) index because memory is a giant array under the hood. Everything else in DSA is a trade-off vs this.
  • Two-pointer — instead of nested for-loops, walk two indices along the array. Each pointer moves at most n times → O(n) total.
  • Sliding window — a special two-pointer where left and right bound a subarray whose property (sum, distinct chars, min/max) is maintained incrementally.

Timeline — how arrays became the default

  1. 1957
    FORTRAN arrays
    The first mainstream language ships with 1-D and multi-D array types built in. Scientific computing takes off.
  2. 1972
    C arrays + pointer arithmetic
    C exposes arrays as raw pointers — a[i] is literally *(a+i). Fast, unsafe, foundational.
  3. 1991
    Python list
    Guido chooses ‘dynamic array of pointers’ over ‘linked list’ as the default sequence. O(1) index for everyone.
  4. 2006
    NumPy ndarray
    Travis Oliphant merges Numeric + Numarray. Contiguous typed arrays give Python C-speed math and become the base of pandas, PyTorch, scikit-learn.
  5. 2015
    SIMD in browsers
    WebAssembly + typed arrays let browsers process pixel arrays at near-native speed. Everything is arrays.

(b) Visual walkthrough · 15 min

Array vs linked list — the trade-off in one diagram

The two-pointer pattern — three shapes

1shape 1
Opposite ends

left=0, right=n-1. Move whichever pointer is at the ‘wrong’ end. Use for: reverse in place, two-sum on sorted array, valid palindrome.

2shape 2
Same start, one lagging

slow=0, fast=0. Advance fast; when a condition is met, bump slow and swap. Use for: remove-duplicates-in-place, move zeros to end.

3shape 3
Sliding window

left=0, right=0. Grow right until the window becomes invalid, then shrink left until valid again. Use for: longest substring without repeats, minimum-size-subarray-sum, max sum of k consecutive.

When to reach for which

Two-pointer (opposite)

Sorted array + pair condition

  • Palindrome check
  • Two-sum on sorted
  • Trapping rain water
  • Container with most water
Two-pointer (same start)

In-place mutation without extra memory

  • Remove duplicates from sorted array
  • Move zeros to end
  • Partition around a pivot
  • Any ‘compact this array’ task
Sliding window

Contiguous subarray / substring property

  • Longest substring without repeating chars
  • Minimum window substring
  • Max sum of k consecutive
  • Longest subarray with at most k distinct
Not two-pointer

Bail to a different tool

  • Unsorted + no way to sort → hashmap (S024)
  • Need pairs from k > 2 → recursion (S027) / DP (S033)
  • Need arbitrary index jumps → probably a graph (S030)
  • Need to modify while iterating → build a new list; iteration + mutation = pain

The sliding-window state machine

The 5 states of a sliding window

1. Initialise
left = right = 0; state (e.g. counter dict, running sum) is empty.
start
2. Grow right
Advance right by 1. Update state to include arr[right].
grow
3. Check invariant
If the window is now INVALID (e.g. duplicate char, sum too big), enter shrink mode.
check
4. Shrink left
While invalid: remove arr[left] from state, advance left. Stop when valid again.
shrink
5. Record answer
Update best (max length, min length, count) using right - left + 1. Then loop back to step 2 until right == n.
record

Common misconception
✗ What most people think

"Building a string in a loop with s += part is fine — it's one operation per iteration, so it's linear. And slicing a string or list to grab a piece is cheap because I'm only taking a few characters."

✓ What is actually true

Both are hidden O(n) operations inside your loop. Strings are immutable, so s += part allocates a new string and copies everything accumulated so far — making the loop O(n²). And a slice copies the elements it selects, so slicing in a loop is quadratic too.

Why the myth is so sticky

Because the syntax is a single short operator that looks atomic, and because CPython has an optimisation that sometimes resizes a string in place when there is exactly one reference to it — so a tight microbenchmark of s += x can look linear and mislead you completely. That optimisation vanishes the moment another reference exists, the moment you're on a different implementation, or the moment the pattern is slightly more complex. The result is code that benchmarks fine on 10,000 rows and takes hours on 10 million, with a profile that points at a line containing nothing but a +=. The same illusion applies to list.pop(0) and list.insert(0, x): one short call, O(n) of shifting underneath.

Prove it to yourself

Quadratic hiding inside a one-character operator:

import timeit
setup = 'parts = ["x"] * 40000'

bad  = 's = ""\nfor p in parts: s += p'
good = 's = "".join(parts)'
print(timeit.timeit(bad,  setup, number=5))
print(timeit.timeit(good, setup, number=5))
# now double 40000 -> join doubles, the loop roughly quadruples
From first principles
Start with the question

Why does the two-pointer technique work — and why is it valid only when the array is sorted? It looks like a trick; derive why discarding half the search space at each step is provably safe.

  1. 1
    Consider finding a pair summing to a target, with pointers at the smallest element (left) and the largest (right).
    forced by · sortedness gives us a known ordering to exploit, which is the only extra information we have
  2. 2
    If a[left] + a[right] > target, the sum is too large. Every element from left to right-1 is ≤ a[right], so pairing a[right] with any of them gives a sum ≥ the current one... no smaller than what we already rejected as too large.
    forced by · sortedness guarantees a[right] is the largest remaining, so it is already paired with the smallest available partner
  3. 3
    Therefore a[right] cannot participate in any valid pair within the remaining window, and can be discarded entirely — not just skipped for this comparison.
    forced by · its best case (pairing with the current minimum) has already failed
  4. 4
    Symmetrically, if the sum is too small, a[left] is already paired with the largest available partner and still falls short, so it can be discarded.
    forced by · no remaining partner is larger than a[right]
  5. 5
    Each comparison eliminates exactly one element permanently, so the window shrinks by one per step and the scan terminates in at most n steps — O(n) after the sort.
    forced by · the window can only shrink, and it starts at size n
⇒ Therefore

Therefore the technique isn't a trick: sortedness converts one comparison into a proof about an entire set of candidates. Without sorting, a failed comparison tells you about that one pair and nothing else, which forces you back to O(n²).

And note what this predicts: the same argument underlies binary search (one comparison eliminates half), the merge step of mergesort, and interval-overlap sweeps — all of them are "sort first, then let ordering license bulk elimination". It also predicts the real decision point, which is a cost comparison rather than a cleverness one: two-pointer costs O(n log n) for the sort but O(1) extra space, while a hash-set solution is O(n) time and O(n) space. So on already-sorted or memory-constrained data two pointers wins, and on unsorted data where memory is free the hash set wins.

Mental modelContiguous memory: cheap at the end, expensive at the front

An array is a single unbroken run of memory. That layout is what makes a[i] a one-step address computation, and it's what lets the CPU prefetch your next elements — which is why a linear scan of an array is dramatically faster than walking a linked structure of the same length, despite both being O(n).

The same layout is what makes the front expensive. Inserting or removing at index 0 requires physically shifting every subsequent element. Appending at the end is nearly free because of over-allocation. Every array performance question reduces to: am I touching the end, or the front?

  • Build strings with "".join(parts) — one pass, one allocation. Never accumulate with += in a loop.
  • Need efficient operations at both ends → collections.deque, which gives O(1) at either end by giving up contiguity and therefore O(1) indexing.
  • Slicing copies. a[i:j] inside a loop is quadratic; use indices, or memoryview for bytes, when you only need to read a window.
  • The three workhorse patterns cover most array problems: two pointers (sorted, pair/partition), sliding window (contiguous subarray with a constraint), and prefix sums (O(1) range queries after O(n) setup).
🔔 Fires when you see

Fire this model the moment you see: string concatenation in a loop · list.pop(0) or insert(0, x) · a nested loop recomputing a sum over a range · df = df.append(row) inside a loop · a job whose runtime quadrupled when input doubled.

The tradeoff

You must find whether any two elements sum to a target. Sort and use two pointers, or use a hash set in one pass?

Sort + two pointers
+ you gain O(1) extra space beyond the sort, cache-friendly sequential access, and it generalises directly to three-sum, closest-pair, and interval problems; the sorted array is often reusable for other queries
− you pay O(n log n) from the sort, and it destroys the original ordering — so if you need the original indices you must carry them along, which reintroduces the memory you saved
pick when the data is already sorted, or memory is the binding constraint, or you need several order-dependent queries over the same data and can amortise one sort across them
Hash set, single pass
+ you gain O(n) time, one pass, trivially simple to write correctly, preserves original ordering and indices, and works on unsorted streaming input where you never see the whole array at once
− you pay O(n) extra memory with substantial per-entry overhead in Python; random access patterns are cache-hostile; and it requires hashable elements, ruling out unhashable keys
pick when the data is unsorted, memory is available, and you need this answer once — the correct default for the standard interview version of the problem
Push it into a database or dataframe join
+ you gain a planner that chooses the strategy using statistics you don't have, spills to disk gracefully, and handles data far larger than memory without you writing any of that
− you pay fixed overhead that dominates for small inputs, less control over the execution strategy, and debugging moves into query plans
pick when the data doesn't fit in memory or already lives in a table — at which point pulling it into Python to loop over it is the actual mistake
What a senior engineer actually does

Hash set unless memory or existing sort order says otherwise. The decision is a straightforward time-versus-space trade, and in most application code space is the cheaper resource — but that inverts sharply at scale, which is precisely why external sort-merge, not hashing, is what handles datasets larger than RAM.

The judgement worth carrying: recognise which of the three regimes you are in before optimising. Hand-optimising a Python loop over data that should have been a SQL join is effort spent on the wrong layer entirely, and no amount of two-pointer cleverness recovers it.


(c) Hands-on · 25 min

Save as arrays_strings.py. Runs top to bottom, prints results, no dependencies beyond stdlib.

"""arrays_strings.py — the two-pointer + sliding-window canon.
 
Run:  python arrays_strings.py
"""
from __future__ import annotations
 
 
# ------------------------------------------------------------------
# 1) Reverse in place — the classic opposite-ends two-pointer
# ------------------------------------------------------------------
def reverse_in_place(a: list) -> list:
    """Reverse the list in-place, O(n) time, O(1) extra memory."""
    left, right = 0, len(a) - 1
    while left < right:
        a[left], a[right] = a[right], a[left]
        left += 1
        right -= 1
    return a
 
 
# ------------------------------------------------------------------
# 2) Two-sum on a sorted array — opposite-ends with a condition
# ------------------------------------------------------------------
def two_sum_sorted(a: list[int], target: int) -> tuple[int, int] | None:
    """Return (i, j) with i<j and a[i]+a[j]==target, else None. O(n)."""
    left, right = 0, len(a) - 1
    while left < right:
        s = a[left] + a[right]
        if s == target:
            return left, right
        if s < target:
            left += 1        # need a bigger sum → move left up
        else:
            right -= 1       # need a smaller sum → move right down
    return None
 
 
# ------------------------------------------------------------------
# 3) Remove duplicates from a sorted array — same-start two-pointer
# ------------------------------------------------------------------
def remove_duplicates(a: list[int]) -> int:
    """Compact the list in place so the first k elements are unique. Return k. O(n)."""
    if not a:
        return 0
    slow = 0
    for fast in range(1, len(a)):
        if a[fast] != a[slow]:
            slow += 1
            a[slow] = a[fast]
    return slow + 1
 
 
# ------------------------------------------------------------------
# 4) Valid palindrome (ignoring non-alphanumerics + case)
# ------------------------------------------------------------------
def is_palindrome(s: str) -> bool:
    left, right = 0, len(s) - 1
    while left < right:
        while left < right and not s[left].isalnum():
            left += 1
        while left < right and not s[right].isalnum():
            right -= 1
        if s[left].lower() != s[right].lower():
            return False
        left += 1
        right -= 1
    return True
 
 
# ------------------------------------------------------------------
# 5) Longest substring without repeating characters — sliding window
# ------------------------------------------------------------------
def longest_unique_substring(s: str) -> int:
    """Return length of the longest substring with all distinct chars. O(n)."""
    seen: dict[str, int] = {}   # char -> most recent index
    left = 0
    best = 0
    for right, ch in enumerate(s):
        if ch in seen and seen[ch] >= left:
            left = seen[ch] + 1     # jump left past the previous occurrence
        seen[ch] = right
        best = max(best, right - left + 1)
    return best
 
 
# ------------------------------------------------------------------
# 6) Minimum-size subarray with sum >= target — classic sliding window
# ------------------------------------------------------------------
def min_subarray_len(target: int, nums: list[int]) -> int:
    left = 0
    total = 0
    best = float("inf")
    for right, x in enumerate(nums):
        total += x
        while total >= target:
            best = min(best, right - left + 1)
            total -= nums[left]
            left += 1
    return 0 if best == float("inf") else best
 
 
# ------------------------------------------------------------------
# 7) Anti-pattern demo — string concat in a loop is O(n²)
# ------------------------------------------------------------------
def slow_join(pieces: list[str]) -> str:
    out = ""
    for p in pieces:
        out += p              # each concat copies the whole prefix — O(n²) total
    return out
 
 
def fast_join(pieces: list[str]) -> str:
    return "".join(pieces)    # O(n) — builds once with the exact final length
 
 
# ------------------------------------------------------------------
# Runner
# ------------------------------------------------------------------
if __name__ == "__main__":
    print("reverse:", reverse_in_place([1, 2, 3, 4, 5]))
    print("two_sum_sorted:", two_sum_sorted([1, 3, 4, 7, 11, 15], 18))
    a = [1, 1, 2, 2, 3, 4, 4, 5]
    k = remove_duplicates(a)
    print(f"remove_duplicates: k={k}  compacted={a[:k]}")
    print("palindrome 'A man, a plan, a canal: Panama':", is_palindrome("A man, a plan, a canal: Panama"))
    print("longest_unique 'abcabcbb':", longest_unique_substring("abcabcbb"))
    print("min_subarray_len target=7 nums=[2,3,1,2,4,3]:", min_subarray_len(7, [2, 3, 1, 2, 4, 3]))
 
    # Prove the concat anti-pattern
    import time
    pieces = ["x"] * 200_000
    t = time.perf_counter(); slow_join(pieces);  print(f"slow_join  200k: {time.perf_counter()-t:.3f}s")
    t = time.perf_counter(); fast_join(pieces);  print(f"fast_join  200k: {time.perf_counter()-t:.3f}s")

Anatomy of the script

What each function teaches

reverse_in_place
The canonical opposite-ends two-pointer. O(n) time, O(1) extra memory. Note the tuple-swap idiom — Pythonic and fast.
two-ptr
two_sum_sorted
The core insight: if the sum is too small, only the left pointer can fix it (moving right would shrink the sum further). This monotonicity is why O(n) is possible.
two-ptr
remove_duplicates
Same-start pattern: slow marks the ‘write head’, fast scans. Nothing gets moved backwards, so total work is O(n).
two-ptr
is_palindrome
Two-pointer with skip logic — non-alphanumerics get skipped without extra memory. Watch the nested while loops that keep left < right.
two-ptr
longest_unique_substring
Sliding window with a hashmap of last-seen index. Left jumps past the previous occurrence — never backwards.
sliding
min_subarray_len
Sliding window with a running sum. Shrink from the left as long as the window remains valid — the classic template.
sliding
slow_join / fast_join
The 200,000-piece benchmark proves ''.join(list) beats += in a loop by orders of magnitude. Live this rule.
immutability
Try itAdd ‘longest substring with at most K distinct characters’ using the same sliding-window skeleton

Starter:

def longest_at_most_k_distinct(s: str, k: int) -> int:
    from collections import defaultdict
    counts: dict[str, int] = defaultdict(int)
    left = 0
    best = 0
    for right, ch in enumerate(s):
        counts[ch] += 1
        while len(counts) > k:
            counts[s[left]] -= 1
            if counts[s[left]] == 0:
                del counts[s[left]]
            left += 1
        best = max(best, right - left + 1)
    return best
 
# Test:
print(longest_at_most_k_distinct("eceba", 2))   # expected 3 ("ece")
💡 Hint · State = dict of char -> count. Window is invalid when len(state) > K. Shrink from the left, decrementing counts, and pop keys that hit 0. Same three-line rhythm as longest_unique_substring.

(d) Production reality · 15 min

War story Instagram · early Django daysrequest latency spike
🔥 What broke
An engineer built a template caption formatter that used caption += tag + " " in a loop over N hashtags. On viral posts with 500+ hashtags, formatter latency exceeded 800 ms — dominating the whole request budget.
🧯 The fix
Replace with " ".join(tags). Latency dropped to sub-millisecond. Same fix applied elsewhere in the codebase found a dozen similar hotspots.
🎓 Lesson to steal
Python string immutability + += in a loop is O(n²). It's fine at n=10, invisible at n=100, and a production incident at n=1000. Grep your codebase for it.
War story Cloudflare · 2016· 2016Cloudbleed — 6-day leak
🔥 What broke

A one-character bug in an HTML parser (== instead of >=) let a pointer walk PAST the end of a buffer. The parser then dumped whatever memory was next to the buffer into HTTP responses — including passwords, private messages, and API keys.

The leak persisted for months until a Google Project Zero researcher noticed sensitive data in cached Cloudflare responses.

🧯 The fix
Emergency patch to the parser. Long-term: rewrote the parser in Rust (which prevents out-of-bounds by design). Cloudflare purged every CDN cache globally.
🎓 Lesson to steal
‘Off by one’ in pointer arithmetic isn't a cute bug — in C/C++/hand-rolled parsers it's a memory-safety vulnerability. In Python, out-of-bounds throws IndexError — small blessing, huge safety win.
Post-mortem
War story Common failure mode · every log parser everuniversal
🔥 What broke
A log parser splits lines by " " and indexes fields[7] for the status code. One request had an extra space in the URL. fields[7] is now the wrong column. Dashboards show 0% error rate. Real error rate is 12%.
🧯 The fix
Never split by whitespace and index into positions. Use a real log format (JSON logs, structured logging via structlog / loguru) so parsing is by NAME, not by index.
🎓 Lesson to steal
Positional indexing into split() output is fragile. Structured logging costs nothing and pays for itself the first time production drifts.

Where this shows up in the rest of the plan

Two-pointer + sliding-window shows up everywhere
S024 · Hashmaps & Sets
Unsorted + two-pointer → falls back to hashmap. Complement pattern.
S026 · Stacks & Queues
Monotonic stack/queue = sliding window's cousin (next-greater-element, sliding max).
S032 · Binary Search
‘Sorted array + condition’ is often binary search OR two-pointer — know both.
S033 · Dynamic Programming
When windows aren't enough (choices per index), DP takes over.
S048 · Log Parsing at Scale
The Instagram + log-parser stories play out at petabyte scale.
S085 · Tokenisation for LLMs
BPE + tokeniser code is 90% string-slicing and sliding-window merges.

(e) Recall + stretch · 10 min

Recall — click to reveal · click to reveal
★ = stretch question

Explain-out-loud test

Teach these three in one minute each, no notes:

  1. Why does Python list index access cost O(1) but list-front-insert cost O(n)?
  2. What is the sliding-window template, and give one problem it solves.
  3. Why is s += 'x' in a loop a bug, and what's the fix?

What comes next

Hub: The 6-Month Learning Plan


Part of a 130-session evergreen learning series. Session structure: intuition → visual → hands-on → production war stories → recall. Duration: 90 minutes.