The one-sentence answer

A parameter is a single adjustable number inside a model, and training is the process of finding good values for all of them. When someone says a model has 70 billion parameters, they mean its behaviour is controlled by 70 billion such numbers. Nothing mystical is hidden in there. A model, once trained, is essentially a very large file of numbers plus a fixed recipe for combining them with your input.

The easiest way to feel this is to start at the opposite end, with a model so small that you can hold every parameter in your head.

A model with exactly two parameters

Suppose you want to predict house prices from size. You collect five sales, and you decide to describe the relationship with a straight line: price equals w times size plus b. That is the entire model. The two numbers w and b are its parameters. The first controls how steeply the price rises with size, and the second is a base price when size is zero. Before training, you have no idea what they should be, so both start at zero.

Training means nudging each number in whichever direction reduces the prediction error, over and over. Here is the complete program, which I ran:

# The smallest "model" there is: price = w * size + b   (2 parameters)
sizes  = [4, 6, 8, 10, 12]            # house size in hundreds of square feet
prices = [31, 43, 55, 69, 79]         # price in lakh rupees

w, b = 0.0, 0.0                       # both parameters start as ignorance
lr = 0.01
for step in range(20000):
    gw = gb = 0.0
    for x, y in zip(sizes, prices):
        err = (w * x + b) - y
        gw += 2 * err * x / len(sizes)
        gb += 2 * err / len(sizes)
    w -= lr * gw                      # nudge each knob against its error
    b -= lr * gb

print(f"learned w = {w:.2f}, b = {b:.2f}   (2 parameters)")
print(f"prediction for a 9-hundred sq ft house: {w * 9 + b:.1f} lakh")

Output:

learned w = 6.10, b = 6.60   (2 parameters)
prediction for a 9-hundred sq ft house: 61.5 lakh

The program discovered that each extra hundred square feet adds about 6.1 lakh rupees, with a base of about 6.6 lakh. Its prediction for a 900-square-foot house was 61.5 lakh. Two numbers, learned from five examples, now carry a small piece of knowledge about the housing market. That is all a parameter ever is: a knob whose setting was found by trial and error against data.

Think of the mixing console in a recording studio. A model with two parameters is a console with two sliders. A model with 70 billion parameters is a console with 70 billion sliders, each set by an automated process that listened to the result millions of times and moved the sliders a tiny amount each time.

From one line to a neural network

A straight line can only describe straight-line relationships. To capture richer patterns, such as recognising a face or continuing a sentence, models stack many simple units called neurons into layers. Each neuron takes numbers in, multiplies each by its own weight, adds them up with a bias (a constant nudge), and passes the result through a small bending function. A weight and a bias are both parameters.

The picture below is the smallest network that can learn the classic XOR puzzle, which outputs 1 when its two inputs differ and 0 when they match. A single straight line cannot solve it, but one hidden layer can.

Counting is mechanical. Every connection between two neurons is one weight. The first gap has 2 × 4 = 8 connections, and the second has 4 × 1 = 4. Every neuron that receives input also has one bias: 4 in the hidden layer plus 1 at the output. The total is 8 + 4 + 4 + 1 = 17 parameters. I trained exactly this network:

import numpy as np

X = np.array([[0, 0], [0, 1], [1, 0], [1, 1]], float)
y = np.array([[0], [1], [1], [0]], float)          # XOR: true when the inputs differ
sig = lambda z: 1 / (1 + np.exp(-z))

def train(seed, steps=5000, lr=0.5):
    rng = np.random.default_rng(seed)               # the starting values are random
    W1, b1 = rng.normal(0, 1, (2, 4)), np.zeros(4)  # 2 inputs -> 4 hidden
    W2, b2 = rng.normal(0, 1, (4, 1)), np.zeros(1)  # 4 hidden -> 1 output
    n = W1.size + b1.size + W2.size + b2.size
    fwd = lambda: (lambda h: (h, sig(h @ W2 + b2)))(np.tanh(X @ W1 + b1))
    loss0 = float(np.mean((fwd()[1] - y) ** 2))
    for _ in range(steps):
        h, p = fwd()
        d2 = (p - y) * p * (1 - p)
        d1 = (d2 @ W2.T) * (1 - h ** 2)
        W2 -= lr * h.T @ d2; b2 -= lr * d2.sum(0)
        W1 -= lr * X.T @ d1; b1 -= lr * d1.sum(0)
    p = fwd()[1]
    return n, loss0, float(np.mean((p - y) ** 2)), p

n, l0, l1, p = train(seed=1)
print("parameters:", n, "(8 + 4 + 4 + 1)")
print("loss before training:", round(l0, 3))
print("loss after training :", round(l1, 4))
for row, out in zip(X, p):
    print(f"  inputs {row.astype(int)} -> {out[0]:.2f}")
ok = sum(train(s)[2] < 0.01 for s in range(20))
print()
print(f"different random starting values: {ok} of 20 runs learned XOR")

Output:

parameters: 17 (8 + 4 + 4 + 1)
loss before training: 0.24
loss after training : 0.0001
  inputs [0 0] -> 0.00
  inputs [0 1] -> 0.99
  inputs [1 0] -> 0.99
  inputs [1 1] -> 0.01

different random starting values: 19 of 20 runs learned XOR

The loss, a measure of how wrong the network was, fell from 0.24 to 0.0001, and the final predictions are almost perfectly 0, 1, 1, 0. Seventeen numbers now implement the XOR rule, and nobody wrote that rule down. Training found it.

The last line of the output deserves attention. Training began from random numbers, and in 19 of 20 attempts the network learned XOR, but in one it got stuck. The starting values matter, and training is a search, not a guarantee. Large models deal with this through careful initialisation and a lot of engineering, but the underlying nature of the process is the same.

How parameter counts are worked out

A fully connected layer with n_in inputs and n_out outputs has n_in × n_out weights and n_out biases. Chain a few layers and add the counts. Consider a simple digit recogniser that looks at a 28 by 28 grid of pixels, which is 784 numbers, passes them through a layer of 128 neurons, and then through a final layer of 10, one for each digit.

def linear(n_in, n_out):
    """A fully connected layer: one weight per connection, one bias per output."""
    return n_in * n_out + n_out

# 1. A small image classifier: 784 pixels -> 128 -> 10 digits
print("digit classifier:", f"{linear(784, 128):,} + {linear(128, 10):,} = {linear(784, 128) + linear(128, 10):,}")

# 2. GPT-2 small, counted from its published shape
d, layers, vocab, ctx = 768, 12, 50257, 1024
per_layer = (4 * d * d + 4 * d)          # attention: query, key, value, output (with biases)
per_layer += (8 * d * d + 5 * d)         # feed-forward: d -> 4d -> d (with biases)
per_layer += 4 * d                       # two layer norms
total = layers * per_layer + vocab * d + ctx * d + 2 * d
print(f"GPT-2 small    : {total:,}  (about {total / 1e6:.0f} million)")

# 3. A 70B-class design (Llama 2 70B shape: grouped-query attention, gated feed-forward)
d, layers, vocab = 8192, 80, 32000
kv_dim, ffn = 8 * 128, 28672             # 8 key/value heads of size 128
attn = 2 * d * d + 2 * d * kv_dim        # query + output, plus smaller key + value
mlp = 3 * d * ffn                        # gate, up and down projections
per_layer = attn + mlp + 2 * d           # plus two norms
total = layers * per_layer + 2 * vocab * d + d
print(f"70B-class model: {total:,}  (about {total / 1e9:.1f} billion)")
print(f"  attention is {layers * attn / total:.1%} of it, feed-forward {layers * mlp / total:.1%}, "
      f"embeddings {2 * vocab * d / total:.1%}")

Output:

digit classifier: 100,480 + 1,290 = 101,770
GPT-2 small    : 124,439,808  (about 124 million)
70B-class model: 68,976,648,192  (about 69.0 billion)
  attention is 17.5% of it, feed-forward 81.7%, embeddings 0.8%

That digit recogniser has 101,770 parameters: 100,480 in the first layer and 1,290 in the second. Notice that most of the parameters sit in the first layer, simply because 784 inputs times 128 outputs is a large product. This pattern repeats at every scale: parameters hide where two big dimensions meet.

Counting a real language model

The same program continues with two real architectures. Modern language models are built from transformers, whose layers each contain two main blocks: an attention block, which lets each word look at the other words, and a feed-forward block, which processes each word on its own. Both are made of big grids of numbers called weight matrices.

For GPT-2 small, a model released in 2019, the published shape is 12 layers, a width of 768, and a vocabulary of 50,257 tokens. Putting those through the counting rules gives 124,439,808 parameters, which matches the 124 million that the model is known by. So the counting really works; there is no hidden accounting.

Then I did the same for a 70-billion-class design, using the architecture numbers Meta published for Llama 2 70B: 80 layers, a width of 8,192, a feed-forward size of 28,672, grouped attention with 8 key-value heads, and a vocabulary of 32,000. The total came out at 68,976,648,192. That is about 69.0 billion, which is what gets rounded and sold as “70B”. Model names are marketing-friendly roundings, so a model called 70B might have 68 or 71 billion in reality.

The breakdown is instructive. About 17.5 percent of the parameters are in the attention blocks, 81.7 percent in the feed-forward blocks, and under one percent in the table that converts words to numbers. If you picture a language model as a giant recall system, the feed-forward blocks are where most of its capacity sits.

What a parameter is not

People pick up some wrong pictures, so let me clear them out.

• It is not a stored fact. No single parameter holds “Paris is the capital of France”. Knowledge is spread across enormous numbers of parameters working together, the way a photograph is spread across millions of pixels. Change one pixel and the picture survives; change one parameter and the model barely notices.
• It is not a neuron. Neurons are the units; parameters are the weights and biases that connect and adjust them. The 17-parameter network had only 5 neurons.
• It is not lines of code. The code that runs a 70B model is short, a few hundred lines. The size is in the numbers, not the instructions.
• It is not a direct measure of intelligence. A newer, well-trained 8B model can beat an older, poorly trained 30B one, because quality depends on the data, the training recipe and the design, not just the count.
• It is not the same as the brain’s connections. You will sometimes see comparisons with the number of synapses in a brain. The two are built and behave very differently, so such comparisons are loose at best.

Why the count decides what hardware you need

Here is the practical consequence, and the reason the number is on every announcement. Each parameter is a number that must be stored, and every one of them is used each time the model produces a word. So size translates directly into memory and computation.

Memory: bytes per parameter

The memory needed to simply hold the model is the parameter count times the bytes used for each parameter. Numbers can be stored with different precision: 32-bit floats take 4 bytes, 16-bit take 2, 8-bit take 1 and 4-bit take half a byte. I worked this out for 70 billion parameters:

import math

params = 70e9
print(f"Storing {params / 1e9:.0f} billion parameters")
for name, nbytes in (("32-bit float", 4), ("16-bit (fp16/bf16)", 2), ("8-bit integer", 1), ("4-bit integer", 0.5)):
    gb = params * nbytes / 1e9
    print(f"  {name:<20}{gb:>6.0f} GB   80 GB cards for the weights alone: {math.ceil(gb / 80)}")

print()
print("Training with Adam in mixed precision, a common recipe (bytes per parameter):")
parts = [("16-bit weights", 2), ("16-bit gradients", 2), ("32-bit master weights", 4),
         ("Adam momentum", 4), ("Adam variance", 4)]
for n, b in parts:
    print(f"  {n:<24}{b}")
total = sum(b for _, b in parts)
print(f"  {'total':<24}{total}   ->  {params * total / 1e12:.2f} TB before any activations")

Output:

Storing 70 billion parameters
  32-bit float           280 GB   80 GB cards for the weights alone: 4
  16-bit (fp16/bf16)     140 GB   80 GB cards for the weights alone: 2
  8-bit integer           70 GB   80 GB cards for the weights alone: 1
  4-bit integer           35 GB   80 GB cards for the weights alone: 1

Training with Adam in mixed precision, a common recipe (bytes per parameter):
  16-bit weights          2
  16-bit gradients        2
  32-bit master weights   4
  Adam momentum           4
  Adam variance           4
  total                   16   ->  1.12 TB before any activations

At 16-bit precision, the common format, the weights alone weigh 140 gigabytes. No single 80 GB graphics card can hold that, so you need at least two. Shrink each parameter to 8 bits and it just about squeezes onto one card; shrink to 4 bits and it takes about 35 gigabytes. That shrinking is called quantisation, and it is the reason that large models can run on much more modest hardware. The trade-off is a small loss in accuracy, which is usually acceptable at 8 bits and noticeable more often at lower precision.

Notice the caveat in the output: those figures are for the weights alone. Running the model also needs working memory for the conversation, which grows with the length of the text, so real deployments need headroom.

Training is far hungrier. A common recipe stores, for every parameter, a 16-bit copy, a 16-bit gradient, a 32-bit master copy and two 32-bit running statistics for the optimiser. That is 16 bytes per parameter, which for 70 billion comes to about 1.12 terabytes, before counting the temporary values computed during the forward pass. This is why training happens across racks of GPUs, while running a finished model can sometimes be done on one.

Computation: operations per word

The second cost is arithmetic. A widely used rule of thumb says that generating one token needs about two operations per parameter, and training needs about six operations per parameter per token of training data. Here is what that gives:

params, tokens = 70e9, 2e12                 # 70 billion parameters, 2 trillion training tokens
train_flops = 6 * params * tokens           # rule of thumb: about 6 operations per parameter per token
per_token = 2 * params                      # generating one token: about 2 operations per parameter
print(f"training work : {train_flops:.2e} operations")
print(f"one new token : {per_token / 1e9:.0f} billion operations")

peak, utilisation = 312e12, 0.40            # an A100-class GPU peak in 16-bit, and a realistic share of it
hours = train_flops / (peak * utilisation) / 3600
print(f"on one GPU at {utilisation:.0%} of {peak / 1e12:.0f} TFLOP/s: {hours / 1e6:.2f} million GPU-hours")
print(f"spread over 2,000 GPUs: about {hours / 2000 / 24:.0f} days")
print()
print(f"tokens per parameter in this recipe: {tokens / params:.0f}")

Output:

training work : 8.40e+23 operations
one new token : 140 billion operations
on one GPU at 40% of 312 TFLOP/s: 1.87 million GPU-hours
spread over 2,000 GPUs: about 39 days

tokens per parameter in this recipe: 29

Generating a single token with a 70B model takes roughly 140 billion arithmetic operations. Training on 2 trillion tokens needs around 8.4 times ten to the 23rd operations. On one A100-class GPU, using realistic assumptions of 40 percent of its peak speed, that is about 1.9 million GPU-hours, or roughly 39 days if you spread it over 2,000 GPUs. For comparison, Meta reported a figure in the same ballpark, about 1.7 million GPU-hours, for training the 70B model, so the rule of thumb lands close to reality. It is also why training a model is a headline-grabbing investment, while using it is comparatively cheap.

One last number in the output is worth pausing on: tokens per parameter. A 2022 study from DeepMind suggested that, for a fixed compute budget, models were being made too big and fed too little text, and recommended roughly 20 training tokens per parameter as a good balance. Their 70B-parameter model trained on 1.4 trillion tokens beat much larger models that had seen less text. The lesson is that parameter count is only half of the story. The other half is how much data the parameters were tuned against.

Size words you will meet

The mixture-of-experts row catches many people out. In such a model, the layers contain several specialist blocks and a small router chooses a few of them for each word. The full set of experts must sit in memory, so the memory bill follows the total parameter count, but only the chosen experts do any arithmetic, so the speed follows the active count. A model can therefore be large to store but quick to run.

Model sizes in real life

Different jobs call for very different sizes. Click through five everyday situations and notice that bigger is not always the right answer.

How to choose a size

When people ask me which model size they need, I do not start from the parameter count. I start from the job and the hardware. A short sequence of questions settles most cases.

Common mistakes and misconceptions

• Assuming more parameters always means a better model. Data quality, training length and design matter as much. Compare actual results on your task.
• Confusing parameters with training data. Parameters are the learned numbers; training data is the text the numbers were tuned on. Both are measured in billions, in different units.
• Forgetting the precision. “70B” says nothing about file size until you know the bytes per parameter. The same model is 140 GB at 16-bit and 35 GB at 4-bit.
• Ignoring working memory. Weights are the minimum. Long conversations need extra room on top.
• Treating the headline number as exact. Names are rounded; the real count of my 70B-class calculation was 69.0 billion.
• Mixing up total and active parameters. A mixture-of-experts model can quote a big total but compute with a fraction.
• Expecting one parameter to hold one fact. Knowledge is distributed across many parameters at once.
• Using a giant model for a tiny task. A spam filter does not need 70 billion parameters, and paying for them wastes money and energy.

A mental model that sticks

Picture an orchestra of seventy billion musicians, each holding a single instrument that can play only one note at a time, loud or soft. No individual musician plays a melody, and none of them knows the song. Yet when each one adjusts their volume to exactly the right amount, the combined sound becomes music. Training is the long rehearsal in which a conductor, listening to the result, tells every musician to turn up or down by a hair, repeated trillions of times. The “70 billion” is the head count of the orchestra, and the size of the concert hall you need follows directly from it.

Practice problems with answers

Try each one on paper before you tap to check.

Test yourself: five quick questions

Tap a question to reveal the answer. The correct option is marked.

Frequently asked questions

The post What Does “70 Billion Parameters” Actually Mean? appeared first on Learn With Examples.

A different way to look for a needle

Imagine you are a security guard checking faces against a wanted poster. You could memorise the poster and study every visitor’s eyes, nose, jaw and hairline one feature at a time. Or you could take a quick glance at each visitor and ask a much cheaper question first: does this person even roughly resemble the photo? Ninety-nine visitors fail that glance instantly. For the rare one who passes, then you do the careful feature-by-feature check.

That is Rabin-Karp in one paragraph. The glance is a hash, a number computed from the text. The careful check is the ordinary letter-by-letter comparison. The algorithm is named after its two inventors, the computer scientists Michael Rabin and Richard Karp, who published it in 1987, and its central idea has aged remarkably well.

Step one: turn a string into a number

Computers love numbers and are slow at comparing long strings, so let us convert. The easiest example uses digits, because a string of digits already is a number. The text “31415” becomes the number thirty-one thousand four hundred fifteen. Two strings of digits are equal exactly when their numbers are equal, and comparing two numbers is one operation no matter how many digits are inside.

That already sounds like a win, but there is a catch. With a needle 50 letters long, the number can be enormous, far too big to handle in one machine word. The fix is to keep only the remainder after dividing by some fixed number, the modulus. In my first example I use 13, which is tiny on purpose so that you can do the arithmetic in your head. The number 31415 divided by 13 leaves a remainder of 7. So the fingerprint of “31415” is 7.

Notice what we gave up. Many different strings leave the same remainder: 7 is also the fingerprint of other five-digit numbers. We will deal with that in a moment, because it is the whole reason the algorithm has a verification step. First, the clever bit.

Step two: the rolling update

To search a long text, you need the fingerprint of every window of five consecutive digits. Computing each one from scratch would cost five operations per window, which is no better than comparing the digits directly. The rolling hash avoids that by reusing the previous answer.

Take the window “31415” and slide one place to the right so that it becomes “14152”. Three things happened: the leading 3 fell off the left, every remaining digit shifted one place up, and a new 2 arrived at the right. In arithmetic terms:

Check it with real numbers: (31415 − 3 × 10000) × 10 + 2 = 1415 × 10 + 2 = 14152. Correct, and it took three arithmetic operations, regardless of how long the window is.

The picture shows the same update done modulo 13, which is what the program really does. The weight of the leading digit, 10⁴, becomes 3 mod 13. Starting from the fingerprint 7, you compute (7 − 3 × 3) × 10 + 2 = −18. That is negative, so you add 13 until it is non-negative, giving 8. And 14152 divided by 13 does leave a remainder of 8. The shortcut and the long way agree.

Negative intermediate results deserve a warning. Python’s remainder operator always returns a non-negative result for a positive modulus, so the code above just works. In C, C++ and Java, the remainder of a negative number can be negative, which silently breaks the fingerprints. If you write this in one of those languages, add the modulus back before taking the remainder.

Walking through a whole search

Here is a complete search on digits, with the numbers printed so nothing is hidden. The haystack is 2359023141526739921, the needle is 31415, and the modulus is 13. For every window the program shows its fingerprint and whether it equals the needle’s fingerprint of 7. It also asserts, on every step, that the rolling update gives the same answer as computing the fingerprint from scratch.

# Textbook-style walk-through with digits so you can follow the arithmetic.
text, pat, mod = "2359023141526739921", "31415", 13
m = len(pat)
target = int(pat) % mod
print(f"pattern {pat} -> {int(pat)} mod {mod} = {target}")
print()
print("shift  window   hash   verdict")
high = 10 ** (m - 1) % mod
h = int(text[:m]) % mod
for s in range(len(text) - m + 1):
    window = text[s:s + m]
    assert h == int(window) % mod            # rolling value equals the direct value
    verdict = ""
    if h == target:
        verdict = "REAL match" if window == pat else "spurious hit (same hash, different text)"
    print(f"{s:>5}  {window}  {h:>5}   {verdict}")
    if s < len(text) - m:
        h = ((h - int(text[s]) * high) * 10 + int(text[s + m])) % mod

Output:

pattern 31415 -> 31415 mod 13 = 7

shift  window   hash   verdict
    0  23590      8   
    1  35902      9   
    2  59023      3   
    3  90231     11   
    4  02314      0   
    5  23141      1   
    6  31415      7   REAL match
    7  14152      8   
    8  41526      4   
    9  15267      5   
   10  52673     10   
   11  26739     11   
   12  67399      7   spurious hit (same hash, different text)
   13  73992      9   
   14  39921     11

Read down the hash column. At shift 6 the window is 31415 and the fingerprint is 7, the same as the needle: a real match. But look at shift 12. The window is 67399, which is a completely different string, and yet its fingerprint is also 7. This is called a spurious hit or false positive. If the algorithm trusted fingerprints alone, it would report a match there that does not exist.

That is why the verification step is not optional. Whenever fingerprints agree, Rabin-Karp compares the real letters. In this example it would check the window at shift 6, confirm it, check the window at shift 12, reject it, and ignore the other thirteen windows entirely. Two letter-by-letter comparisons instead of fifteen. On a longer text with a longer needle, the saving grows.

From digits to letters

Letters are not digits, but every character has a numeric code, such as 65 for A and 97 for a. So we treat a string as a number written in base 256 (a base large enough to cover ordinary byte values), where each character is one digit. The text “ABC” becomes 65 × 256² + 66 × 256 + 67 = 4,276,803. Everything we did with base 10 works with base 256, as long as we replace 10 by 256 in the update formula.

Here is the complete function, followed by a check of the rolling update on a small example and a tiny search:

def rabin_karp(text, pat, base=256, mod=1_000_000_007):
    """Return (match positions, hash hits that needed checking, false alarms)."""
    n, m = len(text), len(pat)
    if m == 0 or m > n:
        return [], 0, 0
    high = pow(base, m - 1, mod)             # weight of the leading character
    hp = ht = 0
    for i in range(m):                       # fingerprints of pattern and first window
        hp = (hp * base + ord(pat[i])) % mod
        ht = (ht * base + ord(text[i])) % mod
    hits, checks, false_alarms = [], 0, 0
    for s in range(n - m + 1):
        if ht == hp:                         # fingerprints agree: now compare letters
            checks += 1
            if text[s:s + m] == pat:
                hits.append(s)
            else:
                false_alarms += 1
        if s < n - m:                        # slide the window by one letter
            ht = ((ht - ord(text[s]) * high) * base + ord(text[s + m])) % mod
    return hits, checks, false_alarms


if __name__ == "__main__":
    base, mod = 256, 1_000_000_007
    h = lambda w: sum(ord(c) * base ** (len(w) - 1 - i) for i, c in enumerate(w)) % mod
    old = h("ABC")
    new = ((old - ord("A") * pow(base, 2, mod)) * base + ord("D")) % mod
    print("hash(ABC)                  =", old)
    print("slide: drop A, add D       =", new)
    print("hash(BCD) computed afresh  =", h("BCD"))
    print("rolling update is correct  =", new == h("BCD"))
    print()
    print(rabin_karp("the quick brown fox jumps over the lazy dog", "brown"))

Output:

hash(ABC)                  = 4276803
slide: drop A, add D       = 4342596
hash(BCD) computed afresh  = 4342596
rolling update is correct  = True

([10], 1, 0)

The top four lines confirm the rolling update: the hash of “ABC” is 4,276,803, and dropping the A and adding a D gives 4,342,596, exactly the same as computing “BCD” from scratch. The last line is the search result for “brown” in “the quick brown fox jumps over the lazy dog”: one match at position 10, one hash hit that needed checking, and zero false alarms.

The function returns three numbers: the match positions, how many fingerprint agreements triggered a letter check, and how many of those checks turned out to be false alarms. That last number is the interesting one, because it is where the algorithm can go wrong.

Collisions and the choice of modulus

A fingerprint collision happens when two different strings produce the same number. They cannot be avoided entirely, since there are far more possible strings than possible remainders. What you control is how often they happen, and the main lever is the size of the modulus. If the fingerprints behave like random numbers, the chance that a wrong window collides with the needle is about one in the modulus.

I tested that on 200,000 random letters with an 8-letter needle that does not occur, using three moduli:

import random
from rk import rabin_karp

random.seed(5)
text = "".join(random.choice("abcdefghijklmnopqrstuvwxyz") for _ in range(200000))
pat = "qzjxkvwp"
print(f"{len(text) - len(pat) + 1:,} windows, pattern of {len(pat)} letters")
print(f"{'modulus':>15}{'hash hits':>11}{'false alarms':>14}")
for mod in (101, 10007, 1_000_000_007):
    hits, checks, bad = rabin_karp(text, pat, 256, mod)
    print(f"{mod:>15,}{checks:>11}{bad:>14}   real matches: {len(hits)}")

Output:

199,993 windows, pattern of 8 letters
        modulus  hash hits  false alarms
            101       2041          2041   real matches: 0
         10,007         19            19   real matches: 0
  1,000,000,007          0             0   real matches: 0

With a tiny modulus of 101, about 1 window in 100 was a false alarm: 2,041 wasted letter checks across nearly 200,000 windows. With 10,007 the number dropped to 19, and with a modulus around a billion it was zero. The pattern matches the “one in the modulus” rule almost perfectly.

So why not always use a gigantic modulus? Because arithmetic on huge numbers costs more, and in languages with fixed-width integers a product can overflow. A common compromise is a prime modulus around a billion, with care that intermediate products fit in 64 bits. Python hides this problem because its integers grow as needed, which is also why Python code like mine runs slower than the same idea in C.

Why a prime, and why random?

Prime moduli spread remainders more evenly, which reduces patterned collisions. There is also a security angle. If a program uses a fixed, publicly known base and modulus, a hostile user can craft inputs that collide on purpose, forcing many false alarms and slowing the search to a crawl. Careful implementations choose the base, or the modulus, at random when the program starts. This is a real concern whenever the text comes from outside.

How fast is it, really?

The running time is about n + m in the typical case, where n is the length of the haystack and m the length of the needle. Computing the first two fingerprints costs about m steps, and each of the n slides costs a constant amount. Verification only happens on fingerprint hits, which are mostly real matches or rare false alarms, so it adds little.

The worst case is n × m. It happens when nearly every window collides, for example with a terrible modulus or an adversarial input, so that you verify almost every position. With a well-chosen, randomised modulus that is extremely unlikely. This is the reverse of the trade-off in KMP, which guarantees linear time on every input but is not always the fastest on friendly text. Rabin-Karp is expected linear time with a small, controllable risk.

For a single short needle, honestly, Rabin-Karp is not the champion. The fingerprint arithmetic costs more per step than comparing a single letter, and skip-ahead methods or your language’s built-in search beat it. Its real strength lies elsewhere.

The superpower: many needles at once

Suppose you have 2,000 banned words, or a database of known phrases, and one document to scan. With one-needle-at-a-time methods you repeat the entire search 2,000 times. Rabin-Karp lets you do it in a single pass. Compute the fingerprint of every needle once and put them in a set. Then slide the window over the document one time, and at each position ask a single question: is this window’s fingerprint in the set? A set lookup takes constant time, so the cost of one pass hardly depends on how many needles you have.

The one restriction is that this simple version needs all the needles to be the same length, so that one window size fits them all. For different lengths, you run one pass for each distinct length.

I measured it with 200 needles of 8 letters, five of them planted in a 200,000-letter text:

import random, time
from rk import rabin_karp
from kmp import kmp_search

random.seed(3)
letters = "abcdefghijklmnopqrstuvwxyz"
text = "".join(random.choice(letters) for _ in range(200000))
patterns = ["".join(random.choice(letters) for _ in range(8)) for _ in range(195)]
patterns += [text[i:i + 8] for i in (1000, 50000, 90000, 150000, 199000)]   # five planted
M = 8; BASE = 256; MOD = (1 << 61) - 1

def multi_rabin_karp(text, patterns):
    """One pass over the text, any number of equal-length patterns."""
    table = {}
    for p in patterns:
        hp = 0
        for ch in p:
            hp = (hp * BASE + ord(ch)) % MOD
        table.setdefault(hp, []).append(p)
    high = pow(BASE, M - 1, MOD)
    ht = 0
    for ch in text[:M]:
        ht = (ht * BASE + ord(ch)) % MOD
    found = {}
    for s in range(len(text) - M + 1):
        if ht in table:
            for p in table[ht]:
                if text[s:s + M] == p:
                    found.setdefault(p, []).append(s)
        if s < len(text) - M:
            ht = ((ht - ord(text[s]) * high) * BASE + ord(text[s + M])) % MOD
    return found

if __name__ == "__main__":
    def timed(fn):
        t = time.perf_counter(); r = fn(); return r, (time.perf_counter() - t) * 1000

    a, ta = timed(lambda: multi_rabin_karp(text, patterns))
    b, tb = timed(lambda: {p: h for p in patterns for h in [kmp_search(text, p)[0]] if h})
    d, td = timed(lambda: {p: True for p in patterns if text.find(p) >= 0})
    print(f"{len(patterns)} patterns, {len(text):,} letters of text")
    print(f"patterns found by Rabin-Karp : {len(a)}")
    print(f"patterns found by KMP loop   : {len(b)}")
    print(f"patterns found by str.find   : {len(d)}")
    print()
    print(f"one Rabin-Karp pass (pure Python) : {ta:8.1f} ms")
    print(f"one KMP run per pattern (Python)  : {tb:8.1f} ms")
    print(f"str.find per pattern (C built-in) : {td:8.1f} ms")

Output:

200 patterns, 200,000 letters of text
patterns found by Rabin-Karp : 5
patterns found by KMP loop   : 5
patterns found by str.find   : 5

one Rabin-Karp pass (pure Python) :    144.5 ms
one KMP run per pattern (Python)  :   4373.2 ms
str.find per pattern (C built-in) :     26.8 ms

All three methods found the same five planted needles. The single Rabin-Karp pass took roughly 70 to 150 milliseconds across my runs, depending on how busy the machine was. Running my Python KMP once per needle took about four seconds, roughly thirty to sixty times slower. But notice the last line: Python’s built-in find, written in C, finished the 200 searches in under 30 milliseconds, and beat my pure Python Rabin-Karp. The language matters as much as the algorithm.

The real question is what happens as the number of needles grows. So I scaled it up:

import random, time
from multi import multi_rabin_karp, text, letters

random.seed(11)
print(f"{'patterns':>9}{'Rabin-Karp one pass':>22}{'str.find per pattern':>23}")
for k in (200, 2000, 20000):
    pats = ["".join(random.choice(letters) for _ in range(8)) for _ in range(k)]
    t = time.perf_counter(); multi_rabin_karp(text, pats); a = (time.perf_counter() - t) * 1000
    t = time.perf_counter(); [text.find(p) for p in pats]; b = (time.perf_counter() - t) * 1000
    print(f"{k:>9,}{a:>19.1f} ms{b:>20.1f} ms")

Output:

 patterns   Rabin-Karp one pass   str.find per pattern
      200              119.1 ms                31.5 ms
    2,000              125.6 ms               277.1 ms
   20,000              107.5 ms              2757.4 ms

This is the picture that justifies the method. The per-needle approach grows in a straight line: ten times more needles took about ten times longer, reaching 2.7 seconds for 20,000. The single Rabin-Karp pass stayed roughly flat around a tenth of a second, because walking through the text dominates and each lookup costs the same. Somewhere between 200 and 2,000 needles, the crossover happens even against a C built-in, and beyond it the gap keeps widening.

Fingerprinting documents: a plagiarism detector in 40 lines

Here is where the rolling hash really surprised me early in my career. Instead of searching for a known needle, you can fingerprint every window of a document and keep the whole set of fingerprints. Two documents that share long passages will share many fingerprints. Two unrelated documents will share almost none.

The recipe: lowercase the text, strip everything except letters so that punctuation and spacing tricks do not matter, then compute the rolling hash of every 12-letter window. The fingerprint set is a compact summary of the document. Comparing two summaries means intersecting two sets.

import re

BASE, MOD, K = 256, (1 << 61) - 1, 12

def fingerprints(text, k=K):
    """Set of rolling hashes of every k-letter window (letters only, lower-case)."""
    t = re.sub(r"[^a-z]", "", text.lower())
    high = pow(BASE, k - 1, MOD)
    h = 0
    for ch in t[:k]:
        h = (h * BASE + ord(ch)) % MOD
    out = {h}
    for s in range(len(t) - k):
        h = ((h - ord(t[s]) * high) * BASE + ord(t[s + k])) % MOD
        out.add(h)
    return out

original = ("Rolling hashes let a program fingerprint every window of a document in constant time, "
            "which is why plagiarism checkers can compare thousands of essays quickly.")
edited   = ("Rolling hashes let a program fingerprint every window of a document in constant time, "
            "which is why copy detectors can compare thousands of essays very quickly.")
unrelated = ("The monsoon reached the hills early this year, and the apple orchards along the river "
             "were heavy with blossom by the middle of April.")

base_fp = fingerprints(original)
print(f"original has {len(base_fp)} fingerprints")
for name, doc in (("light edit", edited), ("unrelated", unrelated), ("exact copy", original)):
    shared = len(base_fp & fingerprints(doc))
    print(f"{name:<11}: {shared:>3} shared  ->  {100 * shared / len(base_fp):5.1f}% of the original")

Output:

original has 122 fingerprints
light edit :  88 shared  ->   72.1% of the original
unrelated  :   0 shared  ->    0.0% of the original
exact copy : 122 shared  ->  100.0% of the original

The original sentence produced 122 fingerprints. A lightly edited copy, where I swapped two phrases, still shared 88 of them, 72 percent. An unrelated sentence about apple orchards shared none. An exact copy shared all 122. Notice that the edit did not destroy the comparison: because each fingerprint covers only 12 letters, the untouched stretches of the sentence still match even after the changed words break the windows around them.

Real systems build on exactly this. They pick the window length carefully, since too short gives false similarity from common phrases and too long misses lightly reworded copying. Some, such as the approach behind Stanford’s well-known MOSS code-similarity tool, keep only a well-chosen sample of the fingerprints so that the summary stays small. The core machinery is still the rolling hash.

All the methods side by side

Where rolling fingerprints are used

You may never write Rabin-Karp for a Ctrl+F box, but its idea runs through systems you use daily. Click through five situations and notice what they share: the data is big, the comparison is repeated, and a cheap fingerprint filters out nearly everything before any expensive work.

Which approach should you choose?

Here is the short checklist I use. The first question settles most real-world cases.

Common mistakes and how to avoid them

• Trusting the fingerprint without verifying. A hash match only means “probably equal”. Always compare the real letters before reporting a hit, as the spurious match at shift 12 showed.
• Choosing a tiny modulus. A modulus of 101 gave about one false alarm per hundred windows in my test. Use a large prime.
• Forgetting the negative remainder. After subtracting the outgoing letter, the value can be negative. In C, C++ and Java, add the modulus before taking the remainder.
• Getting the leading weight wrong. The weight must be base to the power m − 1, taken modulo the modulus. A wrong power gives fingerprints that look plausible but never match.
• Letting integers overflow. With a modulus near a billion and a base of 256, one product fits in 64 bits. Choose both values together.
• Using a predictable base and modulus on untrusted text. Attackers can craft collisions. Randomise one of them when the program starts.
• Mixing pattern lengths in the multi-needle version. One window length checks one needle length. Group the needles by length.
• Reaching for it when the built-in would do. For a single search in ordinary code, the language’s own find is simpler and faster.

A mental model that sticks

Think of a coat check. Each coat gets a small numbered ticket, and instead of describing the coat every time, staff just compare tickets, which is instant. Occasionally two coats might share a ticket number because the ticket numbers are short, so before handing a coat back, the attendant glances at it to make sure it really is yours. A rolling fingerprint is the ticket, the glance is the verification step, and the rolling update is a ticket machine that can print the next ticket from the previous one without looking at the coat again.

Practice problems with answers

Try each one on paper before you tap to check.

Test yourself: five quick questions

Tap a question to reveal the answer. The correct option is marked.

Frequently asked questions

The post Rabin-Karp: Searching With Rolling Fingerprints appeared first on Learn With Examples.

The problem: a needle in a haystack

Computer scientists call the long text the haystack and the thing you are looking for the needle. Searching a web page, a Word document, a source file or a chromosome all reduce to the same question: at which positions does the needle appear in the haystack?

It sounds trivial because your eyes do it so easily. But a computer has no eyes. It can only compare one character with another, one pair at a time. The whole art of string matching is about how few of those comparisons you can get away with. Fewer comparisons mean a faster search, and when the haystack holds a hundred million characters, the difference between a clever method and a careless one is the difference between a blink and a coffee break.

The obvious method, and why it is not enough

Imagine you write the needle on a strip of tracing paper and lay it over the start of the haystack. If every letter lines up, you have a match. If any letter disagrees, you slide the strip one place to the right and start checking again from the strip’s first letter. This is the naive or brute-force method, and it is the one most people invent on their own.

For ordinary text it is fine. Searching for “budget” in an essay, most positions fail on the very first letter, so each slide costs one comparison. The trouble starts when the text is repetitive. Suppose the haystack is a thousand letter A’s followed by a B, and the needle is fifty A’s followed by a B. At every starting position the first fifty letters match, and only the last comparison fails. The method then slides by one and repeats almost all the same work.

Look at the picture. The first attempt matched five letters, ABABA, and then failed at the sixth. A naive search forgets everything and starts again one step to the right. But we are not ignorant: we just saw that the text contains ABABA at that spot. The last three of those letters, ABA, are also the first three letters of the needle. So the needle can slide two places and carry on from its fourth letter, without re-checking ABA at all. That one observation is the whole of KMP.

Before we get there, let me put numbers on the pain. I wrote the naive search with a counter on every character comparison, and ran it on three kinds of text:

import random
from kmp import kmp_search

def naive_search(text, pat):
    hits, comps = [], 0
    for s in range(len(text) - len(pat) + 1):
        j = 0
        while j < len(pat):
            comps += 1
            if text[s + j] != pat[j]:
                break
            j += 1
        if j == len(pat):
            hits.append(s)
    return hits, comps

def horspool_search(text, pat):
    m = len(pat)
    shift = {c: m - 1 - i for i, c in enumerate(pat[:-1])}   # bad-character table
    hits, comps, s = [], 0, 0
    while s <= len(text) - m:
        j = m - 1
        while j >= 0:                    # compare from the END of the pattern
            comps += 1
            if text[s + j] != pat[j]:
                break
            j -= 1
        if j < 0:
            hits.append(s)
        s += shift.get(text[s + m - 1], m)
    return hits, comps

if __name__ == "__main__":
    essay = ("every time you press the find shortcut the page has to look through thousands of "
             "characters for a short phrase. a careless method restarts at every position, while "
             "a smarter one remembers what it has already seen and skips ahead. looking for a needle "
             "in a haystack sounds slow, yet careful programs do it in the blink of an eye. ") * 3
    random.seed(7)
    dna = "".join(random.choice("ACGT") for _ in range(100000))
    cases = [
        ("Repeated A, then B", "A" * 1000 + "B", "A" * 50 + "B"),
        ("English paragraph",  essay, "blink of an eye"),
        ("Random DNA",         dna,   "GATTACAGATTACA"),
    ]
    print(f"{'case':<20}{'text':>8}{'pat':>5}{'hits':>6}{'naive':>10}{'KMP':>9}{'Horspool':>10}")
    for name, text, pat in cases:
        a, ca = naive_search(text, pat)
        b, cb = kmp_search(text, pat)
        c, cc = horspool_search(text, pat)
        assert a == b == c
        print(f"{name:<20}{len(text):>8}{len(pat):>5}{len(a):>6}{ca:>10}{cb:>9}{cc:>10}")

Output:

case                    text  pat  hits     naive      KMP  Horspool
Repeated A, then B      1001   51     1     48501     1951      1001
English paragraph        987   15     3      1018      990       163
Random DNA            100000   14     0    133365   124971     47163

On the repeated-A text the naive method needed 48,501 comparisons to scan a text only 1,001 characters long, roughly 48 for every character. The same job took KMP 1,951 and the Horspool method (more on that shortly) 1,001. On a plain English paragraph the three were all close, because ordinary text rarely has long near-misses. This matters: the naive method is not slow on average. It is slow on the wrong input, and a program that handles user-supplied text cannot choose its input.

In the worst case the naive method needs about n × m comparisons, where n is the haystack length and m the needle length. A 100,000-character log and a 1,000-character needle could mean a hundred million comparisons. KMP needs about n + m.

The big idea: never throw away what you learned

Here is a human version of the idea. You are reading a phone book looking for the name “ANANAS”. You read A, N, A, N, A, and then the next letter is not S, it is N. A naive reader would go back to the second letter of the page and start over. But you already know that the last few letters you read were A, N, A, and that is exactly how ANANAS begins. So you just carry on as if you had started a short while ago, already three letters into the name.

KMP formalises that gut feeling. Before it touches the haystack, it studies the needle alone and prepares a small cheat sheet: for each position in the needle, how much of the needle’s own beginning is also its ending? That cheat sheet is the failure table, also called the prefix function or LPS table, short for “longest proper prefix which is also a suffix”.

Prefixes, suffixes and borders

These words sound heavy but the idea is plain. A prefix is the beginning of a word: for ABABA the prefixes are A, AB, ABA and ABAB. A suffix is the ending: A, BA, ABA and BABA. A border is something that is both. In ABABA, the string ABA is a prefix and also a suffix, so ABA is a border of length 3. (We exclude the whole word, which is why the table says “proper”.)

Why does a border matter? Because it tells you what the needle can safely fall back to after a failure. If you have matched ABABA and then hit a mismatch, you know the last three letters of what you matched are ABA, which is the start of the needle. So you resume as though you had matched only ABA. No searching backward in the haystack is needed.

Reading a failure table

The table holds one number per letter of the needle: the border length of the needle up to and including that letter. I printed the tables for three needles:

def build_lps(pat):
    """lps[i] = length of the longest proper prefix of pat[:i+1]
    that is also a suffix of it."""
    lps = [0] * len(pat)
    k = 0                                # length of the prefix matched so far
    for i in range(1, len(pat)):
        while k > 0 and pat[i] != pat[k]:
            k = lps[k - 1]               # fall back to a shorter prefix
        if pat[i] == pat[k]:
            k += 1
        lps[i] = k
    return lps


def kmp_search(text, pat):
    """Return (list of match positions, number of character comparisons)."""
    lps = build_lps(pat)
    hits, comps, k = [], 0, 0
    for i, ch in enumerate(text):        # the text pointer only moves forward
        while True:
            comps += 1
            if ch == pat[k]:
                k += 1
                break
            if k == 0:
                break
            k = lps[k - 1]               # reuse what we already matched
        if k == len(pat):
            hits.append(i - k + 1)
            k = lps[k - 1]               # allow overlapping matches
    return hits, comps


if __name__ == "__main__":
    for pat in ("ABABAC", "ABCDABD", "AAAB"):
        lps = build_lps(pat)
        print(f"pattern {pat}")
        print("  char:", " ".join(pat))
        print("  lps :", " ".join(map(str, lps)))
    print()
    hits, comps = kmp_search("ABABABAC", "ABABAC")
    print("search ABABAC in ABABABAC -> start", hits, "after", comps, "comparisons")

Output:

pattern ABABAC
  char: A B A B A C
  lps : 0 0 1 2 3 0
pattern ABCDABD
  char: A B C D A B D
  lps : 0 0 0 0 1 2 0
pattern AAAB
  char: A A A B
  lps : 0 1 2 0

search ABABAC in ABABABAC -> start [2] after 9 comparisons

For ABABAC the table reads 0, 0, 1, 2, 3, 0. Read it this way: after matching the first five letters ABABA, the border is 3, so if the sixth letter fails, fall back to having matched 3 letters. The final C has no border, hence 0. For ABCDABD the table shows 1 and 2 under the second A and B, because the needle starts again with AB halfway through. For AAAB the table climbs 0, 1, 2 because every A is a continuation of the same run; this is precisely the shape of needle that wrecks the naive method, and also precisely where KMP shines.

Notice the first entry is always 0, since a single letter has no proper prefix. A needle with no repeated structure, such as ABCDEF, gets all zeros, and then KMP behaves almost like the naive method, because there is nothing to reuse.

The search itself, step by step

With the table ready, the search runs two pointers. One walks through the haystack and only ever moves forward. The other, call it k, counts how many letters of the needle are currently matched. At each haystack letter you do one of three things:

• It matches the next needle letter. Increase k by one. If k now equals the needle length, you found a match.
• It does not match and k is above zero. Fall back: set k to the table entry just before it, then try the same haystack letter again against the shorter match.
• It does not match and k is zero. Nothing to fall back to. Move to the next haystack letter.

Here is the whole algorithm, which is shorter than most people expect:

def kmp_search(text, pat):
    """Return (list of match positions, number of character comparisons)."""
    lps = build_lps(pat)
    hits, comps, k = [], 0, 0
    for i, ch in enumerate(text):        # the text pointer only moves forward
        while True:
            comps += 1
            if ch == pat[k]:
                k += 1
                break
            if k == 0:
                break
            k = lps[k - 1]               # reuse what we already matched
        if k == len(pat):
            hits.append(i - k + 1)
            k = lps[k - 1]               # allow overlapping matches
    return hits, comps

Trace the earlier example. The haystack is ABABABAC and the needle ABABAC. The first five letters match, so k climbs to 5. The sixth haystack letter is B but the needle wants C, a mismatch. The table entry for the fifth needle letter is 3, so k drops to 3, meaning “ABA is still matched”. The needle’s fourth letter is B, which matches the haystack’s B, so k rises to 4. Then A matches, k is 5, then C matches, k is 6, and 6 equals the needle length: a match starting at position 2. The program output said exactly that, found at start 2 after 9 comparisons, for a haystack of 8 letters.

Why it is fast: the counting argument

The inner loop looks scary because of the fallback, which could in theory repeat many times. Here is why it cannot get out of hand. Each time the haystack pointer advances, k can grow by at most one. Each fallback strictly lowers k. So the total number of fallbacks over the whole search can never exceed the total number of increases, and that is at most n. Add the n forward steps and the fallbacks and you get at most about 2n comparisons. Building the table works the same way and costs about m. The total, n plus m, is called linear time, and it holds for any input.

That is why the experiment gave 1,951 comparisons for the 1,001-character repeated text. It lands just under the theoretical ceiling of 2n, which is 2,002. The algorithm is nearly as slow as it is allowed to be on that input, and still about twenty-five times better than the naive method.

Is KMP always the fastest? Honestly, no

This is the part most tutorials skip. Look again at the third row of the comparison output. On 100,000 random letters of DNA, using a 14-letter needle, KMP made 124,971 comparisons against naive’s 133,365. That is barely better. And the Horspool method, a cousin I have not explained yet, needed only 47,163. KMP’s selling point is the guarantee: it never degrades. It does not promise to win on friendly input.

Why did Horspool win so clearly on ordinary text? Because it uses the opposite trick. Instead of comparing the needle from the front, it compares from the back. If the haystack letter lined up with the needle’s last position is a letter that does not appear in the needle at all, say a Z when searching for “needle”, then no placement overlapping that Z can possibly match, and the whole needle can jump forward by its full length in one step. On English text with a 15-letter needle, that means looking at only about one letter in every several. The English row shows it: 163 comparisons for a 987-character text.

The family of methods built on this backward comparison is called Boyer-Moore, and Horspool is its simplified cousin. Well-known command line tools such as GNU grep are widely documented as using skip-ahead ideas of this kind for plain-text patterns, which is a large part of why searching a gigabyte of logs takes seconds. The longer your needle, the more you can skip, which feels backward until you think about it: more letters to check also means more chances to rule out a placement early.

What Ctrl+F is actually doing

I want to be careful here. Browsers and editors do not publish the exact algorithm they use, it varies between programs and versions, and I would not want to state one as fact. What I can describe with confidence is the shape of the job, because it is the same everywhere:

• Normalise. Most find boxes ignore case by default, so the program compares in a lowercase form, and often ignores accents too.
• Scan. It walks through the visible text of the page, using some string matching routine like the ones in this article.
• Collect every hit. Not just the first. Each match position is stored so the program can count them and jump between them.
• Highlight and report. It paints the matches, shows “3 of 17”, and scrolls to the one you are on.

And here is the honest truth about the word “instantly”. A typical page holds maybe fifty thousand characters. Even the naive method on ordinary text does that in a tiny fraction of a second on a modern machine. My own measurement scanned 300,000 characters in about 35 milliseconds with the slowest method, and I wrote that one in Python, which is itself slow. The instant feel comes mostly from raw hardware speed and the fact that ordinary text is friendly. The cleverness of KMP matters when the text is huge, repetitive, or arrives from an untrusted source and you cannot risk a pathological case.

Counting matches is trickier than it looks

One subtle thing your find box must decide: do matches overlap? Search for “aa” in “aaaaa”. Do you report two hits (positions 0 and 2, taking turns) or four (positions 0, 1, 2 and 3)? Different tools answer differently, and the choice matters when you display a count. I tested three common approaches:

import re
from kmp import kmp_search

text, pat = "aaaaa", "aa"
print("str.count      :", text.count(pat), "(non-overlapping)")
print("re.findall     :", len(re.findall(pat, text)), "(non-overlapping)")
print("KMP, all hits  :", len(kmp_search(text, pat)[0]), "at", kmp_search(text, pat)[0])

doc = "The cat sat. The Cat ran. A cat, the CAT."
print()
print("exact 'cat'    :", doc.count("cat"))
print("ignore case    :", doc.lower().count("cat"))
print("whole word     :", len(re.findall(r"\bcat\b", doc, re.I)))

Output:

str.count      : 2 (non-overlapping)
re.findall     : 2 (non-overlapping)
KMP, all hits  : 4 at [0, 1, 2, 3]

exact 'cat'    : 2
ignore case    : 4
whole word     : 4

Python’s count and the regular expression finder both say 2, because they skip past each match before looking again. KMP, as I wrote it, resets to the border after a match and finds all 4 overlapping hits. Neither is wrong; they answer different questions. The second block shows the other classic gotcha: exact search for “cat” finds 2 hits in my sentence, while ignoring case finds 4. If a user types a word and sees “2 results” when they can plainly see four, that is a case-sensitivity bug, not an algorithm bug.

The race: real timings

I timed four methods on two inputs: ordinary text, and the worst case for the naive method. These are pure Python implementations, so absolute numbers are modest, but the shape is what counts.

import random, time
from kmp import kmp_search
from compare import naive_search, horspool_search

def best(fn, *a, runs=3):
    t = 1e9
    for _ in range(runs):
        s = time.perf_counter(); fn(*a); t = min(t, time.perf_counter() - s)
    return t * 1000

random.seed(1)
text = "".join(random.choice("abcdefghijklmnopqrstuvwxyz ") for _ in range(300000))
pat = "zzqxjvkw"
print("Ordinary text, 300,000 characters, 8-letter pattern (milliseconds)")
for name, fn in (("naive", naive_search), ("KMP", kmp_search), ("Horspool", horspool_search)):
    print(f"  {name:<10}{best(fn, text, pat):>9.1f}")
print(f"  {'str.find':<10}{best(lambda t, p: t.find(p), text, pat):>9.1f}")

bad = "a" * 60000 + "b"
badpat = "a" * 150 + "b" + "c"
print()
print("Worst case for naive, 60,000 letters (milliseconds)")
print(f"  {'naive':<10}{best(naive_search, bad, badpat, runs=1):>9.1f}")
print(f"  {'KMP':<10}{best(kmp_search, bad, badpat):>9.1f}")
print(f"  {'str.find':<10}{best(lambda t, p: t.find(p), bad, badpat):>9.1f}")

Output (your numbers will differ a little from run to run):

Ordinary text, 300,000 characters, 8-letter pattern (milliseconds)
  naive          36.9
  KMP            27.8
  Horspool        8.5
  str.find        0.2

Worst case for naive, 60,000 letters (milliseconds)
  naive         709.9
  KMP             7.8
  str.find        0.1

Two lessons jump out. First, on ordinary text the order was Horspool, then KMP, then naive, all within a factor of five. The built-in str.find beat all of them by a huge margin, because it is written in C, and recent versions of Python combine several strategies inside it, including a method called Two-Way for longer needles. In real programs, use the built-in. Writing KMP yourself is for learning, for languages without a good built-in, or for streaming problems where you cannot hold the whole text.

Second, on the nasty input the naive search took several hundred milliseconds while KMP took under ten, roughly ninety times faster on my run, and the gap grows as the text grows. That is the scenario that justifies the algorithm: it turns a catastrophic input into an ordinary one.

Other string matching ideas worth knowing

Rabin-Karp: compare fingerprints

Instead of comparing letters, compute a numeric fingerprint, a hash, of the needle, and a rolling hash of each window of the haystack. Sliding the window one step updates the hash in constant time. Only when two fingerprints agree do you check the actual letters. The method is commonly described as a building block in plagiarism detection, because it extends naturally to checking many fragments at once.

Aho-Corasick: many needles at once

What if you have ten thousand needles, such as a database of virus signatures or a list of banned words? Running KMP ten thousand times is wasteful. Aho-Corasick builds one automaton from all the needles, a generalisation of the KMP failure table, and scans the haystack once. Antivirus engines such as ClamAV are known to use it.

Suffix structures: search many times

If the haystack is fixed and you will search it again and again, as genome databases do, you can pay once to build an index such as a suffix array, and then each query takes time depending mostly on the needle rather than the haystack. Scanning three billion DNA letters for every query would be hopeless.

Where string matching shows up in daily life

You use these algorithms far more often than you press Ctrl+F. Click through five everyday situations and notice how the needs differ: sometimes you want every match, sometimes only a guarantee, sometimes thousands of needles at once.

Which method should you reach for?

When someone asks me which algorithm to use, I answer with a short sequence of questions. In practice the first question ends the conversation nine times out of ten.

Common mistakes and how to avoid them

• Reimplementing what the language already does well. Python’s find, JavaScript’s indexOf and Java’s indexOf are tuned and tested. Write KMP when you need streaming, custom matching or a guaranteed bound.
• Forgetting overlapping matches. Decide up front whether “aa” in “aaaaa” counts as two hits or four.
• Ignoring case and accents. Normalise both strings the same way before searching, or users will report missing results.
• Treating text as plain letters when it is not. In Unicode, an accented é can be one character or an e followed by a combining accent. Normalise the text, or two identical-looking strings will not match.
• Off-by-one errors in the failure table. The fallback is lps[k - 1], not lps[k]. This is the most common bug in hand-written KMP.
• Forgetting the empty needle. Decide what searching for an empty string means and handle it before you build a table.
• Passing user input to a regular expression. Characters such as a dot or a bracket have special meaning. Use re.escape or a plain string search.
• Searching inside a loop over a huge list. One search is cheap. Ten thousand searches over a megabyte each is not. Build an index or use a multi-pattern method.

A mental model that sticks

Think of reading a street name on a long road sign while driving past. The naive method is a driver who, on spotting one wrong letter, stops, reverses to the beginning of the sign, and starts reading again. KMP is a driver who has memorised the sign’s own pattern of repeats and never reverses: after a wrong letter he glances at his memory and says, “fine, I have already read the last three letters of the right word”, and keeps driving. Horspool is a driver who reads from the far end of the sign first, and when that letter is not in the word at all, leaps ahead by the whole sign’s length.

Practice problems with answers

Try each one on paper before you tap to check.

Test yourself: five quick questions

Tap a question to reveal the answer. The correct option is marked.

Frequently asked questions

The post How Ctrl+F Finds Text Instantly: KMP and String Matching Explained appeared first on Learn With Examples.

Start with the simple idea: a variable is a label

Before comparing keywords, let us agree on what a variable is. A variable is a name attached to a value so that you can use that value later. Imagine a jar in your kitchen with a sticky label that says “sugar”. The label lets you find what is inside without opening every jar. In code, price is the label and 250 is what is in the jar.

When you declare a variable, you are doing two things: creating the label, and deciding the rules for that label. Can you peel it off and stick it on a different jar later? Who in the house is allowed to see it? Can you use the label before you have written it? Those three questions, reassignment, scope and timing, are exactly where var, let and const differ.

Why does JavaScript have three?

JavaScript was created in 1995 and for its first twenty years it had exactly one way to declare a variable: var. It worked, but its rules were different from nearly every other popular language, and those differences caused bugs that confused beginners and experts alike.

In 2015, the language standard known as ES2015 (many people call it ES6) added two new keywords, let and const, with rules that behave the way programmers coming from other languages expect. The old keyword could not be changed or removed, because millions of existing websites depend on it. So JavaScript ended up with all three. That is the whole story: two modern tools sitting next to an old one that has to stay for compatibility.

The first big difference: scope

Scope answers the question: where in my code can I use this variable? A useful way to picture it is a house. A function scope is the whole house: anyone inside, in any room, can use what is stored in the hallway. A block scope is a single room: what you put in the bedroom stays in the bedroom. A block in JavaScript is any pair of curly braces, such as the body of an if statement or a for loop.

• var is function-scoped. It ignores the curly braces of an if or a loop and is visible everywhere in the surrounding function.
• let and const are block-scoped. They exist only inside the braces where you declared them.

I do not want you to take that on trust, so here is a short test. It declares one variable of each kind inside an if block and then tries to use each one outside it.

// 1. Scope: does the variable survive outside the block?
if (true) {
  var oldWay = "I am var";
  let newWay = "I am let";
  const fixed = "I am const";
}
console.log("var outside the block  :", oldWay);
try { console.log(newWay); } catch (e) { console.log("let outside the block  :", e.name + ": " + e.message); }
try { console.log(fixed); } catch (e) { console.log("const outside the block:", e.name + ": " + e.message); }

// 2. A loop counter
for (var i = 0; i < 3; i++) {}
console.log("var i after the loop  :", i);
for (let j = 0; j < 3; j++) {}
console.log("typeof j after loop   :", typeof j);

// 3. Redeclaring
var price = 100;
var price = 250;          // allowed, silently
console.log("var redeclared        :", price);

Output (run with Node 22):

var outside the block  : I am var
let outside the block  : ReferenceError: newWay is not defined
const outside the block: ReferenceError: fixed is not defined
var i after the loop  : 3
typeof j after loop   : undefined
var redeclared        : 250

The var value leaked out of the block and printed normally. The let and const values did not exist outside the block, so JavaScript threw a ReferenceError. The loop counter shows the same behaviour: after a for (var i...) loop, i is still alive and holds 3. After a for (let j...) loop, j is gone.

Leaking sounds harmless, but think about a long function with several loops that all use var i. They share one variable. A value left behind by the first loop can quietly change the behaviour of the third. Block scope is a safety feature: it keeps each piece of code from trampling the variables of its neighbours.

The second difference: redeclaring

With var, you can declare the same name twice in the same scope and JavaScript says nothing. The last line of the scope test shows it: var price = 100; followed by var price = 250; is allowed, and the value simply becomes 250.

That is dangerous in a big file. Suppose you are 400 lines into a function and you declare var total, forgetting that you already used total near the top for something else. You have just overwritten the earlier value with no warning. With let and const, declaring the same name twice in the same scope is a syntax error, and your editor will usually underline it before you even run the code. A loud mistake is far cheaper than a silent one.

The third difference: hoisting and the “dead zone”

This is the topic that scares people, but the idea is simple. Before JavaScript runs your code, it reads through it once and notes every declaration. This is called hoisting, as if the declarations were lifted to the top of their scope. What happens to the variable next depends on the keyword.

• var is hoisted and given the starting value undefined. You can read it before its line, and you just get undefined.
• let and const are hoisted too, but they stay uninitialised until execution reaches their line. Touching them earlier throws an error. The stretch between the top of the scope and the declaration line is called the temporal dead zone.

// Hoisting: using a variable before the line that declares it
console.log("var before its line   :", early);
var early = 5;

try {
  console.log(late);
} catch (e) {
  console.log("let before its line   :", e.name + ": " + e.message);
}
let late = 5;

try {
  console.log(never);
} catch (e) {
  console.log("const before its line :", e.name + ": " + e.message);
}
const never = 5;

// A function declaration is hoisted completely
console.log("function hoisted      :", greet());
function greet() { return "hello"; }

Output:

var before its line   : undefined
let before its line   : ReferenceError: Cannot access 'late' before initialization
const before its line : ReferenceError: Cannot access 'never' before initialization
function hoisted      : hello

The var variable quietly printed undefined, which is the worst kind of result: no crash, but a wrong value flowing into the rest of your program. The let and const versions failed immediately with a clear message: “Cannot access ‘late’ before initialization”. The error names the variable and tells you what went wrong, so you fix it in seconds.

The last line shows that function declarations are the exception. A function declared with the function keyword is hoisted completely, body and all, so you can call it above where it is written. A function stored in a variable, such as var greet = function() {...}, follows the variable rules instead: the name exists but holds undefined, and calling it early gives a TypeError.

The fourth difference: reassigning

Here let and const part company. A let variable can be given a new value as often as you like. A const variable cannot be pointed at something else once it has been set. It also must be given a value at the moment it is declared.

Use let for things that genuinely change: a running total, a counter, a flag that flips when something happens. Use const for everything else. In real programs, the large majority of variables never need to change after they are first set, so const ends up being the most common keyword in well-written code.

The part everyone gets wrong: const is not “frozen”

New developers often assume const means the value can never change. It does not. It means the label cannot be moved to a different jar. If the jar holds an object or an array, you can still change what is inside it. Watch:

"use strict";
const rate = 0.18;
try { rate = 0.2; } catch (e) { console.log("reassign const      :", e.name + ": " + e.message); }

const cart = ["rice", "dal"];
cart.push("tea");                       // allowed: the array changes, the label does not
console.log("push into const array :", cart);

try { cart = ["new"]; } catch (e) { console.log("point const elsewhere :", e.name + ": " + e.message); }

const user = { name: "Asha", plan: "free" };
user.plan = "pro";                      // allowed
console.log("edit const object     :", user);

const locked = Object.freeze({ plan: "free" });
try { locked.plan = "pro"; } catch (e) { console.log("frozen object edit   :", e.name + ": " + e.message); }
console.log("frozen stays         :", locked);

let count = 0;
count = count + 1;
console.log("let can be reassigned:", count);

Output:

reassign const      : TypeError: Assignment to constant variable.
push into const array : [ 'rice', 'dal', 'tea' ]
point const elsewhere : TypeError: Assignment to constant variable.
edit const object     : { name: 'Asha', plan: 'pro' }
frozen object edit   : TypeError: Cannot assign to read only property 'plan' of object '#<Object>'
frozen stays         : { plan: 'free' }
let can be reassigned: 1

Reassigning the constant failed, as expected. But pushing a new item into a const array worked, and so did changing a property of a const object. The label still points at the same array and the same object; only their contents changed. If you truly need an object that cannot be edited, Object.freeze() locks it, as the last lines show. The freeze is shallow, meaning it protects the top level only, so nested objects still need their own freezing if you want them locked.

The sneaky one: var and the global object

There is one more difference that matters in browsers. A var declared at the top level of a normal script becomes a property of the global window object. That means it can collide with other scripts on the same page, including ad scripts, analytics and plugins. A let or const at the top level does not attach to window. I tested it in a real browser:

// Run as a normal <script> in a browser page
var a = 1;
let b = 2;
const c = 3;
console.log(window.a, window.b, window.c);

Output:

1 undefined undefined

Only the var landed on window. On a WordPress site with a dozen plugins each adding scripts, that is not a small matter. Two plugins that both declare var data at the top level can overwrite each other and produce bugs that appear only on certain pages.

The bug that made everyone switch: loops and callbacks

If you only remember one example from this article, make it this one. You want three delayed messages, each reporting its own loop number. Here is the code with var and then with let:

// The classic bug: three delayed callbacks
console.log("with var:");
for (var a = 0; a < 3; a++) {
  setTimeout(() => console.log("  var  sees", a), 0);
}
setTimeout(() => {
  console.log("with let:");
  for (let b = 0; b < 3; b++) {
    setTimeout(() => console.log("  let  sees", b), 0);
  }
}, 10);

Output:

with var:
  var  sees 3
  var  sees 3
  var  sees 3
with let:
  let  sees 0
  let  sees 1
  let  sees 2

With var every callback printed 3. With let they printed 0, 1 and 2, as you probably hoped. Why? A var loop counter is one single variable shared by the whole function. By the time the delayed callbacks run, the loop has already finished and the variable holds its final value of 3, so every callback sees 3. A let loop counter gets a fresh copy for each pass through the loop, and each callback remembers its own copy.

This is not an obscure puzzle. It shows up in real pages: a row of buttons where each one should report its own number, a photo gallery where each thumbnail should open its own image, a set of tabs. Before let existed, the standard fix was to wrap the body of the loop in an extra function, a pattern that confused many newcomers. Today the fix is to change one word.

All the differences in one table

Real-life examples: which keyword fits?

Rules are easier to remember when they are tied to everyday situations. Click through the five tabs below. Each one is a small, real scenario and the keyword that suits it.

A simple decision guide

When I review code, I ask the same short sequence of questions. It takes a few seconds, and after a week it becomes automatic.

Habits that save you from bugs

• Default to const. If the linter or your editor complains that you tried to reassign, you have just learned something useful: this variable changes, so switch that one to let. Starting with const makes the changing variables stand out.
• Declare as late as you can. Put the variable close to where it is first used and in the narrowest block possible. Short lifetimes mean fewer places for things to go wrong.
• One declaration per line. Writing let a = 1, b = 2, c = 3; on one line is legal, but separate lines are easier to read and easier to change.
• Use a linter. ESLint has the rules no-var and prefer-const, which flag exactly the habits in this article automatically.
• Pick descriptive names. const taxRate tells you a lot more than const t. Constants that are truly fixed, like a maximum limit, are often written in capitals, such as MAX_ITEMS.
• Do not choose for speed. For ordinary code, the three keywords perform about the same. Pick for clarity, not performance.

Common mistakes and how to avoid them

• Thinking const means immutable. It locks the label, not the contents. Objects and arrays stored in a const can still change.
• Using var in a loop with callbacks. Every callback ends up seeing the final value. Use let.
• Declaring let twice in one scope. That is a syntax error. Check whether you really want a second variable or just a new value.
• Using a variable before its declaration. With let and const this throws an error. Move the declaration up, or restructure the code.
• Declaring the same let inside two switch cases. All the cases of a switch share one block, so two let x lines clash. Wrap each case body in its own curly braces.
• Declaring a const without a value. const total; is a syntax error. A const must be set when it is created.
• Rewriting old var code blindly. Changing every var to let can break code that relied on function scope or hoisting. Convert gradually, and test as you go.
• Forgetting that top-level var joins window. In a browser script, that can clash with other scripts. Use let, const or a module.

What about reading old code full of var?

You will meet it. Older tutorials, Stack Overflow answers from before 2015 and many WordPress plugins still use var everywhere. That is not a reason to panic, and it is not a reason to copy the habit. Read the code knowing that every var is function-scoped and hoisted, mentally move its declaration to the top of the function, and you will understand how it behaves.

When you write new code, use the modern keywords. When you edit old code, resist the urge to convert everything in one sitting. Change a variable when you are already touching the lines around it, run your tests, and move on. This is how experienced teams modernise large codebases without breaking them.

A mental model that sticks

Here is how I explain it to new developers. Imagine a whiteboard in a meeting room. A let variable is something written in erasable marker: you can wipe it and write a new value. A const variable is written in permanent marker: you cannot rewrite it, though if the thing you wrote is the name of a notebook, the notebook’s pages can still change. A var variable is a note that gets written on the whiteboard in the corridor as well: everyone in the building can read it, and anyone can overwrite it by accident.

Practice problems with answers

Try each one before you tap to check.

Test yourself: five quick questions

Tap a question to reveal the answer. The correct option is marked.

Frequently asked questions

The post Var vs Let vs Const: Why JavaScript Has Three Ways to Declare Variables appeared first on Learn With Examples.

Why computers need types at all

A computer’s memory is a very long row of tiny switches, each either off or on. That is all it holds: zeros and ones. There is no such thing as a “number” or a “letter” down there. The pattern 01000001, for example, can mean the whole number 65, or it can mean the capital letter A, or it can be one tiny piece of a colour in a photograph. The bits themselves do not say which.

The data type is the label that tells the computer how to read a pattern. It answers three questions at once: how many switches does this value use, how should those switches be interpreted, and which operations make sense on it. You can add two numbers, but adding two pieces of text sticks them together instead, and asking whether a yes/no answer is “greater than 7” makes no sense at all.

Think of a kitchen. A measuring jug, a kitchen scale and a thermometer all give you a number, but you would not use the thermometer to weigh flour. Each tool suits a kind of question. Data types are the same: tools for different kinds of facts.

Integers: whole numbers, exact answers

An integer is a number with no fractional part: −3, 0, 7, 52,314. It is the type for anything you count. The number of people in a stadium, the number of items in a shopping cart, a person’s age in years, the page you are on: all integers. You cannot have 2.5 people in the stadium, so the type simply has no room for a fraction.

The big strength of integers is that they are exact. If you add 3 and 4, you get precisely 7, every time, forever. There is no rounding and no hidden error. That reliability is why money, counters, loops and array positions all lean on integers.

How an integer is stored

Computers store integers in binary, using a fixed number of bits. With 8 bits you have 256 possible patterns. If all of them are used for non-negative numbers (an “unsigned” integer), the range is 0 to 255. That is exactly why colour values on a screen run from 0 to 255: each of red, green and blue gets one byte.

If you also want negative numbers (a “signed” integer), one bit effectively goes to the sign, and the same 256 patterns cover −128 to 127. Wider integers get far more room: 16 bits cover −32,768 to 32,767, 32 bits cover about −2.1 billion to 2.1 billion, and 64 bits cover roughly −9.2 quintillion to 9.2 quintillion.

Overflow: when the number does not fit

Here is where integers can bite. A fixed-width integer has a biggest value, and adding one more does not produce an error in many languages. It wraps around, like a car odometer rolling from 99999 back to 00000.

I ran the numbers to be sure I was not just repeating folklore. This short program simulates what a signed integer of each width does when you add one to its maximum, and also shows the famous Unix time limit.

import struct
from datetime import datetime, timezone

def wrap(value, bits):
    """What a signed integer of this width does when it overflows."""
    m = 1 << bits
    v = value % m
    return v - m if v >= m // 2 else v

for bits in (8, 16, 32):
    top = 2 ** (bits - 1) - 1
    print(f"{bits:>2}-bit signed: max {top:>11,}   max + 1 wraps to {wrap(top + 1, bits):>14,}")

print()
last = 2 ** 31 - 1
print("32-bit Unix time limit:", datetime.fromtimestamp(last, tz=timezone.utc))
print("one second later wraps to:", datetime.fromtimestamp(wrap(last + 1, 32), tz=timezone.utc))

print()
print("Python int keeps going: 2**63 =", 2 ** 63)
print("float loses whole numbers:", float(2 ** 53) + 1 == float(2 ** 53))
print()
for code, name in (("b", "tiny int"), ("h", "short int"), ("i", "int"), ("q", "long int"), ("f", "float32"), ("d", "float64"), ("?", "boolean")):
    print(f"{name:<10} {struct.calcsize(code)} byte(s)")

Output:

 8-bit signed: max         127   max + 1 wraps to           -128
16-bit signed: max      32,767   max + 1 wraps to        -32,768
32-bit signed: max 2,147,483,647   max + 1 wraps to -2,147,483,648

32-bit Unix time limit: 2038-01-19 03:14:07+00:00
one second later wraps to: 1901-12-13 20:45:52+00:00

Python int keeps going: 2**63 = 9223372036854775808
float loses whole numbers: True

tiny int   1 byte(s)
short int  2 byte(s)
int        4 byte(s)
long int   8 byte(s)
float32    4 byte(s)
float64    8 byte(s)
boolean    1 byte(s)

Look at the date line. Many systems count time as the number of seconds since 1 January 1970, stored in a signed 32-bit integer. That counter runs out at 03:14:07 UTC on 19 January 2038. One second later it wraps to a large negative number, which a computer reads as December 1901. This is the “Year 2038 problem”, and engineers have been moving systems to 64-bit time to avoid it.

Real overflow stories

Overflow is not only a classroom curiosity. In 1996 the first launch of Europe’s Ariane 5 rocket ended in self-destruction after a value converted from a 64-bit floating-point number into a 16-bit signed integer was too large to fit. In December 2014, the view counter for the video “Gangnam Style” passed 2,147,483,647, the top of a 32-bit signed integer, and YouTube said it had to switch the counter to 64 bits. Both are reminders that the width of an integer is a decision, not a detail.

Python has an unusual comfort here: its integers grow as large as memory allows, so 2¹⁰⁰ is simply 1,267,650,600,228,229,401,496,703,205,376. Languages such as C, Java and Go use fixed widths, and there you must choose carefully.

Floats: numbers with a decimal point

A float, short for floating-point number, is for values that can have a fractional part: 3.14, 36.6, 0.0001, 6.02 × 10²³. Use it for things you measure rather than count: body temperature, the length of a table, the speed of a car, the voltage across a battery.

The name “floating point” describes how the number is stored. Instead of fixing the decimal point in one place, the computer keeps two pieces: the significant digits, and an exponent that says where the point floats to. It is the same idea as scientific notation. The number 6,020,000 can be written 6.02 × 10⁶: the digits 6.02 and the exponent 6. A float stores a pair like that, but in binary.

This design gives floats a huge range. A standard 64-bit float, which most languages call a “double”, reaches from about 2.2 × 10⁻³⁰⁸ up to about 1.8 × 10³⁰⁸. No integer of the same size comes close. The price is exactness.

The 0.1 + 0.2 mystery, solved

Let me show you the thing that confuses everyone, then explain it. In Python:

import math

print("7           ->", type(7).__name__)
print("7.0         ->", type(7.0).__name__)
print("True        ->", type(True).__name__)
print('"7"         ->', type("7").__name__)
print()
print('"5" + "5"           =', repr("5" + "5"))
print('5 + 5               =', 5 + 5)
print('int("5") + int("5") =', int("5") + int("5"))
print()
print("10 / 4  =", 10 / 4)
print("10 // 4 =", 10 // 4)
print("10 % 4  =", 10 % 4)
print()
print("True + True       =", True + True)
print("isinstance(True, int) =", isinstance(True, int))
print("0.1 + 0.2         =", 0.1 + 0.2)
print("0.1 + 0.2 == 0.3  =", 0.1 + 0.2 == 0.3)
print("isclose           =", math.isclose(0.1 + 0.2, 0.3))
print("2 ** 100          =", 2 ** 100)

Output:

7           -> int
7.0         -> float
True        -> bool
"7"         -> str

"5" + "5"           = '55'
5 + 5               = 10
int("5") + int("5") = 10

10 / 4  = 2.5
10 // 4 = 2
10 % 4  = 2

True + True       = 2
isinstance(True, int) = True
0.1 + 0.2         = 0.30000000000000004
0.1 + 0.2 == 0.3  = False
isclose           = True
2 ** 100          = 1267650600228229401496703205376

Toward the bottom you will see 0.1 + 0.2 comes out as 0.30000000000000004, and the comparison with 0.3 is false. Python is not being silly. The same result appears in JavaScript, Java, C and nearly every language, because it comes from how binary stores fractions.

Here is the analogy I use with students. Try writing one third in decimal. You get 0.3333… and it never ends. You can write as many 3s as you like, but you can never write it exactly. Now notice that one third is a perfectly ordinary number. It is only awkward because our counting system is base ten, and ten is not divisible by three.

Computers use base two. In base two, one tenth is the awkward one. It becomes 0.0001100110011… repeating forever, so it has to be cut off somewhere. What the computer actually stores for 0.1 is a very slightly different number. I printed it:

from decimal import Decimal
import struct, sys

print("0.1 is really stored as:", Decimal(0.1))
print("0.5 is stored as       :", Decimal(0.5), "(a power of two, so exact)")
print("0.75 is stored as      :", Decimal(0.75))
bits = format(struct.unpack(">Q", struct.pack(">d", 0.1))[0], "064b")
print("64 bits of 0.1:", bits[0], bits[1:12], bits[12:])
print("largest float  :", sys.float_info.max)
print("smallest float :", sys.float_info.min)
print("digits of care :", sys.float_info.dig)
import math
print()
print("gap between neighbouring floats:")
print("  near 1             :", math.ulp(1.0))
print("  near 1e15          :", math.ulp(1e15))
print("  near 2**53 (9e15)  :", math.ulp(2.0 ** 53))

Output:

0.1 is really stored as: 0.1000000000000000055511151231257827021181583404541015625
0.5 is stored as       : 0.5 (a power of two, so exact)
0.75 is stored as      : 0.75
64 bits of 0.1: 0 01111111011 1001100110011001100110011001100110011001100110011010
largest float  : 1.7976931348623157e+308
smallest float : 2.2250738585072014e-308
digits of care : 15

gap between neighbouring floats:
  near 1             : 2.220446049250313e-16
  near 1e15          : 0.125
  near 2**53 (9e15)  : 2.0

So the stored 0.1 is 0.1000000000000000055511… Not exactly one tenth, but close enough that you normally never notice. Add two such slightly-off values and the tiny errors can show up at the sixteenth or seventeenth digit. Notice too that 0.5 and 0.75 are stored exactly, because they are built from powers of two (a half, a quarter). A float is only exact when the fraction is made of halves, quarters, eighths and so on.

Floats lose precision as they get big

There is a second quirk. A 64-bit float has only about 15 to 16 reliable decimal digits, and the gaps between neighbouring floats grow as the numbers grow. That is the output at the bottom of the block above: near 1, neighbouring floats are about 0.0000000000000002 apart. Near 9 quadrillion, they are 2 apart, so the float simply cannot represent every whole number.

The practical rule: beyond 2⁵³, which is 9,007,199,254,740,992, floats skip whole numbers. In JavaScript, where every number is a float, this is why the biggest integer you can trust is 9,007,199,254,740,991. It is also why you should never store something like a 19-digit ID in a float.

How to compare floats safely

Because of those tiny errors, never ask whether two floats are exactly equal after calculation. Ask whether they are close enough. In Python, the function math.isclose does it. In other languages, take the absolute difference and compare it with a small tolerance, such as 0.000001, chosen to suit your problem. This one habit prevents a surprising number of bugs.

Booleans: the simplest type of all

A boolean has exactly two possible values: true or false. It is named after the mathematician George Boole, who built a branch of logic around exactly this idea in the nineteenth century. In everyday life, a boolean is any yes/no fact: is the user logged in, is the light on, has the invoice been paid, is the stock above its limit.

Booleans are the type that makes decisions possible. Every if statement and every loop condition eventually boils down to a boolean. When you write “if age is at least 18”, the comparison produces true or false, and the program picks a path based on it.

Storage: one bit of information, one byte of space

A boolean carries one bit of information, since there are only two answers. Yet most systems store it in a whole byte, because computers address memory in bytes and it is faster to read a full byte than to dig out a single bit. The little table at the end of the overflow code printed this: a boolean takes 1 byte, the same as a tiny integer. Programmers who need to store millions of flags sometimes pack eight of them into a single byte for exactly this reason.

Booleans in Python are secretly integers

In Python, True behaves like 1 and False behaves like 0. The code above showed that True + True gives 2 and that a boolean passes the test for being an integer. That is occasionally handy, for example counting how many items in a list satisfy a test by adding up their true/false results. It is also a source of odd bugs, so keep it in mind.

A more important trap involves text. The string “False” is not the boolean False. In Python, any non-empty string counts as true, so bool(“False”) is True. This bites people reading settings from a file or a web form, where everything arrives as text. Convert the value deliberately, instead of trusting it.

Why they differ: a side-by-side view

With all three in front of you, the differences are easier to see in one place.

Notice that each type’s strength and weakness come from the same decision. Integers are exact because they cannot hold fractions. Floats can hold fractions because they give up exactness. Booleans are simple because they hold almost nothing. There is no free lunch, just a trade-off chosen for each job.

The fourth type that fools people: strings

Strictly speaking this article is about three types, but one more deserves a mention because beginners confuse it with integers all the time. A string is text: letters, digits, spaces, anything in quotation marks. The string “5” and the integer 5 look alike but behave very differently. Adding the two integers 5 + 5 gives 10. Adding the two strings “5” + “5” gives “55”, the text glued together.

This matters whenever data comes from outside your program. Form fields, CSV files and web pages deliver everything as text. If you forget to convert with int() or float(), you will get joined-up text instead of sums, or an error. The sample output earlier shows all three results side by side.

The reverse mistake is just as common. A phone number, PIN code or postal code is made of digits, but you never add or average them. Store them as strings. A PIN of 0482 stored as an integer becomes 482, and the leading zero is gone for good.

The money rule: never use floats

If there is one lesson from this article that will save you real trouble, it is this: do not store money in floats. It is the most common serious data-type mistake in shops, billing systems and spreadsheets. Here is a test. A shop sells 1,000 sweets at 0.10 each. I added the price one thousand times three ways:

from decimal import Decimal

# A shop sells 1,000 sweets at 0.10 each. Add the prices one by one.
total_float = 0.0
for _ in range(1000):
    total_float += 0.10
print("float total   :", total_float, "| equals 100.0?", total_float == 100.0)

total_dec = Decimal("0")
for _ in range(1000):
    total_dec += Decimal("0.10")
print("Decimal total :", total_dec, "| equals 100?", total_dec == 100)

paise = 0
for _ in range(1000):
    paise += 10                      # store money as whole paise (integers)
print("integer paise :", paise, "-> rupees", paise / 100)

print()
print("19.99 * 3       =", 19.99 * 3)
print("1999 * 3 paise  =", 1999 * 3, "-> 59.97")

Output:

float total   : 99.9999999999986 | equals 100.0? False
Decimal total : 100.00 | equals 100? True
integer paise : 10000 -> rupees 100.0

19.99 * 3       = 59.97
1999 * 3 paise  = 5997 -> 59.97

The float total is 99.9999999999986 and fails the check against 100. The Decimal type, which stores decimal digits exactly, gets 100.00. And whole units, in this case paise, get exactly 10,000, which is 100 rupees. The tiny 0.0000000000014 error would never show up on a receipt, but it can break an equality check, trip a reconciliation report or make a daily total off by a cent when multiplied by millions of transactions.

To be fair, floats do not always misbehave. At the bottom of the output, 19.99 × 3 prints a clean 59.97. That is the danger: it works nearly every time, so you trust it, and then one day it does not. Whenever correctness matters, use one of two safe approaches:

• Integer smallest units. Store 19.99 rupees as 1999 paise, or 19.99 dollars as 1999 cents. Calculate in whole numbers, and divide by 100 only when displaying.
• A decimal type. Python has Decimal, Java has BigDecimal, and most databases have a DECIMAL or NUMERIC column. These store decimal digits exactly.

Data types in everyday life

You do not need to be a programmer to meet these types. Click a tab to see five everyday situations and which type fits each one, and why.

Choosing the right type: a quick decision guide

When I design a new database table or a form, I go through the same short list. Read the questions top to bottom and stop at the first yes.

Converting between types

Programs constantly change a value from one type to another, and some conversions lose information. Knowing which is useful.

• Integer to float is usually safe for small numbers. 7 becomes 7.0. Very large integers can lose digits, as we saw past 2⁵³.
• Float to integer throws away the fraction. In Python, int(3.9) gives 3 and int(−3.9) gives −3, since it chops toward zero instead of rounding. If you want rounding, call round() on purpose.
• Rounding has quirks. Python’s round() uses “round half to even”, so round(2.5) is 2 and round(3.5) is 4. It is designed to avoid a bias when many values are rounded, but it surprises people who expect 2.5 to become 3.
• String to number needs int() or float(), and it fails with an error if the text is not a valid number. Always handle that case when reading user input.
• Anything to boolean follows the language’s own rules. In Python, zero, empty text and empty lists count as false, and almost everything else counts as true, including the string “False”.
• Division is a type decision too. In Python, 10 / 4 is 2.5, a float, while 10 // 4 is 2, whole-number division, and 10 % 4 is the remainder, 2. In some older languages, dividing two integers silently gives an integer, which has caused many off-by-a-fraction bugs.

Common mistakes and how to avoid them

• Using floats for money. Use integer paise or cents, or a decimal type.
• Comparing floats with ==. Compare with a tolerance instead.
• Storing IDs, phone numbers and PINs as numbers. If you never do arithmetic on it, it is probably a string.
• Ignoring the width of an integer. If a counter could ever pass about two billion, use a 64-bit integer from the start.
• Trusting text as a boolean. The text “false” is true in most languages. Convert it deliberately.
• Forgetting that form data is text. Everything from a web form arrives as a string until you convert it.
• Assuming integer division behaves the same everywhere. Check how your language divides whole numbers.
• Storing huge whole numbers in floats. Beyond about nine quadrillion, floats skip numbers.

A mental model that sticks

If you remember only one picture, make it this. An integer is a staircase: you are on step 4 or step 5, never in between, and each step is exactly where it says it is. A float is a ramp with markers painted on it: you can stand anywhere, but the markers are only approximate, and they are spaced further apart as you climb. A boolean is a light switch: on or off, nothing else. Pick the staircase when you count, the ramp when you measure and the switch when you decide.

Practice problems with answers

Try each one before you tap to check.

Test yourself: five quick questions

Tap a question to reveal the answer. The correct option is marked.

Frequently asked questions

The post Data Types Explained: Integers, Floats, Booleans and Why They Differ appeared first on Learn With Examples.

The rule everyone dreads

If you have a function made by dividing one function by another, say f(x) divided by g(x), its derivative is given by the quotient rule:

That is the whole rule. It looks harmless on one line. In practice, it is where marks get lost: a sign flipped, a square forgotten, an algebra slip while expanding the top. After many years of tutoring and teaching this, I can tell you that most of the errors come from using the rule when you did not have to.

The good news is that the quotient rule is not a new idea. It is the product rule and a bit of algebra wearing a different coat, and that insight is what lets you skip it so often. Let us start by getting the rule itself solid, then earn the right to avoid it.

First, a trap: you cannot divide the derivatives

When you meet the product rule or the quotient rule for the first time, a natural thought arises: surely the derivative of f divided by g is just f′ divided by g′? It would be so convenient. It is also wrong, and a two-line test shows it.

Take f = x² and g = x. Their quotient is x²/x = x, whose derivative is clearly 1. But f′/g′ is 2x/1 = 2x. At x = 3, the true slope is 1 and the “divide the derivatives” answer is 6. They disagree, so the shortcut is false. Derivatives of fractions need their own rule, and that rule is where the extra terms in the formula come from.

The rule in a picture

The formula has four ingredients: the top f, the bottom g, and their derivatives f′ and g′. A well-known chant helps you keep them in order. Call the top “high” and the bottom “low”:

Say it aloud a few times: low d-high minus high d-low, over low squared. It sounds silly, but it works, and it pins down the part that hurts students most. The first product in the numerator is the derivative of the top times the bottom. Swap the order and the sign of your whole answer flips.

A full worked example

Let us differentiate y = (x² + 1) / (x − 3) with the rule, slowly. I always begin by writing the four ingredients in a small table, because it prevents most slips.

Now fill in the formula:

• Numerator: f′g − fg′ = 2x(x − 3) − (x² + 1)(1).
• Expand: 2x² − 6x − x² − 1.
• Simplify: x² − 6x − 1.
• Denominator: (x − 3)², kept as a square.

So the derivative is (x² − 6x − 1) / (x − 3)². At x = 2, that is (4 − 12 − 1)/1 = −9. I checked it against a numerical slope in the code later in this article, and it matches. Notice that the top of the answer will not factor nicely and the bottom will not cancel. This is a fraction that genuinely needs the rule, and we will use it as our example of “a case you cannot skip.”

Where the rule comes from

Understanding the origin removes the fear. Suppose y = f/g. Multiply both sides by g and you get y · g = f. That is a product of two functions, y and g, and we know how to differentiate a product:

Now solve for y′. Move the y·g′ term across and divide by g:

Put y = f/g back in, so the numerator becomes f′ − (f/g)g′. Multiply the top and bottom of the whole fraction by g to clear the little fraction, and you get exactly (f′g − fg′)/g². That is the quotient rule, derived in four lines from the product rule.

There is a second way to see it. Write f/g as f · g⁻¹, a product with a negative power. The product rule gives f′g⁻¹ + f · (−1)g⁻²g′, and putting that over a common denominator gives the same formula again. The practical meaning is liberating: anything the quotient rule can do, you can also do with the product rule and a negative exponent. If you only ever remember one of the two, the product rule is the safer one to keep.

Why you can often skip it

Here is the heart of this article. The quotient rule is the tool for the hardest case, where a fraction has a complicated top and a complicated bottom that do not simplify. But a large share of the fractions you will meet in homework, exams and real formulas are easier than that. They can be rewritten, and once rewritten they need only the power rule and the constant multiple rule from earlier lessons.

I use a four-question checklist. Ask the questions in this order and stop at the first “yes”.

Shortcut 1: the denominator is just a number

If you are dividing by a constant, you are really multiplying by a constant, and the constant multiple rule says a number out in front simply comes along for the ride.

Shortcut 2: the numerator is a number, or the denominator is a single power of x

Fractions like 12/x² are secretly powers with negative exponents. Rewrite and use the power rule.

The same idea handles a numerator with several terms over a single power. Split it into separate fractions first.

Shortcut 3: split and cancel

When the denominator is a single term, you can divide it into every term of the numerator and cancel as you go.

Shortcut 4: factor the top and cancel

Sometimes the numerator and denominator share a factor, and the fraction collapses to something simple. Always look for this before reaching for the rule.

A small point of honesty here. The original function has a hole at x = 3, because you cannot divide by zero, so the neat answer applies at every x except 3. For most coursework that is fine, but if your question is about behaviour exactly at x = 3, remember the hole.

Shortcut 5: use a negative exponent with the product rule

This is the “quotient rule in disguise” trick. If the denominator is a simple power but the top is complicated, rewrite the fraction as a product with a negative exponent and use the product rule. Many students find it less error-prone than memorising the quotient formula, because the product rule has no minus sign to flip.

A side-by-side comparison

Here is a summary you can keep. Each fraction is paired with the quickest method and the final answer.

Five of the seven rows needed no quotient rule at all. In my experience that ratio is about right for typical textbook sets, and it is one reason I tell students to try simplification first.

Proof by computer: both routes agree

I do not like asking you to trust me, so I tested every row. The program below computes each derivative three ways at x = 2: a numerical slope from two nearby points, the quotient rule applied mechanically, and the shortcut answer from the table.

def slope(f, x, h=1e-6):
    return (f(x + h) - f(x - h)) / (2 * h)          # numeric derivative

def quotient_rule(f, fp, g, gp, x):
    """(f/g)' = (f'g - f g') / g^2"""
    return (fp(x) * g(x) - f(x) * gp(x)) / g(x) ** 2

# name, f, f', g, g', shortcut answer, can we skip the quotient rule?
cases = [
    ("(x^3 + 4x) / 5",   lambda x: x**3 + 4*x,    lambda x: 3*x**2 + 4,       lambda x: 5,      lambda x: 0,     lambda x: (3*x**2 + 4) / 5,  "skip: constant denominator"),
    ("12 / x^2",         lambda x: 12,            lambda x: 0,                lambda x: x**2,   lambda x: 2*x,   lambda x: -24 / x**3,        "skip: write as 12x^-2"),
    ("(4x^2+3x+2) / x^2",lambda x: 4*x**2+3*x+2, lambda x: 8*x+3,            lambda x: x**2,   lambda x: 2*x,   lambda x: -3 / x**2 - 4 / x**3, "skip: split into terms"),
    ("(x^3+5x^2-7x) / x",lambda x: x**3+5*x**2-7*x, lambda x: 3*x**2+10*x-7,  lambda x: x,      lambda x: 1,     lambda x: 2*x + 5,           "skip: split and cancel"),
    ("(x^2 - 9) / (x-3)",lambda x: x**2 - 9,      lambda x: 2*x,              lambda x: x - 3,  lambda x: 1,     lambda x: 1,                 "skip: factor and cancel"),
    ("(x^2 + 1) / (x-3)",lambda x: x**2 + 1,      lambda x: 2*x,              lambda x: x - 3,  lambda x: 1,     lambda x: (x**2 - 6*x - 1) / (x - 3)**2, "use the rule"),
    ("5x / (x^2 + 1)",   lambda x: 5*x,           lambda x: 5,                lambda x: x**2+1, lambda x: 2*x,   lambda x: 5*(1 - x**2) / (x**2 + 1)**2, "use the rule"),
]

x = 2
print(f"At x = {x}")
print(f"{'function':<22}{'numeric':>10}{'quotient':>10}{'answer':>10}   note")
for name, f, fp, g, gp, ans, note in cases:
    num = slope(lambda t: f(t) / g(t), x)
    q = quotient_rule(f, fp, g, gp, x)
    print(f"{name:<22}{num:>10.4f}{q:>10.4f}{ans(x):>10.4f}   {note}")

Output:

At x = 2
function                 numeric  quotient    answer   note
(x^3 + 4x) / 5            3.2000    3.2000    3.2000   skip: constant denominator
12 / x^2                 -3.0000   -3.0000   -3.0000   skip: write as 12x^-2
(4x^2+3x+2) / x^2        -1.2500   -1.2500   -1.2500   skip: split into terms
(x^3+5x^2-7x) / x         9.0000    9.0000    9.0000   skip: split and cancel
(x^2 - 9) / (x-3)         1.0000    1.0000    1.0000   skip: factor and cancel
(x^2 + 1) / (x-3)        -9.0000   -9.0000   -9.0000   use the rule
5x / (x^2 + 1)           -0.6000   -0.6000   -0.6000   use the rule

All three columns agree on every row. That confirms two things at once: the shortcuts are not tricks, they are legitimate, and the quotient rule gives the same value whether it was needed or not. The only difference is how much work it took to get there.

When you really cannot skip it

Now the fair half of the story. Some fractions have a complicated top and a complicated bottom that share nothing, and rewriting them does not help. In that case, use the rule, and use it with confidence. Here are the signs:

• The denominator has more than one term and does not cancel. For instance, x² + 1 or x − 3 sitting underneath something that is not a multiple of it.
• The top is not a constant, and nothing factors. You are stuck with two live functions.
• Trigonometric or exponential parts are involved. The derivative of (sin x)/x is (x cos x − sin x)/x², and there is no simpler route.

Here is my method for these cases, which has saved me from countless slips:

• Write the four-row table of f, f′, g and g′ before doing anything else.
• Put brackets around everything when you substitute, especially around g and f in the subtracted term, since a missing bracket is how minus signs go wrong.
• Expand the numerator only and collect like terms. Leave the denominator as a square, because a factored denominator is usually easier to read and to work with later.
• Check one value numerically. Pick an easy x, evaluate your formula, and compare with a rough slope between two nearby points.

The quotient rule in real life

Fractions of changing quantities appear everywhere: cost per unit, speed, electrical current, pressure and the concentration of a medicine. Click a tab below to see five examples, and notice which ones can skip the rule. This is the whole theme of the article in action.

Checking the real examples with numbers

I checked the five examples from the tabs with the same approach: compute the derivative with the shortcut or the rule, then compare with a purely numerical slope.

def slope(f, x, h=1e-6):
    return (f(x + h) - f(x - h)) / (2 * h)

print("Average cost per unit  AC = (500 + 2q) / q  at q = 50")
print("  skip method  -500/q^2          =", round(-500 / 50**2, 4), "dollars per extra unit")
print("  numeric slope                  =", round(slope(lambda q: (500 + 2*q) / q, 50), 4))
print("Average speed  (40t + 5t^2) / t  at t = 2 h")
print("  skip method  derivative of 40+5t =", 5, "km/h per hour")
print("  numeric slope                  =", round(slope(lambda t: (40*t + 5*t**2) / t, 2), 4))
print("Ohm's law  I = 12 / R  at R = 4 ohms")
print("  skip method  -12/R^2           =", round(-12 / 4**2, 4), "A per ohm")
print("  numeric slope                  =", round(slope(lambda r: 12 / r, 4), 4))
print("Boyle's law  P = 100 / V  at V = 5 L")
print("  skip method  -100/V^2          =", round(-100 / 5**2, 4), "kPa per litre")
print("  numeric slope                  =", round(slope(lambda v: 100 / v, 5), 4))
C = lambda t: 5 * t / (t**2 + 1)
Cp = lambda t: 5 * (1 - t**2) / (t**2 + 1) ** 2
print("Drug concentration  C = 5t / (t^2 + 1)  (quotient rule needed)")
for t in (0.5, 1, 3):
    print(f"  t = {t:<3}  C = {C(t):.3f}   rule C' = {Cp(t):+.4f}   numeric = {slope(C, t):+.4f}")

Output:

Average cost per unit  AC = (500 + 2q) / q  at q = 50
  skip method  -500/q^2          = -0.2 dollars per extra unit
  numeric slope                  = -0.2
Average speed  (40t + 5t^2) / t  at t = 2 h
  skip method  derivative of 40+5t = 5 km/h per hour
  numeric slope                  = 5.0
Ohm's law  I = 12 / R  at R = 4 ohms
  skip method  -12/R^2           = -0.75 A per ohm
  numeric slope                  = -0.75
Boyle's law  P = 100 / V  at V = 5 L
  skip method  -100/V^2          = -4.0 kPa per litre
  numeric slope                  = -4.0
Drug concentration  C = 5t / (t^2 + 1)  (quotient rule needed)
  t = 0.5  C = 2.000   rule C' = +2.4000   numeric = +2.4000
  t = 1    C = 2.500   rule C' = +0.0000   numeric = -0.0000
  t = 3    C = 1.500   rule C' = -0.4000   numeric = -0.4000

The first four were handled by shortcuts and the last needed the rule. The two columns agree each time. The drug example is worth a second look, because it shows what a derivative can tell you that a table of values cannot: the slope is positive at t = 0.5, exactly zero at t = 1, and negative at t = 3. A slope of zero between a rising and a falling stretch marks the peak.

And here is how fast average cost slows down as production grows in the first example. The slope is −500/q², so it flattens quickly:

Common mistakes and how to avoid them

• Dividing the derivatives. (f/g)′ is not f′/g′. We showed this with x²/x earlier.
• Swapping the order in the numerator. It is f′g − fg′, derivative of the top first. Reversed, your whole answer has the wrong sign.
• Forgetting to square the denominator. The bottom of the answer is g², not g.
• Squaring only part of the denominator. If g = x − 3, the denominator is (x − 3)², which is not x² − 9.
• Losing a minus sign while expanding. In f′g − fg′, the minus applies to the whole product fg′. Use brackets.
• Using the rule when a shortcut was available. It is not wrong, just slower and riskier. Run the four-question checklist first.
• Ignoring the hole after cancelling. After simplifying (x² − 9)/(x − 3) to x + 3, remember x = 3 is still excluded from the original function.
• Not simplifying the final answer. Leaving an expanded numerator uncollected costs marks and hides useful structure, such as where the slope is zero.

A quick word on logarithmic differentiation

For fractions that are really long chains of multiplications and divisions, such as a product of five factors over three others, there is a third strategy: take logarithms of both sides first. That turns products and quotients into sums and differences, which are far easier to differentiate. It needs the rules for logarithms and the chain rule, so it belongs to a later lesson, but it is worth knowing it exists. It is the polite way of saying that the quotient rule is not the only tool in the box.

Practice problems with answers

Try each one on paper first, ideally running the checklist before you decide on a method. Then tap to check.

Test yourself: five quick questions

Tap a question to reveal the answer. The correct option is marked.

Frequently asked questions

The post The Quotient Rule, and Why You Can Often Skip It appeared first on Learn With Examples.

The rule in one line

Here is the whole thing. If you have a function f(x) = xⁿ, then its derivative is:

That is the power rule. It turns x² into 2x, x³ into 3x², and x¹⁰ into 10x⁹. It is the first rule most students use to differentiate anything, and it handles an enormous share of the problems you will meet: polynomials, roots, reciprocals and many formulas from physics and economics.

I have taught this to students for many years, and the pattern I see is always the same. The ones who only memorise the rule are fine until the first problem that looks slightly different. The ones who understand where the “n” and the “minus one” come from never get stuck, because they can rebuild the rule on the spot.

First, what is a derivative?

Imagine you are driving and glance at the speedometer. It says 60 km/h. That number is a derivative. It is the rate at which your position is changing, right now, at this instant, not your average over the whole trip.

Graphically, the derivative is the slope of a curve at a single point. For a straight line, the slope is the same everywhere: rise over run. For a curve, such as y = x², the steepness changes as you move along it. Near x = 0 it is almost flat. Further right it climbs more steeply. The derivative gives you the exact steepness at any chosen point.

So when we say “the derivative of x² is 2x”, we mean: at any x, the slope of the curve is twice that x. At x = 3 the slope is 6. At x = 10 it is 20. The curve gets steeper the further you go, and the rule tells you exactly how steep.

Using the rule: three steps

Before we get to why, let us make sure the mechanics are comfortable. There are only three steps.

• Step 1. Identify the exponent n.
• Step 2. Multiply by n.
• Step 3. Subtract 1 from the exponent.

Why does it work? Four ways to see it

Now the real question. I will show you four separate explanations. They all lead to the same place, and different people find different ones convincing. Read them all if you can, and treat the one that clicks as your own.

Explanation 1: just try it with numbers

The honest definition of the derivative is: take two points on the curve that are very close together, draw the line through them, and work out its slope. Then slide the second point closer and closer until it practically sits on top of the first. The slope the lines settle towards is the derivative.

That slope, for two points a small distance h apart, is called the difference quotient: (f(x + h) − f(x)) / h. We do not need any theory to compute it. Here is a program that tries it at x = 3, for the powers 2, 3, 4 and 5, with the gap h shrinking each time:

def diff_quotient(f, x, h):
    """Slope of the line through (x, f(x)) and (x+h, f(x+h))."""
    return (f(x + h) - f(x)) / h

x = 3
print(f"{'h':>8} | " + " | ".join(f"x^{n}" .rjust(9) for n in (2, 3, 4, 5)))
for h in (1, 0.1, 0.01, 0.001, 0.0001):
    row = [diff_quotient(lambda t, n=n: t**n, x, h) for n in (2, 3, 4, 5)]
    print(f"{h:>8} | " + " | ".join(f"{v:9.4f}" for v in row))
print(f"{'power rule':>8} | " + " | ".join(f"{n * x**(n-1):9.4f}" for n in (2, 3, 4, 5)))

This is the output I got:

       h |       x^2 |       x^3 |       x^4 |       x^5
       1 |    7.0000 |   37.0000 |  175.0000 |  781.0000
     0.1 |    6.1000 |   27.9100 |  113.5210 |  432.9151
    0.01 |    6.0100 |   27.0901 |  108.5412 |  407.7090
   0.001 |    6.0010 |   27.0090 |  108.0540 |  405.2701
  0.0001 |    6.0001 |   27.0009 |  108.0054 |  405.0270
power rule |    6.0000 |   27.0000 |  108.0000 |  405.0000

Read down any column. For x³, the slope of the line starts at 37 when h is 1, drops to 27.91, then 27.09, then 27.009, closing in on 27. The bottom row is the power rule’s prediction, 3 × 3² = 27. The same story holds for every column: 6, 27, 108 and 405 are exactly what n · 3ⁿ⁻¹ gives. No proof needed, just a calculator.

This is a real piece of evidence. But it does not say why the answer is n · xⁿ⁻¹ and not something else that happens to match a few numbers. For that, we need the pictures and the algebra.

Explanation 2: the growing square

Picture a square whose sides are each x units long. Its area is x². Now let the sides grow by a tiny amount, call it h, and ask: how much extra area did that add?

Look at the new area as three new pieces. A strip along the right side, which is x long and h wide, so its area is x·h. A strip along the top, with the same area, x·h. And a tiny square in the corner, h by h, with area h².

The total added area is 2xh + h². Divide by h to get the growth per unit of nudge: 2x + h. Now let h shrink towards zero. The leftover h disappears, and we are left with 2x. That is the derivative of x².

Notice why the corner does not matter. If h is 0.01, the strips have area 0.01x each, but the corner is only 0.0001. As h gets smaller, the corner shrinks much faster than the strips. In the limit it vanishes, and only the strips remain. The “2” in 2x is simply the number of strips. There were two edges that could grow, so we got two strips.

Now try a cube with side x. Its volume is x³. Grow each side by h, and the cube sprouts new material on three faces: the right, the top and the front. Each is a flat slab of area x·x = x² and thickness h, so each slab adds x²h. Three slabs make 3x²h. Everything else (the thin edges and the corner cube) involves h² or h³ and vanishes in the limit. So the derivative of x³ is 3x².

Here is the pattern emerging: for a shape with n dimensions each of length x, there are n faces that can grow. Each new slab has size xⁿ⁻¹ (every dimension except the one that grew) times the thickness h. So n slabs of size xⁿ⁻¹h, or n·xⁿ⁻¹ per unit of h. The “n” counts the faces. The “n − 1” counts the dimensions that stayed the same size.

Explanation 3: the algebra (binomial expansion)

Now the same idea in symbols. We want the difference quotient of xⁿ:

The binomial theorem tells us that when you multiply out (x + h)ⁿ, you get:

• xⁿ (the original term),
• plus n · xⁿ⁻¹ · h,
• plus a pile of terms that each contain h² or a higher power of h.

Subtract xⁿ and the original term disappears. Divide by h and every remaining term loses one factor of h:

• The middle term becomes n · xⁿ⁻¹ with no h left.
• Every other term still has at least one h in it.

Now shrink h to zero. All the terms that still have an h vanish. What remains is exactly n · xⁿ⁻¹. That is a complete proof of the power rule for whole-number exponents, and it is only five lines long.

For x² the expansion is x² + 2xh + h², the same three pieces as the square picture. For x³ it is x³ + 3x²h + 3xh² + h³. The coefficient in front of the h term is always n.

Explanation 4: “n ways to choose”

This one is my favourite because it needs no formula at all. Write xⁿ as n copies of x multiplied together:

Now replace x by x + h in every factor. You are multiplying out (x + h)(x + h)(x + h)… and, from each bracket, you must pick either the x or the h. Picking all x gives back xⁿ. Picking exactly one h and the rest x gives a term with a single h and (n − 1) x’s: xⁿ⁻¹h. How many ways can you do that? You can choose the h from the first bracket, or the second, or the third, up to the n-th. That is n ways.

So there are n terms of the form xⁿ⁻¹h. Everything that picks two or more h’s has h² or more, and vanishes in the limit. The change is n · xⁿ⁻¹ · h, and the rate is n · xⁿ⁻¹.

The rule for more than a single power

Real functions are rarely a lone xⁿ. Luckily, three small companion rules extend the power rule to cover almost every polynomial you will see.

The constant rule fits the power rule too. A constant like 9 is 9x⁰, because x⁰ equals 1. Apply the rule: 9 · 0 · x⁻¹ = 0. Nothing changes, so the rate of change is zero, just as you would expect from a flat horizontal line.

Here is a small program that applies the rule to a whole polynomial, term by term, with each term stored as a (coefficient, exponent) pair:

def derivative_power(coef, n):
    """d/dx of coef * x**n  ->  (coef * n) * x**(n - 1)"""
    return coef * n, n - 1

# 3x^4 - 5x^2 + 7x - 9, written as (coefficient, exponent) pairs
poly = [(3, 4), (-5, 2), (7, 1), (-9, 0)]
print("f(x)  =", " ".join(f"{c:+d}x^{n}" for c, n in poly))
d = [derivative_power(c, n) for c, n in poly if n != 0]   # constants vanish
print("f'(x) =", " ".join(f"{c:+d}x^{n}" for c, n in d))

Output:

f(x)  = +3x^4 -5x^2 +7x^1 -9x^0
f'(x) = +12x^3 -10x^1 +7x^0

Reading the printout: x¹ is just x and x⁰ is just 1, so this says the derivative of 3x⁴ − 5x² + 7x − 9 is 12x³ − 10x + 7. The 3 × 4 = 12 and 5 × 2 = 10 come straight from the rule, the 7 survives because the derivative of 7x is 7, and the lone −9 has vanished.

Negative and fractional exponents

This is where the power rule really earns its reputation, because it keeps working when the exponent is not a positive whole number. The trick is to rewrite first, then apply the rule.

Look at the first row. After bringing down the exponent −1, the new exponent is −1 − 1, which is −2, not 0. Subtracting one from a negative number makes it more negative. This is the single most common slip in this topic, and we will return to it in the mistakes section.

The square root case is worth pausing on. The derivative of √x is 1/(2√x). That says the square root curve is very steep near zero and flattens as x grows. It matches what you see on a graph: √x shoots up quickly at first and then slows down. The slope of √x at x = 1 is 1/2, and at x = 100 it is only 1/20.

A note on rigour: the algebra proof above works for whole-number exponents. The rule also holds for negative and fractional exponents, but proving that takes extra tools, namely the quotient rule for negative powers and implicit differentiation or logarithms for fractions. You do not need those to use the rule confidently, and the numeric check from Explanation 1 works on any of these cases if you want reassurance.

The power rule in real life

The rule is not just classroom exercise. Here are five places it turns up. Click a tab to explore each one.

Checking the numbers: five real formulas

I took the five examples from the tabs and checked each one two ways: with the power rule, and with a purely numerical slope (the same two-close-points idea as before, using points on both sides of the target). If the power rule is right, the two answers should agree.

import math

def slope(f, x, h=1e-6):
    return (f(x + h) - f(x - h)) / (2 * h)       # centred difference

g = 9.8
print("Falling object  s = 4.9 t^2, at t = 3 s")
print("  power rule  v = 9.8 t       =", round(g * 3, 4), "m/s")
print("  numeric slope               =", round(slope(lambda t: 4.9 * t**2, 3), 4))
print("Circle  A = pi r^2, at r = 5")
print("  power rule  2 pi r          =", round(2 * math.pi * 5, 4))
print("  numeric slope               =", round(slope(lambda r: math.pi * r**2, 5), 4))
print("Sphere  V = 4/3 pi r^3, at r = 5")
print("  power rule  4 pi r^2        =", round(4 * math.pi * 25, 4))
print("  numeric slope               =", round(slope(lambda r: 4/3 * math.pi * r**3, 5), 4))
print("Kinetic energy  E = 1/2 m v^2, m = 1000 kg, at v = 20 m/s")
print("  power rule  m v             =", 1000 * 20, "J per m/s")
print("  numeric slope               =", round(slope(lambda v: 0.5 * 1000 * v**2, 20), 4))
mu = 0.7
d = lambda v: v**2 / (2 * mu * g)
print("Braking distance  d = v^2 / (2 mu g), mu = 0.7")
print("  at 20 m/s =", round(d(20), 1), "m,  at 40 m/s =", round(d(40), 1), "m")
print("  power rule  v / (mu g) at 20 =", round(20 / (mu * g), 3), "m per m/s")
print("  numeric slope                =", round(slope(d, 20), 3))

Output:

Falling object  s = 4.9 t^2, at t = 3 s
  power rule  v = 9.8 t       = 29.4 m/s
  numeric slope               = 29.4
Circle  A = pi r^2, at r = 5
  power rule  2 pi r          = 31.4159
  numeric slope               = 31.4159
Sphere  V = 4/3 pi r^3, at r = 5
  power rule  4 pi r^2        = 314.1593
  numeric slope               = 314.1593
Kinetic energy  E = 1/2 m v^2, m = 1000 kg, at v = 20 m/s
  power rule  m v             = 20000 J per m/s
  numeric slope               = 20000.0
Braking distance  d = v^2 / (2 mu g), mu = 0.7
  at 20 m/s = 29.2 m,  at 40 m/s = 116.6 m
  power rule  v / (mu g) at 20 = 2.915 m per m/s
  numeric slope                = 2.915

Every pair agrees. That is the power rule doing its job on a falling stone, a circle, a sphere, a moving car and a braking distance. One caution on the last one: the braking formula is an idealised model that ignores the driver’s reaction time and assumes a steady friction value of 0.7, so treat the numbers as an illustration, not as a safety guide.

Common mistakes and how to avoid them

• Forgetting to subtract one. The derivative of x⁵ is 5x⁴, not 5x⁵. After you bring the exponent down, the power must come down too.
• Getting the sign wrong with negative exponents. The derivative of x⁻² is −2x⁻³. Subtracting one from −2 gives −3, not −1.
• Mixing up xⁿ and n^x. The power rule needs the variable in the base and a fixed number in the exponent. For 2^x, where the variable is in the exponent, the power rule does not apply. That is an exponential function with a different rule.
• Applying it to something bigger inside. (x² + 1)³ is not 3(x² + 1)². The inside also changes with x, and you need the chain rule to account for it. The power rule alone only works when the base is plain x.
• Treating a constant like a variable. The derivative of 5 is 0, not 5 · x⁻¹. And in a formula like 6x, the 6 is a multiplier, not something you differentiate.
• Not rewriting roots and fractions first. Convert √x, 1/x and 1/x³ into powers of x, and the rule applies directly.
• Dropping the multiplier on the “answer”. The derivative of 7x³ is 21x². You multiply the 7 by the 3, not discard it.

The power rule in reverse

If differentiating lowers the exponent, then undoing it should raise it. That is integration, the reverse process, and its power rule runs backwards: add one to the exponent, then divide by the new exponent.

Check it by differentiating the answer. The derivative of x⁴/4 is (4 · x³)/4 = x³, which is what we integrated. The 4 on top cancelled the 4 underneath, so the two rules are perfect mirror images of one another. The exception n = −1 would mean dividing by zero, and that case leads to the natural logarithm, a story for another article.

Practice problems with answers

Try each one on paper first, then tap to check yourself.

Test yourself: five quick questions

Tap a question to reveal the answer. The correct option is marked.

Frequently asked questions

The post The Power Rule Explained: Why the Exponent Drops Down appeared first on Learn With Examples.

The magic trick, and why it needs engineering

Search-as-you-type looks simple until you ask what the phone is really being asked to do. Every single keystroke is a new question. By the time you type the fourth letter of a name, the phone has answered four separate searches, and it has to finish each one before your finger lands on the next key. Usability researchers have used a rule of thumb for decades: if a response arrives in about a tenth of a second, people experience it as instant. Anything much slower and the interface starts to feel like it is lagging behind you.

Now picture the other side of the problem. A phone might hold a few hundred contacts, but plenty of people have thousands, once you count work contacts, old numbers and group chats. And each contact is not just a name. It has several numbers, an email, a nickname, a company. The search has to be fast, forgiving, and sensible about which result to put first.

I have built search boxes for products of every size over the years, and I always start new engineers with this exact exercise. It teaches normalisation, data structures, ranking and the limits of “just loop over everything” in one small, familiar problem.

The pipeline in one picture

Whatever the app, the path from keystroke to list looks about the same.

Let us walk through the stages in order, starting with the most naive version that works, because the way it fails shows you why each later trick exists.

Step 1: clean up what people type (normalisation)

Before any matching happens, both the typed text and the stored names need to be put into the same shape. People type “jose” for José, they type in lower case, and they put dots and dashes into names such as “Anne-Marie” or “O’Brien”. If you compare raw text, none of those match.

So the first stage lowercases everything, strips accents, replaces punctuation with spaces and splits the name into words. That is called normalisation, and it is the unglamorous step that makes the rest work. Skipping it is the number one reason homemade search feels “broken.”

Below is a first version of the whole search in about twenty lines. It normalises, then checks every contact. A contact matches when every word you typed is the beginning of some word in the name.

import unicodedata, re

def normalise(text):
    """Lowercase, strip accents, turn punctuation into spaces."""
    text = unicodedata.normalize("NFKD", text)
    text = "".join(c for c in text if not unicodedata.combining(c))
    return re.sub(r"[^a-z0-9 ]+", " ", text.lower()).split()   # list of words

def linear_search(contacts, query):
    """Slowest idea that works: look at every contact on every keystroke."""
    q = normalise(query)
    hits = []
    for cid, words in contacts:                  # contacts = [(id, [words...]), ...]
        # every typed word must be the start of some word in the name
        if all(any(w.startswith(t) for w in words) for t in q):
            hits.append(cid)
    return hits

This already behaves like a real contact search. Type “sh” and you find both Onam Sharma (the surname starts with sh) and Shalini Rao (the first name does). Type “on sh” and only Onam Sharma remains, because each typed word must begin some word in the name. Matching at the start of any word is what makes phone searches feel natural: you can type the first or the last name, in any order.

Step 2: watch the results shrink as you type

Here is something satisfying to see. I generated 5,000 made-up contacts (random first and last names) and ran the search above while “typing” the name anit, one letter at a time. My test names lean heavily towards names beginning with A, so the early numbers are larger than a real address book would give, but the shape is what matters.

This shrinking is the whole reason search-as-you-type works so well. The first letter does the least work and returns the longest list, which the phone only needs to display the top of. Every next letter cuts the list further, and by the fourth or fifth letter you are usually looking at a handful of names.

It also hides a second, very practical trick. Every result for “ani” is guaranteed to also be a result for “an”, because anything starting with “ani” also starts with “an”. So instead of searching all contacts again, you can search only the list you already have. I counted the name checks for typing anit across 10,000 contacts: scanning everything on every keystroke took 40,000 checks, while reusing the previous results took 13,852. That is about two-thirds fewer, for a change of a few lines of code.

Step 3: stop scanning, start looking up (the index)

The scan-and-reuse approach is fine for a thousand contacts. But it still touches every candidate on the first keystroke, and the cost grows with the size of the list. Real search systems avoid scanning entirely. They do the hard work once, ahead of time, and build an index, which is a structure designed to answer one specific question quickly. For “which words start with these letters?” the classic answer is a trie.

What is a trie?

A trie (said “try”) is a tree where each step down spells one more letter. All words that share a start share the same path. To find everything starting with “ani”, you walk three steps down and you are standing on the answer: every contact reachable from that spot begins with those letters. You never look at the other thousands of contacts at all.

The cost of a lookup depends on the length of what you typed, not on how many contacts you own. A person with 100 contacts and a person with 100,000 both pay the same few steps for “ani”. That is the property that makes it scale.

Here is a compact version. In this one, every node remembers the set of contacts passing through it, so reading the answer is instant:

class Trie:
    """Every node remembers which contacts pass through it."""
    def __init__(self):
        self.root = {}
    def add(self, word, cid):
        node = self.root
        for ch in word:
            node = node.setdefault(ch, {})
            node.setdefault("#", set()).add(cid)  # '#' holds contact ids
    def prefix(self, text):
        node = self.root
        for ch in text:
            node = node.get(ch)
            if node is None:
                return set()
        return node.get("#", set())

def build(contacts):
    t = Trie()
    for cid, words in contacts:
        for w in words:
            t.add(w, cid)
    return t

def trie_search(t, query_words):
    sets = [t.prefix(w) for w in query_words]
    return set.intersection(*sets) if sets else set()

The trade-off is memory. The trie stores extra information so lookups are fast, which is the oldest bargain in computing: spend space and preparation time to buy speed.

A simpler cousin: the sorted list and binary search

If a trie sounds like more machinery than you want, there is a leaner way to get the same effect. Keep every word in one alphabetically sorted list. All words starting with “ani” sit together in one block, because they sort together. Use binary search to find where that block begins and ends, which takes only about seventeen steps even for a hundred thousand words. If you read our article on heaps and sorted arrays, this is the same “sorted array” idea put to a different job. It uses less memory than a trie and is easier to write, and it is a perfectly good choice.

The numbers: how much faster is an index?

I built all three versions (scan, sorted list with binary search, and trie), checked that they return identical results on 500 random queries, and then timed them. The test is typing the four letters anit, taking the best of five runs, with different address book sizes. These figures come from Python on a cloud machine, so a real phone running compiled code will be quicker in absolute terms. What matters is the ratio.

Look at the first row of the table. With 1,000 contacts, every method finishes in a few milliseconds or less, and nobody would notice a difference. That is the honest takeaway: for a normal-sized address book, even the naive scan is fast enough. The index only earns its keep as the data grows. At 100,000 entries the scan needed about a third of a second to answer four keystrokes, which is long enough to feel sluggish, while the trie needed about a millisecond.

Step 4: searching by phone number

Name search matches the start of words. Number search needs different rules, because people remember numbers differently. They recall the last four digits, or the number without the country code, or they type it with spaces and dashes that were never stored.

The standard solution is to normalise numbers the same way we normalised names. Strip everything that is not a digit, and maybe drop a leading country code such as +91, so that “+91 98765-43210”, “098765 43210” and “9876543210” all become the same string. Then, instead of matching only at the beginning, you let the typed digits match anywhere in the number, or at the end for the “last four digits” habit.

Because digits have only ten possible characters, a similar trie or sorted index works well. For suffix searching, you can index the reversed number so “last four digits” becomes an ordinary prefix lookup.

Step 5: put the most likely person first (ranking)

Finding matches is only half the job. If typing “sha” returns thirty people in alphabetical order, the one you wanted may be on the third screen. So the list has to be ranked.

The idea is to give every match a score from several signals, then sort by score. The signals I always start with are: how well the text matches (the whole name starts with it, a word starts with it, or it appears in the middle), whether you marked the person as a favourite, and how often or how recently you contact them.

from norm import normalise

def score(contact, query):
    """Higher is better. contact = dict(name, favorite, calls)."""
    words = normalise(contact["name"])
    q = normalise(query)
    s = 0
    if " ".join(words).startswith(" ".join(q)):
        s += 100                      # the whole name starts with what you typed
    elif all(any(w.startswith(t) for w in words) for t in q):
        s += 60                       # every typed word starts some word in the name
    elif all(t in " ".join(words) for t in q):
        s += 20                       # typed text appears inside a word
    else:
        return 0
    s += 30 if contact["favorite"] else 0
    s += min(contact["calls"], 50) * 0.5   # frequent contacts float up
    return s

contacts = [
    {"name": "Shalini Rao",   "favorite": False, "calls": 2},
    {"name": "Onam Sharma",   "favorite": False, "calls": 40},
    {"name": "Sharad Verma",  "favorite": True,  "calls": 12},
    {"name": "Ashwin Shah",   "favorite": False, "calls": 5},
    {"name": "Harsh Gupta",   "favorite": False, "calls": 60},
]
query = "sha"
ranked = sorted(((score(c, query), c["name"]) for c in contacts), reverse=True)
for s, name in ranked:
    if s: print(f"{s:6.1f}  {name}")

Running it for the typed text “sha” gives:

 136.0  Sharad Verma
 101.0  Shalini Rao
  80.0  Onam Sharma
  62.5  Ashwin Shah

Notice the order. Sharad Verma is first because his name starts exactly with “sha” and he is a favourite. Shalini Rao is next on pure match quality. Onam Sharma follows, as only the surname starts with “sha”, even though he is called often. Ashwin Shah comes last of the matches, and Harsh Gupta does not appear at all because “sha” never occurs in his name. The weights (100, 60, 20, 30 and the call bonus) are just numbers I chose to demonstrate. In a real product, teams tune them with usage data, and that tuning is the difference between a search that is merely correct and one that feels uncannily helpful.

Step 6: forgiving typos (fuzzy matching)

Thumbs are clumsy and names have many spellings. Is it Anita or Anitha? Sunita or Suneeta? Strict prefix matching treats any typo as a complete failure. Fuzzy matching asks a gentler question: how many small edits would it take to turn what you typed into this name?

The classic measure is edit distance, also called Levenshtein distance, which counts single-letter insertions, deletions and replacements. Here it is in a few lines:

def edit_distance(a, b):
    """Minimum single-letter edits (insert, delete, replace) to turn a into b."""
    prev = list(range(len(b) + 1))
    for i, ca in enumerate(a, 1):
        cur = [i]
        for j, cb in enumerate(b, 1):
            cur.append(min(prev[j] + 1, cur[j-1] + 1, prev[j-1] + (ca != cb)))
        prev = cur
    return prev[-1]

names = ["Anita", "Amit", "Anil", "Ankit", "Anjali"]
typed = "Anitha"           # another common spelling of "Anita"
for name in sorted(names, key=lambda n: edit_distance(typed.lower(), n.lower())):
    print(f"{name:<7} distance {edit_distance(typed.lower(), name.lower())}")

The output:

Anita   distance 1
Amit    distance 3
Anil    distance 3
Ankit   distance 3
Anjali  distance 4

“Anitha” is one deletion away from “Anita”, so Anita comes first, and everything else is at least three edits away. A search that falls back to fuzzy matching only when the exact search finds nothing gives you the best of both: exact results are fast and clean, and typos still find someone.

One small warning from experience: in plain edit distance, swapping two neighbouring letters (typing “Aniat” for “Anita”) costs two edits, not one, even though it is the most common typing mistake. A variant called Damerau-Levenshtein counts a swap as one. Also, comparing your text against every contact is slow, so real systems use fuzzy matching sparingly, usually on a shortlist, or use clever indexes built for it.

A fun relic: the keypad dialer

Many phone dialers let you type digits on the number pad and show contacts whose names could be spelled with those keys. It is the old keypad letter layout (2 for ABC, 3 for DEF and so on) doing a modern job.

KEYS = {"2":"abc","3":"def","4":"ghi","5":"jkl","6":"mno","7":"pqrs","8":"tuv","9":"wxyz"}
LETTER = {ch: d for d, chars in KEYS.items() for ch in chars}

def to_digits(name):
    return "".join(LETTER[c] for c in name.lower() if c in LETTER)

for name in ["Anita", "Amit", "Anil", "Ankit"]:
    print(f"{name:<6} -> {to_digits(name)}")

typed = "26"                          # you pressed 2 then 6 on the dialer
matches = [n for n in ["Anita", "Amit", "Anil", "Ankit", "Rahul"] if to_digits(n).startswith(typed)]
print("typed", typed, "matches", matches)

Anita  -> 26482
Amit   -> 2648
Anil   -> 2645
Ankit  -> 26548
typed 26 matches ['Anita', 'Amit', 'Anil', 'Ankit']

Pressing 2 then 6 matches every name whose first letter lives on the 2 key and whose second letter lives on the 6 key, which is why Anita, Amit, Anil and Ankit all appear. Add a 4 and Ankit drops out, because its third letter, K, sits on the 5 key. Add an 8 and Anil drops out too, leaving Amit and Anita. Working a few of these by hand is a good way to check that you really understand the mapping.

The same idea in five real settings

This pattern appears far beyond contacts. Click a tab to see it in other places.

When the data is not on your phone: debouncing and races

Everything so far searched a local list, where each lookup takes microseconds. Things change when the search has to ask a server, such as a company directory with a million staff or a search box on a website. Now each keystroke could trigger a network request taking 100 to 500 milliseconds, and two new problems appear.

• Too many requests. Typing a seven-letter name could send seven requests in a second. The fix is debouncing: wait until the user pauses for a short time, often 150 to 300 milliseconds, before sending anything. The delay is barely noticeable, and the request count drops sharply.
• Answers arriving out of order. The reply for “an” might arrive after the reply for “ani”, and the screen ends up showing results for the wrong text. The fix is to cancel the old request, or to tag each request with a number and ignore any reply that is not for the latest one.

Phone contact lists are local, so they rarely need debouncing. A server-backed search box almost always does. Knowing which situation you are in is half the design.

Hard cases that break simple search

• Names in other scripts. A contact saved in Devanagari or Arabic script cannot be found by typing Latin letters unless the system stores a transliteration or the user switches keyboards. Many real apps index several forms of the same name.
• Prefixes in surnames. “McDonald”, “de la Cruz” and “Al-Farsi” do not split into words the way English names do. Treat the hyphen and the apostrophe thoughtfully.
• Accent rules differ by language. Stripping the dots from “ü” helps in French and Spanish but may be wrong in German searches, where “ue” is the usual substitute.
• Sorting order. The alphabet itself differs between languages. Sorting by raw character codes puts all capitals before all lowercase letters. Use a locale-aware sort.
• Duplicates. The same person saved twice with slightly different details should usually appear as one result.

Which approach should you choose?

Common mistakes when building search-as-you-type

• Skipping normalisation. Case and accent mismatches make good data look broken.
• Matching anywhere instead of at word starts. Typing “an” returns “Hannah” and “Joanna” before “Anita”, because they contain “an” in the middle. Prefer word-start matches and rank anywhere-matches lower.
• Not ranking at all. An alphabetical list of thirty matches is a correct answer and a poor experience.
• Rebuilding the index on every keystroke. Build once, update when contacts change.
• Doing the work on the main thread. Heavy searching on the thread that draws the screen makes the typing itself stutter.
• Forgetting empty results. Show something helpful, such as “no match” or a fuzzy suggestion, not a blank screen.

Test yourself

Tap a question to reveal the answer. The correct option is marked.

Frequently asked questions

The post How a Phone’s Contact Search Finds Matches as You Type appeared first on Learn With Examples.

How to use this guide

Read the questions in order the first time, because later answers lean on earlier ones. Subnets make more sense after IP addresses, and the “what happens when you type a URL” question ties together half the list. After that, use the headings to jump to whatever you are weakest on.

Questions 1 to 5 cover the foundations. Questions 6 to 11 cover the protocols that make the internet work. Questions 12 to 15 cover devices and routing, and 16 to 20 cover security and troubleshooting.

Part 1: The foundations

Q1 What is a computer network, and what are LAN, WAN and MAN?

A network is two or more devices connected so they can exchange data. Everything else is about scale. A LAN (local area network) covers a small area such as a home, an office floor or a school lab. A MAN (metropolitan area network) spans a city or a campus. A WAN (wide area network) links networks across cities or countries, and the internet is the largest WAN there is.

Q2 Explain the OSI model and the TCP/IP model.

The OSI model is a seven-layer teaching framework for how data travels. Each layer does one job and talks only to the layers directly above and below it. The TCP/IP model is the four-layer version the real internet is built on. Interviewers love asking for both because the layers give you a map for troubleshooting.

A handy memory trick for the OSI layers from the top is: All People Seem To Need Data Processing (Application, Presentation, Session, Transport, Network, Data Link, Physical). In practice, people use the layer numbers as shorthand. When an engineer says “layer 2 problem,” they mean something about switches and MAC addresses. “Layer 3” means IP and routing, and “layer 4” means TCP or UDP ports.

Q3 What is an IP address? What is the difference between IPv4 and IPv6, and between public and private?

An IP address is the logical address of a device on a network, used to route data to it. IPv4 addresses are 32 bits long, written as four numbers such as 192.168.1.10, giving about 4.29 billion possible addresses (2³² is exactly 4,294,967,296). That turned out not to be enough for a world of phones and sensors. IPv6 addresses are 128 bits, which gives roughly 3.4 × 10³⁸ addresses, an almost unimaginable number.

Private addresses are reserved for use inside networks and are not routed on the public internet. The three ranges are 10.0.0.0/8, 172.16.0.0/12 and 192.168.0.0/16. Public addresses are unique across the internet. Because many homes reuse the same private ranges, NAT (Question 13) lets them all share one public address.

Q4 What is a subnet mask and what is subnetting?

A subnet mask tells a device which part of an IP address identifies the network and which part identifies the host. The mask 255.255.255.0 means the first three numbers name the network and the last one names the device. Engineers also write it as CIDR notation: 192.168.10.0/24, where 24 is the count of network bits.

Subnetting means cutting one large network into smaller ones, usually for tidiness, security or to reduce broadcast traffic. The useful formula: a network with a prefix length of n has 2^{(32 − n)} addresses, and two of those are reserved (the network address and the broadcast address), so the usable host count is 2^{(32 − n)} − 2. A /24 gives 254 hosts. A /26 gives 62.

Here is a short Python program that does the arithmetic with the standard ipaddress module. It splits a /24 into four /26 subnets.

import ipaddress

net = ipaddress.ip_network("192.168.10.0/24")
print(net, "has", net.num_addresses, "addresses,", net.num_addresses - 2, "usable hosts")
print("mask:", net.netmask)
print()
print("Split into four /26 subnets:")
for s in net.subnets(new_prefix=26):
    hosts = list(s.hosts())
    print(f"{str(s):<18} hosts {hosts[0]} - {hosts[-1]}   broadcast {s.broadcast_address}")

Output:

192.168.10.0/24 has 256 addresses, 254 usable hosts
mask: 255.255.255.0

Split into four /26 subnets:
192.168.10.0/26    hosts 192.168.10.1 - 192.168.10.62   broadcast 192.168.10.63
192.168.10.64/26   hosts 192.168.10.65 - 192.168.10.126   broadcast 192.168.10.127
192.168.10.128/26  hosts 192.168.10.129 - 192.168.10.190   broadcast 192.168.10.191
192.168.10.192/26  hosts 192.168.10.193 - 192.168.10.254   broadcast 192.168.10.255

Read each line as a mini network: the first usable host, the last usable host, and the broadcast address at the end. Interviewers who ask this question often give you a block like 192.168.10.0/24 and ask you to carve out four equal subnets. The answer is the table above, and the trick is that each /26 holds 64 addresses, so the blocks start at 0, 64, 128 and 192.

Q5 What is the difference between a MAC address and an IP address?

A MAC address is a 48-bit hardware identifier burned into a network card, written like 3C:52:82:AB:12:EF. It is used for delivery within one local network. An IP address is assigned by software or a DHCP server, can change, and is used to route data between networks.

One more detail worth saying: as a packet crosses several routers, the IP addresses at both ends stay the same, but the MAC addresses are rewritten at every hop. That single sentence tells an interviewer you understand how layers 2 and 3 cooperate.

Part 2: The protocols that run the internet

Q6 What is the difference between TCP and UDP?

TCP is connection-oriented. It sets up a connection first, numbers the data, retransmits what is lost and delivers it in order. UDP is connectionless. It just sends packets with no handshake, no retransmission and no ordering guarantee, so it is faster and lighter.

Q7 Explain the TCP three-way handshake.

Before TCP sends any data, the two sides agree to talk. The client sends a SYN (“I want to connect, my starting sequence number is X”). The server answers with SYN-ACK (“Got it, here is mine, Y”). The client finishes with an ACK (“Got yours too”). Now both sides know the other is alive and reachable, and they have agreed on sequence numbers for tracking data.

A bonus line that impresses: a SYN flood attack sends huge numbers of SYNs and never completes the handshake, leaving the server holding half-open connections. Defences include SYN cookies and rate limits.

Q8 What is DNS and how does a lookup work?

DNS, the Domain Name System, is the internet’s phone book. People remember names like example.com, but computers need IP addresses, and DNS translates one into the other.

When your device needs an address, it asks a recursive resolver, usually run by your ISP or a public DNS service. If the resolver does not already have the answer cached, it asks a root server, which points it to the server for the top-level domain (.com, .org, .in). That server points to the domain’s authoritative server, which gives the real answer. The resolver returns it to you and remembers it for the record’s time to live (TTL).

Know the common record types: A (name to IPv4), AAAA (name to IPv6), CNAME (an alias to another name), MX (mail servers) and NS (which servers are authoritative). Plain DNS uses UDP port 53, and TCP port 53 is used for large responses.

Q9 What is DHCP and how does it work?

DHCP, the Dynamic Host Configuration Protocol, automatically gives a device an IP address, subnet mask, default gateway and DNS servers when it joins a network. The exchange is nicknamed DORA: the client broadcasts a Discover, the server sends an Offer, the client sends a Request for that offer, and the server sends an Acknowledge. The address is a lease, not a permanent grant, and the client renews it before it expires.

Q10 What is ARP and why is it needed?

On a local network, devices deliver frames by MAC address, but programs only know IP addresses. ARP, the Address Resolution Protocol, bridges the gap. When your PC wants to reach 192.168.1.1 and does not know its MAC, it broadcasts “Who has 192.168.1.1?” to everyone. The owner replies with its MAC address. Your PC stores the answer in its ARP cache for a while, so it does not need to ask again.

The classic security follow-up is ARP spoofing: an attacker sends false ARP replies so that traffic meant for the gateway flows through their machine. Managed switches can defend against it with features such as dynamic ARP inspection.

Q11 What happens when you type a URL into your browser and press Enter?

Here is the version I recommend you tell, in order:

• 1. The browser checks its caches and the operating system’s cache for a known address.
• 2. DNS lookup. If nothing is cached, it resolves the name to an IP address (Question 8).
• 3. TCP connection. The browser opens a connection to port 443 with the three-way handshake (Question 7).
• 4. TLS handshake. For HTTPS, both sides agree on encryption and the server proves its identity with a certificate.
• 5. HTTP request. The browser sends GET / with headers.
• 6. The server responds with a status code (200 for success) and the page.
• 7. The browser renders the page, requesting images, scripts and styles, often over the same connection.

Mention that on your local network, the first packet needs ARP to find the gateway’s MAC address, and that routers along the way forward the packet by IP. That shows you can connect every layer in one story.

Part 3: Devices and routing

Q12 What is the difference between a hub, a switch and a router?

Q13 What is NAT?

NAT, Network Address Translation, lets many devices with private addresses share one public IP address. The router rewrites the source address and port of outgoing packets and keeps a table. When replies come back, it looks up the table and forwards them to the right internal device. The common form, where ports tell the devices apart, is often called PAT or NAT overload.

Know the side effects. NAT helped the world survive IPv4 exhaustion, and it hides internal addresses from the outside. But it is not a firewall, and it breaks anything that needs inbound connections, which is why we use port forwarding to reach a server inside a home network.

Q14 What is a default gateway, and how does a router choose where to send a packet?

A default gateway is the router a device uses for any destination outside its own subnet. Your laptop does not know how to reach the whole internet, so it hands everything non-local to the gateway.

A router keeps a routing table, a list of destination networks and where to send traffic for each. When a packet arrives, the router finds every entry that matches the destination and picks the most specific one, meaning the longest prefix. That is the longest prefix match. The entry 0.0.0.0/0 is the default route and matches everything, but because it is the least specific, it only wins when nothing better matches.

Here is a tiny router in Python with four routes:

import ipaddress

routes = {
    "0.0.0.0/0":   "ISP (default route)",
    "10.0.0.0/8":  "Head office core",
    "10.1.0.0/16": "Branch office",
    "10.1.2.0/24": "Branch lab network",
}

def lookup(dest):
    ip = ipaddress.ip_address(dest)
    best = None
    for prefix, hop in routes.items():
        net = ipaddress.ip_network(prefix)
        if ip in net and (best is None or net.prefixlen > best[0].prefixlen):
            best = (net, hop)
    return best

for dest in ["10.1.2.50", "10.1.9.9", "10.7.7.7", "8.8.8.8"]:
    net, hop = lookup(dest)
    print(f"{dest:<10} matches {str(net):<12} -> {hop}")

Output:

10.1.2.50  matches 10.1.2.0/24  -> Branch lab network
10.1.9.9   matches 10.1.0.0/16  -> Branch office
10.7.7.7   matches 10.0.0.0/8   -> Head office core
8.8.8.8    matches 0.0.0.0/0    -> ISP (default route)

Look at the first line. The address 10.1.2.50 matches the /8, the /16 and the /24, and the /24 wins because it is the most specific. The last line matches only the default route, which sends it to the ISP.

One more thing interviewers like: routes can be static (written by an administrator) or dynamic (learned from other routers using protocols such as OSPF inside an organisation and BGP between internet providers). Static is simple and predictable. Dynamic adapts when links fail.

Q15 What is a VLAN and why would you use one?

A VLAN, virtual LAN, splits one physical switch into several logical networks. Devices in different VLANs cannot talk directly even if they are plugged into the same switch, and they need a router (or a layer 3 switch) to communicate. Frames carry a VLAN tag following the 802.1Q standard when they travel between switches.

The reasons to use them are security (keep guest Wi-Fi away from the finance team), performance (smaller broadcast domains) and tidiness (group devices by function, not by cable location).

Part 4: Security and troubleshooting

Q16 What is a firewall?

A firewall filters network traffic by rules, allowing or blocking it based on things like source, destination, port and protocol. A stateless firewall checks each packet alone against the rules. A stateful firewall tracks connections, so it knows that an incoming packet is a reply to something you started, and it allows it automatically. Modern “next-generation” firewalls also look at applications and can detect certain threats.

The principle to quote is default deny: block everything, then allow only what you need. That way, a mistake leaves you too closed instead of too open.

Q17 What is the difference between HTTP and HTTPS?

HTTP sends web traffic in plain text on port 80, so anyone on the path can read or alter it. HTTPS is the same protocol carried inside TLS encryption, usually on port 443. TLS gives you three things: confidentiality (nobody can read it), integrity (nobody can change it unnoticed) and authentication (a certificate proves you reached the real site).

If asked how the handshake works, a safe outline is: the client says hello and lists its supported options, the server answers with its certificate, the client checks the certificate against trusted authorities, and both sides agree on session keys, which then encrypt the data. Modern TLS also uses temporary keys, so recorded traffic stays unreadable even if a server key leaks later.

Q18 What is a VPN, and how is it different from a proxy?

A VPN creates an encrypted tunnel between your device and a VPN server, and typically sends all your traffic through it. Companies use VPNs so remote staff can reach internal systems as if they were in the office. A proxy is a server that makes requests on your behalf, usually for particular applications, and it may not encrypt anything. A VPN protects the whole path to the VPN server. A proxy mainly changes where the request appears to come from or adds filtering and caching.

Be careful with claims. A VPN hides your traffic from the local network and your ISP, but the VPN provider can see it instead. It does not make you anonymous.

Q19 How do you troubleshoot “the internet is not working”? What do ping and traceroute do?

Ping sends a small ICMP echo request and waits for a reply. It tells you whether a host is reachable and how long the round trip takes. Traceroute (called tracert on Windows) lists the routers on the path by sending packets with increasing TTL values, so each router in turn reports back when the TTL runs out. It shows you where along the path things slow down or stop.

My own habit is a four-step ladder, working outward until something fails:

• Ping 127.0.0.1. If this fails, the network software on your own machine is broken.
• Ping the default gateway. If this fails, the problem is local: cable, Wi-Fi, switch or gateway.
• Ping a public IP such as 8.8.8.8. If this works but names fail, you are nearly at DNS. If it fails, the problem is beyond the gateway, maybe the ISP.
• Ping a domain name. If the IP works but the name does not, DNS is the culprit.

Also check the basics early: is the address a 169.254 one (DHCP failed), is the cable plugged in, and did anything change recently? In my experience, most “mysterious” outages are one of those three.

Q20 What is the difference between bandwidth, latency and throughput?

Bandwidth is the maximum amount of data a link can carry per second, like the number of lanes on a highway. Latency is how long a single piece of data takes to get there, like the time for one car to complete the trip. Throughput is the amount that actually gets through in practice, which is usually lower than bandwidth because of congestion, errors and protocol overhead.

It is worth adding that a bigger plan does not fix a latency problem. When someone says “my connection is fast but pages feel slow,” look at latency, DNS and packet loss before blaming bandwidth.

How the same question changes by role

The twenty questions above are common, but interviewers adjust the depth to the job. Click a tab to see what each role is really probing for.

Common mistakes candidates make

• Reciting definitions with no example. A definition shows memory. An example shows understanding.
• Mixing up layers. Saying a switch “routes” or that IP is at layer 2 is the quickest way to lose credibility.
• Confusing the port with the protocol. Port 443 is conventionally HTTPS, but a program can use any port.
• Forgetting that DNS uses UDP. Most do not mention that TCP is used for large answers and zone transfers.
• Claiming NAT is a firewall. It hides addresses as a side effect. It does not enforce a security policy.
• Guessing in troubleshooting questions. Describe your method out loud. Interviewers care more about how you think than what you land on.
• Bluffing. “I have not used that, but here is how I would find out” works far better than inventing an answer.

A study plan that actually works

If you have two weeks, here is the order I recommend. Spend the first few days on layers, IP, subnets and TCP versus UDP, and practise subnetting by hand until the powers of two come automatically. In the middle days, trace one full web request (Question 11) on paper, then capture a real one using a packet analyser such as Wireshark and watch the DNS query, handshake and TLS hello appear. In the last days, build a small lab. The free Packet Tracer simulator from Cisco Networking Academy is excellent for this, and you can also run a few Linux virtual machines. Break something on purpose, then fix it using the ping ladder.

Nothing replaces hands-on practice. The people who stand out are the ones who say “when I set this up at home, I noticed…” because that cannot be memorised.

Test yourself: five-question mock round

Tap a question to reveal the answer. The correct option is marked.

Frequently asked questions

The post Top 20 Networking Interview Questions with Plain-English Answers appeared first on Learn With Examples.

The problem ports solve

Think about what happens when you open a website while a video call is running, a music app is streaming and your email is syncing in the background. All of that traffic arrives at your laptop through one network connection, addressed to one IP address. Yet the website data ends up in the browser, the call audio in the call app, and the emails in the mail client. They never get mixed up.

The IP address cannot do that job, because it only identifies the machine. You need a second number that identifies the program or service on that machine. That number is the port.

An everyday analogy: the apartment building

Imagine you send a parcel to “42 Mall Road.” The courier finds the building, which is what an IP address does. But 42 Mall Road is an apartment block with a hundred flats. Without a flat number the parcel sits in the lobby. Write “Flat 443” on it and the right resident receives it.

The analogy also explains why some flat numbers are famous. Everybody knows the front desk is Flat 1, so visitors do not have to ask. Ports work the same way. If every web server on Earth listens on 443 for secure traffic, your browser can reach any of them without being told the number.

Where ports live: the transport layer

Ports belong to the transport layer of networking, and the two protocols that use them are TCP and UDP.

• TCP sets up a connection first, guarantees delivery and keeps the data in order. Web pages, email and file transfers use it.
• UDP just fires packets without a handshake or guarantees. It suits things where speed matters more than perfection, such as DNS lookups, live video, voice calls and games.

TCP port 53 and UDP port 53 are different doors that happen to share a number. Each protocol has its own full set of 65,536 ports, which is why you will see written forms like 443/tcp and 443/udp. In fact, modern HTTP/3 uses UDP port 443, while ordinary HTTPS uses TCP port 443.

Why exactly 0 to 65535?

The port number is stored in 16 bits in the TCP and UDP headers. Sixteen bits give 2¹⁶, or 65,536 possible values, counting from zero. That is the whole reason for the limit. Port 0 is reserved and is not used for ordinary traffic, so in practice you work with 1 to 65535.

The three port ranges

The internet’s numbering authority, IANA, divides the 65,536 numbers into three zones.

One caution: operating systems do not all follow the IANA suggestion for temporary ports. Modern Windows uses 49152 to 65535 by default. Linux commonly uses 32768 to 60999, and you can read the exact range in /proc/sys/net/ipv4/ip_local_port_range. That is a tuning detail worth knowing when you debug connection limits.

The ports every beginner should memorise

These are conventions, not laws. Nothing in physics forces a web server to listen on 80. It is simply the agreed default, and the agreement is what lets browsers and servers find each other without negotiation.

Why 80, 443 and 22 matter so much

Port 80: the original web

When you type http://example.com, your browser quietly adds :80. This is plain, unencrypted HTTP. Anyone on the path, such as the coffee shop Wi-Fi owner, can read or modify what travels over it. Today port 80 mostly survives to redirect visitors to the secure version.

Port 443: the secure web

When you type https://example.com, the browser adds :443. The same HTTP conversation happens, but wrapped in TLS encryption. This is the port that matters most for any site you run. Search engines, browsers and users all expect it. If you want to run a site that ranks and earns trust, port 443 must be open, and the certificate behind it must be valid.

Port 22: the administrator’s door

SSH is how people manage servers from afar. It is also, unfortunately, one of the most attacked doors on the internet. If you have ever put a server online and checked its logs an hour later, you have seen strangers hammering port 22 with guessed passwords. We will come back to what to do about that.

You can use any port in a URL by writing it after a colon. https://example.com:8443 tells the browser to use 8443 instead of 443. Developers do this constantly, and the reason is a ritual every programmer knows: local test servers run on ports like 3000, 5000, 8000 or 8080, so that they do not clash with the real web ports.

The key idea: a connection is four numbers

Here is the part that clears up the most confusion. When your browser connects to a web server, it uses a port on both sides. The server side is the well-known one (443). Your side is a temporary number the operating system picks from the dynamic range.

A TCP connection is uniquely identified by four values: your IP address, your port, the server’s IP address and the server’s port. Change any one of them and it is a different connection. That is how you can open ten tabs to the same website, all talking to the same server on the same port 443, without the answers colliding. Each tab uses a different source port on your side.

See it for yourself in about twenty lines of Python

I like proving things in code rather than asking you to believe them. The script below starts a small server, connects to it three times, and prints which ports each side used. Then it tries to start a second server on the same port.

import socket, threading

# 1. A server listens on one port. We let the OS pick a free one (port 0).
server = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
server.bind(("127.0.0.1", 0))
server.listen()
host, port = server.getsockname()
print("server listening on port", port)

# 2. Three clients connect to that single server port.
def serve():
    for _ in range(3):
        conn, addr = server.accept()
        conn.close()
threading.Thread(target=serve, daemon=True).start()

for n in range(3):
    c = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
    c.connect((host, port))
    print(f"client {n+1}: my port {c.getsockname()[1]} -> server port {c.getpeername()[1]}")
    c.close()

# 3. A second server cannot grab the same port.
dup = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
try:
    dup.bind((host, port))
except OSError as e:
    print("second bind failed:", e.strerror)

This is the output from one run on my machine. Your numbers will differ, because the operating system picks them:

server listening on port 54875
client 1: my port 47676 -> server port 54875
client 2: my port 47684 -> server port 54875
client 3: my port 47700 -> server port 54875
second bind failed: Address already in use

Three things are worth noticing. First, the server uses one port, 54875 in this run, for all three clients. Second, each client received a different temporary port of its own. Third, the attempt to bind a second server to the same address and port failed with “Address already in use.” Only one program can listen on a given port at a time, which explains an error that confuses beginners every single week.

The same idea in five real settings

Click a tab to see ports at work in different places. No reload, nothing to install.

How your router uses ports: NAT in plain English

Your home has one public IP address from your internet provider, yet a dozen devices share it. How do replies find the right device? The router performs NAT, network address translation, and ports are the trick that makes it work.

Suppose your phone (192.168.1.20) opens a connection to a web server from its local port 51000. The router rewrites the outgoing packet so it appears to come from the router’s own public address, using its own chosen port, say 40001. It remembers the pairing in a table. When the reply returns to port 40001, it looks it up and forwards the data to the phone at 192.168.1.20 port 51000. Your laptop doing the same thing would get a different public port.

The numbers in the table are illustrative, since real routers pick their own. Notice that the phone and laptop both used private port 51000 and nothing broke. The router separates them by assigning different public ports.

Port forwarding: the reverse trick

NAT works naturally for connections that start inside your home. Connections that start outside have a problem: the router has no table entry, so it drops them. If you want a friend to join a game server running on your PC, you create a port forwarding rule: “anything arriving on public port 25565 goes to 192.168.1.50, port 25565.” That is exactly what the home-router tab above shows.

Firewalls: deciding which doors are open

A firewall is largely a list of rules about ports. A typical server firewall says: allow incoming 443, allow incoming 22 only from my office, and drop everything else. Because a program can only receive connections if something is listening, closing a port you do not need means that if the software behind it has a flaw, the flaw is unreachable.

When you scan a machine for open ports, there are usually three results:

• Open. A program is listening and accepted the connection.
• Closed. The machine is reachable, but nothing listens there, and it politely refuses.
• Filtered. A firewall dropped the attempt, so you get no answer at all.

The practical rule I teach every junior: every open port is a promise to answer strangers. Keep that list as short as you can.

Commands to see ports on your own machine

Tool options vary slightly between systems and versions, so treat these as reliable starting points. Only scan machines you own or have permission to test. Scanning other people’s networks can break rules or laws.

Limits you can actually hit: running out of ports

Because each outgoing connection needs a temporary port, a single client talking to a single server address and port has a ceiling. With Windows’ default dynamic range of 49152 to 65535, that is 16,384 simultaneous connections. With Linux’s common default of 32768 to 60999, it is 28,232. After that, new connections fail until old ones close.

Most users never see this. Busy servers and load balancers do, and the same arithmetic hits NAT gateways that funnel thousands of devices through one public address. When people say a service “ran out of ephemeral ports,” this is what they mean. The usual fixes are reusing connections (keep-alive and connection pooling), widening the range, or adding more addresses.

Security myths about ports

Myth: changing the SSH port makes it secure

Moving SSH from 22 to something obscure reduces noise in your logs because the lazy scanners that probe only port 22 stop finding you. It does not stop a determined scanner, which checks all ports in minutes. Real protection comes from key-based login instead of passwords, disabling root login, keeping software updated, and restricting who may connect at all. Treat a port change as tidiness, not security.

Myth: a closed port means the machine is invisible

A closed port still reveals that the machine exists. Only a firewall that silently drops traffic hides that, and even then the machine can leak information in other ways.

Myth: databases are safe on their default port because nobody knows the number

Everybody knows the number. MySQL on 3306, PostgreSQL on 5432, Redis on 6379 and MongoDB on 27017 are scanned constantly. A database should normally listen only on a private interface, with the firewall blocking public access.

Common mistakes beginners make

• Confusing the port with the protocol. Port 443 is conventionally HTTPS, but a program can run anything on any port. The convention is not enforced.
• Forgetting TCP versus UDP. Opening TCP 53 in a firewall does nothing for DNS queries that use UDP 53.
• Opening a port without checking what listens behind it. You have just exposed that program to the whole internet.
• Forwarding a port on the router but not opening it on the PC firewall. The traffic arrives and is dropped by the last gate.
• Assuming a number above 1023 needs no thought. Registered ports can be just as dangerous as well-known ones when they hold a database.
• Running a second copy of a server and ignoring “Address already in use.” Something already owns that port.

Test yourself

Tap a question to reveal the answer. The correct option is marked.

Frequently asked questions

The post What Is a Port Number? Why 80, 443 and 22 Matter appeared first on Learn With Examples.