Perceptron

The original artificial neuron: draw one line, and on every misclassified point tilt the line toward fixing it — guaranteed to converge if the data is linearly separable.

8 min read Watch it move Build it

The perceptron (Rosenblatt, 1958) is the original artificial neuron and the ancestor of every neural network today. It draws one straight line to separate two classes. Its learning rule is almost embarrassingly simple: for every point it gets wrong, tilt the line a little toward fixing that mistake.

How a neuron decides

Each input is multiplied by a weight, the products are summed, a bias is added, and the neuron fires +1 if the total is positive, −1 if negative. The weights set the *angle* of the dividing line; the bias *shifts* it so it needn't pass through the origin.

score = w·x + b
ŷ = +1 if score >= 0 else -1

The learning rule

Loop over the training points. When a point is classified correctly, do nothing. When it's wrong, nudge the weights toward that point: w ← w + η·y·x and b ← b + η·y, where y is the true label (+1 or −1) and η is the learning rate. This rotates the boundary so the point is now on (or closer to) its correct side.

1Start with weights at zero (or small random values).
2Pick a training point x with true label y.
3Predict ŷ from the sign of w·x + b.
4If ŷ ≠ y, update: w ← w + η·y·x, b ← b + η·y. Otherwise leave it.
5Repeat over the data until a full pass makes no mistakes.

The convergence theorem

If the two classes are linearly separable — a single straight line can split them perfectly — the perceptron is *guaranteed* to find such a line in a finite number of updates. This is the famous perceptron convergence theorem (Novikoff, 1962).

The XOR limitation

If no straight line can separate the classes — the classic example is XOR — the perceptron never converges; it cycles forever. Minsky and Papert's 1969 book made this limit famous and helped trigger the first 'AI winter'. The fix came later: stack perceptrons into layers (an MLP) and train with backpropagation.

def train(X, Y, eta=1.0, epochs=100):
    w = [0.0] * len(X[0]); b = 0.0
    for _ in range(epochs):
        errors = 0
        for x, y in zip(X, Y):          # y is +1 or -1
            score = sum(wi * xi for wi, xi in zip(w, x)) + b
            if y * score <= 0:           # misclassified
                w = [wi + eta * y * xi for wi, xi in zip(w, x)]
                b += eta * y
                errors += 1
        if errors == 0:                  # separated — done
            break
    return w, b

OperationTimeSpace

Update step · d featuresO(d)O(d)

Convergence · bounded by (R/margin)²finite if separableO(d)

Predict · dot product + signO(d)O(d)

Check yourself

When is the perceptron guaranteed to converge to a separating line?