The Perceptron

The Perceptron, introduced by Frank Rosenblatt in 1958, is the simplest possible model of a neuron: take a set of inputs, multiply each by a weight, add them up, and add one more number called a bias.

Then check whether that sum clears a threshold. If it does, the perceptron fires and outputs 1. If not, it stays silent and outputs 0. That's the entire model: a weighted vote, thresholded.

Why bias matters: without it, the weighted sum is always 0 whenever every input is 0, which forces the decision boundary to pass through the origin. The bias shifts that line freely, so it can separate classes that don't happen to straddle the origin, a real geometric limitation, not just a training nicety.

Geometrically, a perceptron's weights and bias define a straight line, or with more inputs, a flat plane, that cuts the space of possible inputs in two. Everything on one side fires; everything on the other doesn't.

Learning means adjusting the weights and bias so that line ends up in the right place. The perceptron learning rule nudges it a little every time it misclassifies an example; correct answers leave it alone.

Try it yourself: drag the two sliders below until every point is classified correctly. That's exactly what training a perceptron is automating, searching for a weight angle and bias that separate the classes.

A single perceptron can only separate data that's linearly separable, points some straight line can actually divide. Anything more tangled, most famously the XOR pattern, is mathematically impossible for one perceptron to solve, no matter how it's trained.

That limitation, formalized by Minsky and Papert in 1969, is exactly what stacking multiple layers fixes: each layer draws its own line, and combining them can carve out far more complex regions. That combination is a feed-forward network, covered next.