AIM-02-001-P · Primer — The Artificial Neuron

The idea in one sentence

An artificial neuron multiplies each input by a learned weight, adds all those products plus a bias, then passes the result through a nonlinear activation function to produce one output number.

Why it exists

Real decisions rarely depend on one signal in isolation — a loan approval weighs income, debt, and credit history together, with some factors mattering more than others. The artificial neuron is the smallest computational unit that can do this: it needs a way to combine several inputs, let some inputs matter more than others (weights), have an adjustable baseline tendency (bias), and decide whether the combined evidence is strong enough to "fire" (activation). Biological neurons inspired the name — they sum electrical signals from dendrites and fire only past a threshold — but the artificial version is just arithmetic: a weighted sum followed by a nonlinear squashing step. Stack enough of these units together and you get a network that can approximate far more complex functions than any single neuron could.

How it works

The one equation that matters:

\[z = w_1 x_1 + w_2 x_2 + \dots + w_n x_n + b, \quad y = \mathrm{activation}(z)\]

The first part is just a dot product of the weight vector and the input vector, plus a scalar bias; the second part passes that number through a nonlinearity — here, ReLU, defined as \(\mathrm{ReLU}(z) = \max(0, z)\), which keeps positive values unchanged and clamps anything negative to exactly zero.

Worked example: suppose weights \(= [0.5,\ -0.6,\ 0.1]\), inputs \(x = [4,\ 1,\ -2]\), and bias \(= 0.3\). The dot product is \((0.5 \cdot 4) + (-0.6 \cdot 1) + (0.1 \cdot -2) = 2.0 - 0.6 - 0.2 = 1.2\). Add the bias: \(z = 1.2 + 0.3 = 1.5\). Since \(1.5\) is positive, ReLU leaves it untouched, so the neuron's output is \(1.5\). Now flip the sign of two inputs — \(x = [-4,\ 1,\ 2]\) with the same weights and bias — and the dot product becomes \(-2.0 - 0.6 + 0.2 = -2.4\), so \(z = -2.4 + 0.3 = -2.1\). Because \(z\) is negative, ReLU clamps it to \(0\) regardless of how negative it was: the neuron outputs exactly \(0.0\), discarding the magnitude of the negative signal entirely.

This is the unit that gets tiled thousands of times in a layer (each neuron with its own weight vector and bias, all applied to the same input) and stacked in layers to form a network. Training doesn't touch the formula itself — it only adjusts the numbers inside \(w\) and \(b\) via gradient descent so that the neuron's output moves toward what the loss function wants.

See it — One neuron: weighted inputs summed, biased, then activated

flowchart TD
    X1["x1 = 4"] --> P1["×0.5"]
    X2["x2 = 1"] --> P2["×-0.6"]
    X3["x3 = -2"] --> P3["×0.1"]
    P1 --> SUM["Σ + bias<br/>1.2 + 0.3"]
    P2 --> SUM
    P3 --> SUM
    SUM --> Z["z = 1.5"]
    Z --> ACT["ReLU:<br/>max(0, z)"]
    ACT --> OUT["output = 1.5"]

The one gotcha

The most common misconception: treating the neuron as "just the weighted sum" and forgetting the activation function is a separate, mandatory step. Without a nonlinearity like ReLU, stacking any number of neurons collapses algebraically into one giant linear function — a 50-layer network with no activations computes nothing a single-layer network couldn't, because a weighted sum of weighted sums is still just a weighted sum. The nonlinearity is what lets depth actually add expressive power.

The one-line takeaway

Weighted sum, plus bias, then squashed nonlinearly.

Where you'll meet it

PyTorch's nn.Linear layer followed by torch.relu (or nn.ReLU) computes exactly this per output unit; TensorFlow/Keras's Dense(units, activation='relu') does the same for every neuron in the layer in one call.

Named concepts: artificial neuron