PAPERS-02-005 · Paper — Efficient Depth (GoogLeNet / Inception)

The paper

Efficient Depth (GoogLeNet / Inception) by Szegedy et al. (2014, CVPR 2015) is a landmark because it proved that deep convolutional networks could be both accurate and parameter‑efficient. It introduced a new way to scale depth without exploding the model size.

Before this paper

Before GoogLeNet, state‑of‑the‑art networks relied on simply widening layers and using huge fully‑connected classifiers, which made them memory‑hungry and slow. AlexNet, for example, had tens of millions of parameters and required massive dense layers. Multi‑scale feature extraction was limited to stacking separate networks or using large kernels, both of which increased computation dramatically.

The key idea

The core idea is the Inception module, which runs several convolutions of different receptive fields (1×1, 3×3, 5×5) and a pooling operation in parallel on the same input and concatenates their outputs. Crucially, 1×1 convolutions act as bottlenecks that shrink channel depth before the expensive larger kernels. This design lets the network capture rich, multi‑scale information while keeping the parameter budget low.

How it works

An input tensor first passes through a 1×1 convolution that reduces its channel count, creating a thin “bottleneck” representation. The reduced tensor then feeds separate branches: a 3×3 conv, a 5×5 conv, and a max‑pool followed by another 1×1 conv. Each branch produces feature maps that are concatenated along the depth dimension, forming the module’s output. Stacking many such modules yields a 22‑layer deep network with only about 5 million parameters, roughly 12× fewer than AlexNet, while preserving expressive power.

See it — the Inception module

Diagram of an Inception module showing parallel 1×1, 3×3, 5×5 convolutions and a pooling branch whose outputs are concatenated.

flowchart TB
  X["input"] --> A["1x1 conv"]
  X --> B["1x1 then 3x3"]
  X --> C["1x1 then 5x5"]
  X --> D["pool then 1x1"]
  A --> M["concatenate"]
  B --> M
  C --> M
  D --> M

What they showed

GoogLeNet achieved a top‑5 error of 6.67 % on ILSVRC‑2014, winning the classification challenge. Despite being 22 layers deep, it contained only about 5 million parameters—approximately 12 times fewer than AlexNet—demonstrating that smart architecture can replace brute‑force width.

What it changed

The paper sparked a wave of architecture‑centric research, inspiring later Inception variants, ResNet’s bottleneck blocks, and mobile‑friendly designs like MobileNet. It showed the community that depth and efficiency are not mutually exclusive, leading to a focus on parameter‑wise optimization. Today, the bottleneck‑and‑concatenation pattern is a staple in many state‑of‑the‑art models.

Next: Making Depth Free (ResNet).

The one-line takeaway

Depth can be efficient when you let tiny 1×1 gates open the doors to bigger kernels.

Read it / meet it

Read the original GoogLeNet/Inception paper for the full architectural details and experimental results. Modern deep‑learning libraries (TensorFlow, PyTorch, Keras) include ready‑made Inception blocks, and you’ll encounter their descendants in mobile and vision‑transformer hybrids today.