PAPERS-08-003 · Paper — Deep Compression (Pruning)

The paper

The paper “Deep Compression” introduced a landmark method for shrinking neural networks, showing that large models could be made tiny without losing accuracy.

Before this paper

Before this work, neural networks often required hundreds of megabytes of weight storage, making them too large for phones and costly to move in and out of memory. Researchers struggled to deploy state‑of‑the‑art models on mobile and embedded devices because of these size constraints. The community believed that most of the weights were essential, so there was little hope for dramatic compression. Consequently, many practical applications were limited by model size.

The key idea

The key insight is that a trained network can be compressed in three simple stages—pruning, quantization with weight sharing, and entropy coding—without sacrificing accuracy. By removing near‑zero weights, grouping the remaining weights to share a small set of values, and then applying efficient coding, the model becomes dramatically smaller. This shows that most of the parameters are redundant and can be eliminated or represented more compactly.

How it works

First, the method prunes away the many near‑zero weights, turning the dense network into a sparse one and then retrains it to recover any lost performance. Next, it quantizes the surviving weights by clustering them so that each group shares a single representative value, dramatically reducing the number of distinct weight values. Finally, it applies Huffman coding (an entropy‑coding step) to the quantized weights and their indices, producing a compact binary representation. Each step is performed sequentially, and the network is fine‑tuned after pruning and after quantization to maintain its original accuracy.

See it — prune, quantize, encode

flowchart LR
  A["Trained net"] --> B["Prune small<br/>weights"]
  B --> C["Quantize +<br/>share weights"]
  C --> D["Huffman<br/>coding"]
  D --> E["35-49x smaller"]

What they showed

The authors demonstrated that Deep Compression could shrink models 35 to 49 times with no loss of accuracy. For example, AlexNet was reduced from 240 MB to about 6.9 MB, and VGG‑16 from 552 MB to about 11.3 MB. These results proved that massive size reductions were possible without harming performance.

What it changed

This work revealed that most network weights are redundant, making pruning and quantization mainstream tools for model compression. It sparked a wave of research and industry adoption focused on lightweight models for mobile and embedded systems. By providing a practical, high‑impact pipeline, it changed how engineers think about deploying deep learning in resource‑constrained environments. Next: Quantization.

The one-line takeaway

Most neural‑network weights are unnecessary, and a three‑step compress‑then‑retrain pipeline can remove them without hurting accuracy.

Read it / meet it

Reading the original Deep Compression paper gives a clear view of the three‑stage process and its impressive results. Its legacy lives on in today’s mobile and embedded model pipelines, which routinely prune and quantize models before deployment.