PAPERS-05-002 · Paper — The Google File System (GFS)

The paper

The Google File System (GFS) is a 2003 paper by Sanjay Ghemawat, Howard Gobioff, and Shun-Tak Leung, presented at the ACM Symposium on Operating Systems Principles. It is considered a landmark because it showed how to build a reliable, high‑throughput storage system on cheap, failure‑prone hardware.

Before this paper

Before GFS, large‑scale storage relied on traditional file systems that assumed hardware was mostly reliable. When trying to store petabytes across thousands of inexpensive machines, disks failed constantly, causing frequent crashes. Existing systems could not tolerate such frequent failures without manual intervention. This mismatch limited the ability to scale storage economically.

The key idea

The core insight of GFS is to treat failure as a normal condition rather than an exception. It does this by breaking files into large chunks, replicating each chunk on several machines, and keeping only metadata in a single master node. By keeping the master out of the data path, the system can continue serving data even when many disks or machines fail.

How it works

A file is divided into 64‑megabyte chunks, and each chunk is stored on three different chunkservers for redundancy. The master server maintains a map of which chunkservers hold each chunk, but it does not handle the actual data reads or writes. When a client wants to read or write, it first contacts the master to get the locations of the relevant chunkservers. The client then communicates directly with those chunkservers to transfer data, allowing parallelism and high throughput. Writes are coordinated through a lease mechanism so that only one primary replica handles mutations, while the others apply updates later. The system is optimized for large sequential reads and appends, which match many Google workloads.

See it — master plus chunkservers

flowchart TD
  C["Client"] --> M["Master<br/>metadata only"]
  M --> C
  C --> S1["Chunkserver 1"]
  C --> S2["Chunkserver 2"]
  C --> S3["Chunkserver 3"]

What they showed

GFS demonstrated that Google could run its storage on commodity hardware at massive scale while automatically handling constant disk and machine failures. The design used 64‑megabyte chunks and typically three replicas per chunk, achieving reliable operation without manual repair.

What it changed

The paper introduced a template for scale‑out storage that many later systems copied. It directly inspired the Hadoop Distributed File System (HDFS) and influenced most cloud object stores, spreading the idea of a single metadata master with replicated data nodes. This shift enabled the industry to build cheap, large‑scale storage clusters that tolerate frequent failures. Next: MapReduce.

The one-line takeaway

Treat failure as normal and let simple replication plus a lightweight master provide reliable, scalable storage.

Read it / meet it

Reading the original GFS paper gives insight into the practical engineering choices that made massive data processing possible. Its legacy lives on in HDFS, cloud storage services, and any system that stores huge amounts of data across unreliable machines.