Hadoop for Beginners (7): HDFS Data Read and Write Flow

Tutorial index: Big Data for Beginners: tutorial series

Last time we could already operate HDFS from code, but what actually happens inside the Hadoop cluster? This post gives a quick overview of how HDFS reads and writes data.

Write Path

1. The client asks the NameNode to upload a file; the NameNode checks whether the target file already exists and whether its parent directory exists.
2. The NameNode replies whether the upload may proceed.
3. The client asks which DataNodes the first block should be uploaded to.
4. The NameNode returns three DataNodes: n1, n2, n3.
5. The client requests that n1 upload the data; n1 receives the request and calls n2, which in turn calls n3, completing the communication pipeline.
6. n1, n2 and n3 acknowledge back up the chain to the client.
7. The client starts uploading the first block to n1 (reading data from disk into a local memory buffer first), packet by packet. As soon as n1 receives a packet it forwards it to n2, and n2 forwards it to dn3; for each packet sent, n1 puts an entry into an acknowledgement queue and waits for the ACK.
8. Once a block finishes transferring, the client asks the NameNode again for servers to upload the second block to (repeating steps 3-7).

Read Path

1. The client asks the NameNode to download a file; the NameNode looks up the metadata and finds the DataNode addresses holding the file’s blocks.
2. It picks one DataNode (nearest first, then random) and requests the data.
3. The DataNode begins streaming data to the client (reading from disk as an input stream, validating packet by packet).
4. The client receives it packet by packet, buffering locally before writing to the target file.

Rack Awareness

For data safety we spread our data out, deploying across racks or even availability zones so a single rack failure can’t take the whole cluster down. Distance affects performance, so Hadoop also has rack awareness. I mostly followed the official docs here: https://hadoop.apache.org/docs/r2.10.1/hadoop-project-dist/hadoop-hdfs/HdfsDesign.html#Data_Replication

Distance really just comes down to how many hops a network packet takes. Machines on the same rack are on the same switch, so it’s nearly direct with no hops; going farther takes several router hops — which is how distance gets measured.

The official documentation puts it this way (excerpt):

When the replication factor is three, HDFS’s placement policy is to put one replica on the local machine if the writer is on a datanode, otherwise on a random datanode; HDFS then places another replica on a node in a different (remote) rack, and the last one on a different node in that same remote rack. This policy cuts inter-rack write traffic, which generally improves write performance. The chance of rack failure is far lower than the chance of node failure, and this policy does not compromise data reliability or availability guarantees.