ZooKeeper for Beginners (1): Introduction and Installation

Tutorial index: Big Data for Beginners: tutorial series

What Is ZooKeeper?

ZooKeeper is a distributed, open-source coordination service for distributed applications, an open-source implementation of Google’s Chubby, and a key component of Hadoop and HBase.

It accepts watchers’ registrations and notifies them when data changes — think of it as ZooKeeper = filesystem + notification mechanism.

A ZooKeeper cluster consists of one Leader and multiple Followers. As long as more than half the nodes are alive, service stays up, and it guarantees globally consistent data, ordered request execution, and atomic updates.

Its data structure resembles Unix: starting at a root with child nodes beneath, arranged in levels like folders. Each node holds up to 1 MB of data by default.

ZooKeeper official site: https://zookeeper.apache.org

Installing ZooKeeper Locally

Prerequisites

Install the JDK first — that’s out of scope here, so I’ll assume you already have one configured. It’s basic groundwork; if not, look up how to install a JDK. Then download ZooKeeper and extract it to a local folder.

Editing the Configuration

I extracted ZooKeeper to /Users/renfei/apache-zookeeper-3.6.2-bin on my machine, with configuration files in apache-zookeeper-3.6.2-bin/conf. Copy zoo_sample.cfg to zoo.cfg — that zoo.cfg is the file we’re going to edit.

The main change is the data directory: dataDir=/Users/renfei/apache-zookeeper-3.6.2-bin/zkData. Point it at your own local path.

Other Configuration Notes

• tickTime = 2000: heartbeat count. The heartbeat interval between ZooKeeper servers and clients, in milliseconds — how often heartbeats are exchanged between servers or between client and server. One heartbeat goes out every tickTime, and it also sets the minimum session timeout to twice the heartbeat time.
• initLimit = 10: initial communication timeout for Leader-Follower pairs. The maximum number of heartbeats (tickTime units) tolerated when a Follower first connects to the Leader; it bounds how long a ZooKeeper server has to connect to the Leader.
• syncLimit = 5: synchronous communication timeout. The maximum response time unit between Leader and Follower — if the response exceeds syncLimit * tickTime, the Leader considers the Follower dead and removes it from the server list.
• clientPort = 2181: the port clients connect to.

Starting ZooKeeper

Run:

bin/zkServer.sh start

Starting the Client

bin/zkCli.sh

Installing ZooKeeper Distributed

Prerequisites

In the earlier tutorial Hadoop for Beginners (4): A Fully Distributed Hadoop Cluster we built out a cluster, so we’ll install ZooKeeper onto it now and skip the cluster setup details. Again, download ZooKeeper onto the cluster nodes and extract it into a local folder on each machine.

Editing the Configuration

Unlike local mode, here we create the data directory first — for example mkdir /Users/renfei/apache-zookeeper-3.6.2-bin/zkData — then create a myid file with touch myid and write that server’s number into it.

Copy apache-zookeeper-3.6.2-bin/conf/zoo_sample.cfg to zoo.cfg as before; that’s the file to edit.

Set the data directory: dataDir=/Users/renfei/apache-zookeeper-3.6.2-bin/zkData, adjusted to your own local path. Then add the following, modified for your cluster:

server.1=n1.renfei.net:2888:3888
server.2=n2.renfei.net:2888:3888
server.3=n3.renfei.net:2888:3888

The format is server.A=B:C:D:

• A is a number identifying which server this is. In cluster mode there’s a myid file under the dataDir directory holding this value; ZooKeeper reads it at startup and compares it against the configuration in zoo.cfg to determine which server it is.
• B is that server’s address.
• C is the port this server’s Follower uses to exchange information with the cluster Leader.
• D is the port used for leader election: if the Leader dies, servers need a port to communicate over to elect a new one.

Starting ZooKeeper

In a cluster you start ZooKeeper on each node separately:

bin/zkServer.sh start

Client Command Line

Command	Description
help	Show all available commands
ls path [watch]	List what the current znode contains
ls2 path [watch]	Show node data plus update counts and similar
create	Create a node normally
-s	With sequence numbers
-e	Ephemeral (disappears on restart or timeout)
get path [watch]	Get a node’s value
set	Set a node’s value
stat	Show node status
delete	Delete a node
rmr	Recursively delete a node

For programmers, Linux operations shouldn’t need much explanation — follow the syntax above or the official docs. Most of the time we drive ZooKeeper from code anyway, so I’ll leave it there.