To change our server to be distributed, we need to do a few things:

• integrate the raft library into our code

• store our data in a way that can be replicated

• add a way for the replicas to tell each other about themselves

  We will go over this last part in more detail later, but this is a nice reminder that libraries are not magic. Replicas do not just automatically know about each other. This surprised us when we learned it - we had really hoped ZooKeeper (the first distributed lock server we ever used) would just know about other ZooKeeper entities automatically, and instead you had to know the IP addresses of every node. It was a real pain.

The first step in dealing with our rewrite is move functions to a separate package. This makes documentation and testing easier, and means our code can be depended on by multiple libraries.

Set up the store folder#

To do this, we will create a new folder called store. All Go files in this folder will have package store as one of their first lines.

Note that it doesn't declare the entire package path - that is determined by the parent go.mod file, which we're not creating here as it causes all sorts of issues (if we updated store, we would need to update the go.mod of the root package, which we can't do until we've deployed and waited for all of the caches to clear). It works fine, but it tends to cause confusion around what code you are actually calling.

Go modules are usually best tied to their version control. That is the go.mod should be at the root of your version control. So if your Git repository contains multiple packages, that's fine, but try to keep just one Go module (ie one go.mod file) to keep things sane. The downside is then all of your packages share a dependency tree, because dependencies are defined in go.mod. This won't affect your built binaries though, just how many dependencies you download when you build your package.

Inside of this folder, we can create Go files with any name. We will call our first file store.go but it could be literally anything and still work.

Set up the Config struct#

type Config struct {

	raft *raft.Raft

	fsm  *fsm

}

First things first, we will create a Config struct for whomever uses our server to initialize. We will add all of the data access functions to this struct.

func (c *Config) Set(ctx context.Context, key, value string) error {

func (c *Config) Delete(ctx context.Context, key string) error {

func (c *Config) Get(ctx context.Context, key string) (string, error) {

We mention this in other chapters as well, but using a struct pointer allows for us to do a few things easily:

• share configuration without globals

• do testing using mocks

• make sure the server is initialized before calling

We are making the fields of the struct private because we do not want anyone outside of this package to be able to modify the values, but we still want to be able to access and mutate them in the store package. Note here that:

• lowercase fields are the equivalent of private

• capitalized functions and types are public

We will implement our methods shortly, but we need two more very important functions, one that creates the config and one that returns a middleware for the server to use. We will walk through the middleware in the "Connecting the servers" section, but first let us build our config factory (a factory is a common name for a function that generates an instance of a type).

// NewRaftSetup configures a raft server.

func NewRaftSetup(storagePath, host, raftPort, raftLeader string) (*Config, error) {

	cfg := &Config{}

	if err := os.MkdirAll(storagePath, os.ModePerm); err != nil {

		return nil, fmt.Errorf("setting up storage dir: %w", err)

	}

	cfg.fsm = &fsm{}

	cfg.fsm.dataFile = fmt.Sprintf("%s/data.json", storagePath)

	ss, err := raftbolt.NewBoltStore(storagePath + "/stable")

	if err != nil {

		return nil, fmt.Errorf("building stable store: %w", err)

	}

	ls, err := raftbolt.NewBoltStore(storagePath+"/log")

	if err != nil {

		return nil, fmt.Errorf("building log store: %w", err)

	}

	snaps, err := raft.NewFileSnapshotStoreWithLogger(storagePath+"/snaps", 5, log)

	if err != nil {

		return nil, fmt.Errorf("building snapshotstore: %w", err)

	}