My Favourite Setup for REST Microservices in Go

A template for microservices in Go

Photo by Fotis Fotopoulos on Unsplash

In this story, I want to share my personal template for microservices that serve REST APIs in Go. Since I’ve started to write a few services from scratch, observing that particular code parts always repeat, I decided to export them to a template and share it.

While some parts may be adapted in case other frameworks are preferred, I hope the overall structure can help in implementing your next project in an easier and faster way.

In this story, I present an example Project-Service, which offers API endpoints to manage projects, milestones, and tasks.

Folder Structure

First, I want to share my preferred folder structure, where you can find all the files to run the example project-service.

• api/ contains one or more versions of the REST API
• dbs/ contains the code to initialize database connections and perform migrations
• models/ contains all model definitions and potential model-related logic
• pkg/**/* contains the service logic for the models
• seeds/ contains the code to initially seed the database with some data

Models

To simplify database operations, ORMs are a pretty neat choice. I personally prefer gorm. Gorm works through tag reflection, meaning you add tags to your model’s fields and gorm will process them. A simple model that can be processed by gorm looks like this:

Despite being super flexible, gorm provides simple usage with strong defaults. By extending the gorm.Model struct, your model will get an ID field that is used as an identifier (by default it will be an integer with auto increment).

Untagged fields are translated to snake case column names. Via field tags, we can configure gorm’s behavior, in this case, we just add an index to the name column.

Other models can be referenced by using their struct as field, or even as slices. Our service has three models: Project, Milestone, Task . One project has one or more milestones, each milestone may have one or more tasks.

I would propose creating a separate .go file for each model, which makes it easier to maintain if the models implement also other interfaces or have for example specific (de)serialization logic.

Finally, I always add one migrate.go file in the models folder, which contains the automigration code. With gorm, this is a no-brainer if you follow the guides, you can simply put references to all your models into gorm.AutoMigrate():

Database Initialization

The next part of my template is the logic to setup database connections and to perform initial operations like migrations. In the dbs folder, I have one file called setup.go that runs the code to initialize the preferred database connection. The code looks like the following:

The code is straightforward, there is one InitializeDatabaseLayer function that passes the DB environment variable and checks, which database driver should be used. In this sample, SQLite for development and Postgres for production setups are implemented, but it can be easily extended to all other supported gorm drivers.

For each of the two drivers, I’ve added a method that loads the required information from the environment and connects to the database. Afterward, the presented automigrate method is called to ensure, that the database is in sync with the models. Finally, an instance of gorm.DB will be returned to the application. I like to save this instance in a variable and export a function to obtain it, in this case, the GetDB method.

Service instances to perform database logic

Now that the database connection is set up, the next part to consider is the actual database-related logic. In my showcase, I have CRUD operations for all three models that need to be implemented. In my template, I prefer to create a folder for each model and locate all related implementations there, beginning with a service.go file. The following code shows this file for the projects:

The service implementation starts with a ProjectService struct and a singleton instance of it, that will be returned from calling projects.GetService(). The service itself contains the logic to perform database operations for the Project model. The basic CRUD operations are implemented as dedicated methods using gorm’s DB instance that was initialized in the setup.go file. Since gorm gives us a nice featureset, implementing these CRUD operations is straightforward. TODO: may reuse in different APIs

For the remaining models, the service implementations look similar. While this looks quite verbose for our three models and their simple logic, I think it will provide a huge benefit when your codebase grows, having things separated and implemented that way.

REST Endpoints using Gin Gonic

After implementing the core database-related logic, we need to expose it via REST API. Therefore, I mostly create an api folder that contains versioned APIs, e.g. in this simple case v1.go, in more complex APIs it might also be clever to create dedicated folders for each version.

The API with its specific version number has always its own URL context, e.g. /v1/* for version1. The API code is comparably simple since our core logic is implemented in services:

I implement each API version by using a struct, e.g. RESTAPIV1 for the REST API version 1. As an HTTP framework, I personally like to use Gin Gonic, since it is straightforward and powerful.

I put the gin router as a field in the API struct, and add Serve method, which can be called from the main program.

In the constructor NewRestApiV1, I instantiate this router and register all routers, via e.g. router.POST or router.GET etc. Parameters can be simply configured via :paramName in routes. As we can see in the EditProject method (which gets a reference to the gin.Context as the only parameter), the param can be fetched via c.Param("paramName"). Gin Gonic provides a great developer experience for working with JSON, through the ShouldBindJSON method.

Finally, return codes and also JSON responses are easily possible via c.JSON and the shorthand gin.H which can be used as a generic map.

While REST via Gin Gonic is only one example, many other combinations are possible, e.g. GRPC or Websockets, or simply using another HTTP library if preferred.

Seeds

While we can sync our model representation to the database via migration, it’s often very helpful to add initial data to the database, for example, an admin user. This avoids connecting with an external management tool to the database to add this data manually.

I prefer to pass the path to a seed file as a command line argument to the service, and when this file exists and the database is empty, the service will load the seed file and insert all the contained data.

To make editing easy, I prefer to use JSON for the seeds file. The code to run the seeds looks like this:

At first, all service instances are obtained. Afterward, the code checks if there are any tasks in the database, and returns the method if this is the case, since we want the seed data to be inserted only a single time.

If there are no tasks in the database, the code opens the seed, creates a JSON encoder and parses the file, in case it’s valid JSON. The JSON is expected to be a Seeds object, which has projects, milestones and tasks slices as fields. Finally, overall these slices will be iterated to insert the data.

One important note here is, that the seeded projects could also have milestones and these milestones could also have tasks.

The Main Program

After presenting all parts in detail, the last missing part is the main program, which is stored in main.go :

The main program is quite short since this template has dedicated folders for all core parts of the microservice. It starts with setting some command line flags via Go’s internal flag package. The flags are the following: a local address laddr on which the HTTP endpoints listen, the logleveland a path to the initial seed file initialSeedFile. I like to use logrus for logging, which accepts a list of different loglevels. The configureLogging method parses the loglevel from the flag and sets it accordingly, in addition to some preferred formatting settings. The main function calls configureLogging, initializes the database layer (including migrations), then runs seeds if initialSeedFile is set, and then creates the REST API and serves it under the configured laddr.

Dockerfile

Finally, I want to shortly present the Dockerfile that I always add to my Go services, to deploy them as containers. It’s a multi-stage dockerfile based on alpine, which creates comparably small containers.

It starts using the 1.18 Golang alpine container, copies the current folder into it, and builds it via CGO_ENABLED=0 go build.

While the CGO_ENABLED=0part may not be necessary here, if all stages are based on similar alpine versions, it helps to avoid issues when changing the first or second-stage base containers.

After building, I use the default alpine image as the second stage, installing ca-certificates (in case you want to communicate to external endpoints that are secured via TLS, this is mandatory), and then copying the built binary from the first stage. The last step is simply setting the docker ENTRYPOINT .

In this story, I’ve shared my preferred template for microservices in Go. I think it provides a good starting point, which also scales for larger services. I’m happy to get any feedback or recommendations to improve the story.

The complete code can be found on GitHub.