Investigate Functional Programming Concepts in Go

Applying the concepts

Photo by Vino Li on Unsplash

In this article, we will investigate how to leverage functional programming concepts in Go.

We will touch on the possibilities opened by the introduction of generics in go 1.18 and some limitations. Functional programming style helps us write code that is easy to understand, maintainable and testable.

Many of the underlying concepts have found their way into the latest versions of widely used languages already, notably the map/reduce pattern or the concept of optional values.

Guiding Principles

I would like to emphasize the following principles that are the basics of functional programming style:

Pure Functions

As the name suggests, functional programming is all about writing functions. A function is considered pure if its output only depends on its input and if it does not have any side effects. Such kind of function offers several benefits:

• It’s typically easy to understand what the function is doing because you know that it does not have any side effects — by definition
• The function can be unit tested without having to setup complex mocks
• Pure functions can easily be used in multi-threaded environments. Since they do not have side effects, they also do not require synchronization of external data structures

Functions having side effects are considered impure. A side effect in this sense is any modification of non-local state (i.e. changing global variables, mutating input arguments) or any form of I/O (reading/writing from streams or files, printing to the console, etc.).

Side effects on the other hand are a key feature of programs, so functional programming offers mechanisms to combine them with pure functions, e.g. the I/O Monad. We will not go into details in this article.

Immutability

The key idea behind immutable data structures is that the behavior of a function is much easier to predict if that function cannot change any data. This goes hand in hand with the concept of pure functions that imply that a pure function cannot change any of its inputs. Instead of changing data we make slightly modified copies of data.

Composition

One of the advantages of having pure functions is that we can use these functions and compose them into more complex (pure) functions. Since each of the building blocks of such a composition can be tested as a unit, we can derive more advanced functions with confidence until we finally arrive at the level of the desired program.

It turns out that it is possible to come up with a set of composition functions that are useful for many cases, so we can implement them generically. This in turn improves readability because the same set of composition functions occurs over and over, always with the same semantic.

Application to Go

Let’s investigate how the before-mentioned concepts apply to the Go language. We use a simple example for illustration purposes. Write a function that does the following:

• Take a map of unknown objects as an input
• Read one entry of this map
• If the entry is a string, try to convert it to an int
• Return a default value if any of this fails

In idiomatic Go this code might look like this:

In Go, the operations that might fail model their return value as a tuple with the actual value in the first element and a flag or the error in the second element. This tuple, however, is not represented as a top-level type in Go, i.e. we can only destructure it into two variables, but we cannot write a function that accepts the tuple as an input, to compose the individual steps together.

Let’s introduce the concept of an Option to address this issue. An Option is a datatype that either carries a value or the notion of no-value:

With this datatype we can write composable functions that represent our primitive operations:

Notice the use of generics for the helper functions. The Lookup method is completely generic, it works with any kind of map, so it is a good candidate for a utility library. The ToString and ParseToInt methods specialize the generic Options type for their use case.

The first advantage of separating out the primitive operations into their own functions is that we can now write unit tests for those functions, including full boundary checks. The real advantage comes in when we compose these together. For this purpose we define reusable composition operations:

Note in particular the Chain method. It transforms an Option that may wrap one type A into one that may wrap another type B . The relation between the types is given by a transformation function. All of our helper functions are of this particular data type, they take a plain type as an input and return an Option of another type as an output. So they immediately work together with our new Chain operation. Let’s see how to apply this to our original problem:

This example has the advantage over the idiomatic one in that it avoids the repetition of the if clauses for error handling. It shows the logical, sequential flow of operations and moves the error handling into the implementation of the composition functions. Still, the code is looking a bit complex.

We notice two aspects in this code:

• each line has the same structure. First, we create a function that describes the desired operation, e.g. Chain(ToString) then we apply this function to a value
• the argument to each function is the return value of the previous function and formally we do not need any of the intermediate values

So how can we compose these functions together in a more readable and compact way?

In a first attempt, we could simply try to chain the functions together as nested functions. This works but it severely lacks readability, mainly because the reading order of the functions does not match the execution order. We read O.GetOrElse first but it’s actually the last function to be invoked. Also the more steps we add to this picture the worse the reading experience will be.

Let’s fix this by introducing another utility function, Pipe, that accepts the initial value and then a number of functions that consecutively apply to the output of the previous function:

This code looks much cleaner, it shows the operations in their logical order, it is easy to understand and it does not clutter the code with explicit error handling. We can still do a little better than this because we notice that the input variable data is not needed explicitly other than as the seed to the first function.

So instead of using the Pipe helper function let’s introduce a Flow input function with this signature func Flow(f1, f2, ...) func(T)R , i.e. it creates a new function with the same input as the function it receives as its first argument and the same return value as the return value of the last function. Then our example finally looks like this:

Monads

During the course of developing the last function we discovered the usefulness of the Options type with its key operations Of and Chain . In functional programming this is what we call a Monad and it turns out that the underlying pattern can be applied to many more usecases.

The Of method in our implementation is sometimes called unit , return or just and it’s meant to wrap a value into a boxed value, where that box is capable of implementing common operations generically. The Chain method is also known as bind , flatMap or mergeMap and allows to apply a function to the wrapped value, while letting the generic code run on the result.

Golang Features

The introduction of generics in Go 1.18 makes it possible to write code for a variety of data types. With the syntax type Option[A any] interface we tell that we intend to represent a wrapper for any kind of data without specifying this type at coding time.

Another key aspect of the functional programming approach was the ability to compose functions together. It turns out to be helpful to define functions with only one single input argument and one output because then we can easily pass the output of one function as the input of the next function and create pipes. In order to use this feature, we leverage the ability of go to create higher-order functions.

Our Chain method is declared as func Chain[A, B any](f func(A)Option[B])func(Option[A])Option[B] , i.e. it is a function that takes one input, a transformation function f, and it returns another function as an output. This function accepts in turn one input, an Option[A] and returns one output, an Option[B] . This structure allows for efficient function composition.

The functional programming concept of pure functions is a good match to Go language features. Although Go does not enforce this pattern it allows for implementation.

Together with good support of higher-order functions it lies in the discipline of the programmer to write pure functions.

Limitations

While it is possible to implement programs in a functional style there exist several language limitations:

Type Variance

Go does not support the notion of immutable data structures. If desired it lies in the hand of the programmer to ensure that data structures stay unmodified during the course of a function invocation.

This lack of concept however has the consequence that all data types, in particular container types, are considered invariant. If the language supported immutable types, we would prefer the concept of covariant types for pure functions.

Consider the following example:

We can easily call the TakeAny function with an int as an input, despite the fact that it accepts an any.

But we cannot make the second call where TakeAnyArray accepts an []any and we try to call it with an []int.

This totally makes sense, because the []int is modifiable and if we allowed it to be passed into a function taking an []any then that function could try to add a float or any other data type into the array, which of course would not work. If we had a way to tell that we only intend to read from the array, then it would be acceptable to pass in the int array, but that is not the case.

Note that the fact that a primitive int is inherently immutable is the reason we can call TakeAny without problems.

The issue applies to all container types, including our Option[A]. We know that Option is inherently immutable but there is no way to tell the compiler.

So we if have e.g. a function returning a Option[*File] (e.g. Open) we cannot directly chain it to a function accepting a Reader without explicit type conversion.

Function Overloading

Go does not support to overload functions, i.e. defining a function with the same name but with different parameters.

The closest to overloading are variadic functions but these require the variable function arguments to be of the same type.

This restriction directly applies to our Pipe function. Ideally, we would like to define that function for an arbitrary number of inputs while preserving type safety.

Because of the lack of function overloading, however, we need to explicitly define a different function for each number of arguments. And we need to remember to use the correct numeric suffix when adding and removing steps to the pipe. But at least this gives us type safety and the compiler will let us know if we call the function with a wrong number of arguments.

Type Parameters on Methods

In the previous example, you might have wondered why we bother to write complex Pipe functions instead of using methods for chaining.

In the fictional example above we do not need a Pipe if the Optional type contained Chain and other operations as instance methods.

However, this is not possible, because Chain requires two type parameters, one for the Option it is operating on and one for the type of the return value, because we change the type using a transformation function. This would require Chain carrying its own type parameter, which is prohibited by the generics spec.

Conclusion

The functional programming style offers a very compelling programming model but at the same time, its use with the Go language exposes some downsides, notably because of (temporary?) language restriction and a deviation from the idiomatic coding style. Is it still worth using it?

To me, the answer is a clear yes. Pure functions, immutable data structures, and function composition almost automatically lead to testable and clean code. If writing tests is easy, it becomes second nature to write test code and functions at the same time and this pays off over time.

Function composition allows moving much of the boilerplate code into utility libraries. Those are written and tested once and can then be reused for many projects.

Since the same composition functions Chain, Map, Reduce … appear over and over again, even across Monads the code is easily readable despite the fact that it deviates from idiomatic coding style. It’s a matter of getting used to it, but this is only a shallow learning curve in my experience.

But why not use a language that has better built-in support for functional patterns such as Rust? The choice of a language not only depends on the syntax and language features but also on the ecosystem. Today, Go is the lingua franca of cloud-based applications, so if you want to integrate with the huge ecosystem of libraries, Go is a good choice. Also tooling and IDE support is pretty good, so we can easily cross-compile to many platforms in a breeze.

So let’s combine the best of both worlds to write awesome code!