Intuitive REST APIs and JSON Handling With Ballerina Programming Language

A detailed walkthrough

The niche of the Ballerina programming language is that it has been designed to allow developers to easily use, combine, and create network services to build real-world, cloud native applications. In other words, it’s a programming language that gives the right set of tools and level of abstraction to developers focusing on creating new APIs, new integrations, and new logic for network interactions. The latest Swan Lake release further simplifies these aspects among many others.

Using Ballerina can forever change the way you develop and problem-solve APIs. I intend to show you how via a walkthrough of a simple yet interesting example that uses and creates REST APIs with Ballerina.

REST and JSON

REST APIs have become the de-facto standard for companies building web services on the web and launching developer platforms because they’re easy to build and easy to consume. REST uses the standard CRUD HTTP Verbs (GET, POST, PUT, DELETE) and leverages HTTP conventions centered around data resources (HTTP URIs). For example, a retailer with a resource api.exampleshop.com/orders can behave similarly to a restaurant with a resource api.examplerestaurant.com/reviews. Both would need CRUD operations on that resource and prefer to cache queries. While Java, Python, Ruby, Perl, C, C++, Javascript, and many more languages can be used to build REST endpoints, and all with good working results, they weren’t purpose-built for working with networked resources. They don’t provide the right set of tools or abstractions that are built into the language, which can make the API developer’s life so much easier.

On a side note, one might argue that GraphQL fixes many problems that users have found with RESTful architecture and that it has taken up the world by storm. But GraphQL also comes with its own set of challenges. REST vs GraphQL is a topic that warrants an article of its own and there are many many articles out there that cover this. (And yes, Ballerina also has extensive support for GraphQL.)

JSON represents structured data based on JavaScript object syntax, is human-readable, and is the most widely used data format for data interchange on the web. JSON exists as a string and this is useful when you want to transmit data across a network. When you want to access the data inside it using dot/bracket notation, the JSON string must be parsed (or deserialized) to some kind of object or map representation in the language. This is also known as data binding.

The Nobel Prize laureates API

First things first. Let’s discuss the API and its data that I’ll be leveraging to explain some of the remarkable features of the Ballerina language.

Nobelprize.org offers free, open data about the Nobel Prizes and the Nobel laureates to developers. Let’s explore this data to get a sense about the laureates and their achievements. I’ll be using API Version 1, which returns data as JSON or CSV, and I’ll be talking to the laureate method, which allows anyone to find and list all Nobel laureates (persons and organisations).

You can use cURL to retrieve data from the laureate endpoint. Type the following command in your terminal:

curl https://api.nobelprize.org/v1/laureate.json

You will be able to fetch a JSON array of laureates:

{
 “laureates”: [
  {"id": "1", .. },
  {"id": "2", .. },
  {"id": "3", .. },
   ..
  {"id": "1009", .. }
 ]
}

A single laureate is represented by the following JSON structure:

A single laureate JSON object comes with a few primitive fields along with a couple of nested arrays and objects.

Ballerina’s first-class support for REST APIs and JSON

For instructions on how to set up the nobel_prize project in order to run the code, read this post.

Consume an API

You must first create a client object, nobelPrizes, to consume an HTTP service within a Ballerina program as shown below. Clients have remote methods that represent outbound interactions with a remote system. The fetched payload is assigned to the variable laureateResults (I’ll be covering the json data type in the next section). The check expression provides a mechanism to handle errors in Ballerina concisely.

In-built JSON

Ballerina comes with a network-friendly type system with powerful features to handle data on the wire because JSON is the ‘lingua franca’ in Ballerina — the data types in Ballerina are very close to JSON. This allows a JSON payload from the wire to come immediately into the language and be operated on without transformation or deserialisation. For example, you can convert from JSON to the json type in Ballerina or to a user-defined subtype of an in-built type such as map or record. After that, you can process that data, and convert it back to JSON. Read more about how Ballerina supports various types of network-data here.

And you can bring this data into your Ballerina program in 2 ways:

1. Using the json type:

Notes: Field access is allowed on the json-typed variables via the dot notation even though we don’t know what the fields are during compile time but you need to safely cast a value to a type via the ensureType function. ensureType casts a value to a type in the same way as a type cast expression but returns an error if the cast cannot be done. The error can be handled explicitly or through the use of the check expression. See handling errors for more information.

2. Using the record type:

Notes: A record is a collection of fields of a specific type. With record types, you have control over what your keys are. Typically, a record type is combined with a type definition, and by default, a record type declared in Ballerina is open. This means that you can add more fields to it than those specified. Ballerina also allows closed records, which can only have a fixed set of fields.

In this scenario we have four record types:

1. LaureateResults — to hold the root array of Laureates.
2. Laureate — to hold Laureate information.
3. Prize — to hold information about prize details (though it’s rare, a laureate can win more than one Nobel Prize in a lifetime. There have been four such laureates!)
4. Affiliation — to hold information about the laureate’s affiliation to an organisation or educational institute that supported their efforts that led them to winning a Nobel Prize.

If you inspect the fields in the JSON data hierarchy that represent a laureate, you will see that I have used the same field names in the record too. This allows a laureate to be automatically mapped to a Laureate record.

You will see some fields suffixed by a ?. These are optional fields, which means that they could be missing sometimes. For example, some laureates may not have a surname if they are an organisation, or a diedCountry if they are still alive, or an affiliation if they worked independently. Ballerina’s type system is flexible, like a schema language. And one of the ways it does this is by using optional fields.

You may have also noticed (Affiliation|json[0])[] affiliations?.This indicates that affiliations is an optional field. Also, the return type would be a union of Affiliation[] and json[0] (which means that the affiliations can hold either an array of the record type Affiliation or a JSON array). This was done to convert the affiliation data to an Affiliation record array or leave it empty based on how the JSON payload contained it:

or

When the API is called, the entire JSON payload gets assigned to a LaureateResults record and the array of laureates will be mapped to the laureates array. You can access the individual laureates via the foreach statement as shown in the code above.

The spread operator …rest in foreach Laureate {id, firstname, …rest} allows destructuring by spreading out the members of the Laureate type, where rest is a list or mapping that contains the unspecified members of Laureate (e.g., surname, born, died, etc.). This is equivalent to specifying them separated by a comma.

Since the surname field is an optional one, we must first check if it’s available and assign an empty string if not, like this: string surname = rest.surname ?: “”;

I’ll be going ahead with the approach of using records for the rest of this article because it’s more straightforward and suitable for our use case.

JSON manipulation with query expressions

Let’s assume that you want to return the same set of laureates to another program but with a limited set of fields. Let’s say you want to scrap the fields bornCountryCode, bornCity, died, diedCountry, diedCountryCode, and diedCity, and add a new field called isAlive to indicate whether the laureate is still living.

Here’s the code that allows you to do it:

To highlight the flexible and dynamic aspect of the language and its in-built support for JSON manipulation, I have used the same record type, but you can also create a different record type, which can be a closed record, to work with the required fields from the Laureate record type.

Notes

I have used a query expression on the laureates array to modify each Laureate record in it to contain selected fields and add them to a new Laureate array called formattedLaureates. Query expressions are incredibly useful when it comes to working with data and contain a set of clauses similar to SQL to process the data. They start with the from clause and can perform various operations such as filter, join, sort, limit, and projection. The from clause is used to define the input iterator source (laureates, in this case) that is considered for processing the data.

The let clause allows you to define variables that can be used only within the scope of the query expression. I have defined such a variable, isAlive, to hold true or false based on the date string value of the died field of a Laureate record.

Finally, with the select clause, we can define what fields should be present in each Laureate record in the formattedLaureates array, including the new isAlive field. Remember I mentioned that records are open by default? Because of that flexibility offered by open records, we were able to introduce a new field to the Laureate record. If it was a closed record, we wouldn’t have been able to do that.

Create a REST API

Along with a network-aware type system, Ballerina comes with fundamental syntax abstraction for working with network services. If you want to create a service, you can define service types to produce services, and a service can be written in just three or four lines of Ballerina code.

Service objects are attached to listeners, such as HTTP (or even GraphQL or gRPC), and receive network input such as JSON payloads. A resource function is named by the HTTP method (e.g. GET, POST, PUT) and a noun (such as laureate, order or review). A basic example of how you could create your own REST API for the laureates data is shown below:

Put it all together in your own REST API

Now that you have a fair idea about accessing an external API, manipulating the data it returns, and creating an API using Ballerina, let’s see how you can expose the laureates data via your own REST API.

Let’s define some functional requirements of the API.

Provide only a limited set of data fields from the original Nobel Prize laureates API.

Users should be able to view all the laureates:

curl http://localhost:9090/nobelprize/laureates

Users should be able to filter results via one or more of the following query parameters: name, year, category, gender, country, or/and isAlive status. Here are some cURL examples:

curl http://localhost:9090/nobelprize/laureates?name=Marie%20Curie
curl http://localhost:9090/nobelprize/laureates?gender=female
curl http://localhost:9090/nobelprize/laureates?gender=female&isAlive=true
curl http://localhost:9090/nobelprize/laureates?country=Pakistan

Users should be able to view a laureate by his/her id: <API_URL>/laureate/id

curl http://localhost:9090/nobelprize/laureate/100

Users should be able to add a laureate by providing a JSON string via HTTP POST. Here’s an example:

curl -X POST -H "Content-Type: application/json" \
-d '{
"id": "",
"firstname": "Dakshitha",
"surname": "Ratnayake",
"bornCountry": "Sri Lanka",
"gender": "female",
"prizes": [
 {
  "year": "2042",
  "category": "Physics",
  "share": "1",
  "motivation": "\"for inventing time travel\"",
  "affiliations": [
   {
    "name": "University of Colombo",
    "city": "Colombo",
    "country": "Sri Lanka"
    }
   ]
  }
],
"isAlive": true
}
' \
http://localhost:9090/nobelprize/laureate/

Structure the data and service

In a single file, laureate_service.bal, I have added some import statements and the record definitions that were discussed earlier. (You can remove the main.bal file or comment out the code you have worked with so far to avoid import errors and the output to the console from the main function.)

In addition to what was covered already, the global variables laureates and TEST_ID along with the functions initializeLaureates and getLaureateById have been added, followed by the service definition.

Notes

I won’t be persisting the laureates in a database in this post but will be using the laureates array to hold all the laureates instead of calling the Nobel Prize API every time we need to fetch information about the laureates. The array will be initialised via the initializeLaureates function.

initializeLaureates will fetch the laureates from the Nobel Prize API, format each Laureate by removing unwanted fields and adding the isAlive field, and return the formatted laureates.

TEST_ID will be used when the user wants to add a new user via the POST method and will be incremented by 1 with each addition.

The getLaureateById function will match the string parameter with the id of a Laureate in the laureates array and return a Laureate or an error.

Return all laureates with the option to filter results based on query parameters

Here’s the first resource function in the Nobel prize API: get laureates. This translates to a GET method to fetch an array of laureates or an error with the option of including input parameters to filter results.

Notes

• name, year, category, gender, country, and isLiving are all optional input parameters. If none of the parameters are present, the full list of laureates will be returned.
• The filtering happens within a query expression. Each Laureate in laureates will be checked to see if the criteria given by the query parameters are satisfied (if they are present) in the where clause.
• Several string variables are created to access optional variables/fields. If the query parameter name is not present, assign “” to searchTerm. The variable surname will also be assigned a blank string if laureate.surname? is missing. I’m also going to let users search the entire name to match a name they provide, so I’ll be constructing the fullName variable too. The expression var isAlive = laureate[“isAlive”] allows us to access the isAlive field via the bracket notation and assign it to isAlive, which is of type var. A single let clause can be used to define multiple variables separated by commas within the scope of the query expression.

The where clause allows you to filter by condition. You can define any conditional expression, which returns a boolean.

• First, it checks whether a name is present and if it is, it uses a regular expression to match the full name with the name provided.
• Next, it checks whether a year is provided. Since the year is in the prizes array (within the Laureate record), we need to iterate through the array and go through each Prize to find the matching year. We could have accessed the year with prizes[0].year, but remember, there are laureates who have won more than once.
• The lang library for arrays defines a function called filter, which accepts a function to perform specific filtering operations on arrays. The expression laureate.prizes.filter(prize => prize.year == year).length() > 0) allows us to do this. It is an anonymous function expression with an expression function body that can infer the parameter type. The above expression is equivalent to: laureate.prizes.filter(function (Prize prize) returns boolean { return prize.year == year;})
• It basically takes the array laureate.prizes and creates a new array with the Prize members where prize.year is equal to year.If that array is empty, that would mean that there were no prizes awarded that year. If the length of the array is > 0, there’s at least one prize that matched. The same approach has been followed to check the category as well.
• Checking the gender, country, and isAlive status is quite straightforward to see if it’s a match or is not provided as a query parameter.

If conditions return true, a Laureate in a given iteration will be selected and added to the results array.

If the results array is empty, an error will be returned. If not, the full array or a subset of it will be returned.

Return a laureate by their ID

The second resource function in the nobelprize API is get laureate/string id]. This translates to a GET method to fetch a single Laureate if the path parameter id is matched or an error if not.

Add a laureate

The final resource function in the nobelprize API is post laureate. This translates to a POST method to add a single Laureate whose data should be provided in the body of the request.

Notes

newLaureate is the Laureate that will be contained in the request payload as a JSON object. As explained previously, Ballerina allows you to parse the JSON directly into the language.

The incoming Laureate will not contain an id. Hence, TEST_ID will be assigned as the id. TEST_ID will be incremented by 1 for the next incoming Laureate.

newLaureate will be added to the laureates array via the push function.

If the Laureate was added successfully, newLaureate will be returned along with the assigned id or an error if any errors occur.

Test the API

With the laureate_service.bal file in your nobel_prizes package, you can run the API by running the bal run command from the nobel_prizes directory. You can test the API externally using these curl commands.

To test HTTP services internally, or test the package in isolation, you can use the Ballerina Test Framework. Testing Ballerina services involves sending specific requests to the service using a client and verifying the responses using the assertion functions. The aim is to make sure that the service and client behave as expected when sending and receiving both expected requests and malformed ones. See testing services and clients for more information.

You can view the full code of the service in laureate_service.bal

Conclusion

That brings us to the end but that’s not it. This REST API implementation by no means mocks a production-ready setup — I didn’t cover persistence, security, logging, API documentation, and cloud deployment, all of which can be implemented with Ballerina elegantly. The purpose of this implementation was to show you the language’s powerful but simple model to seamlessly create APIs and handle data on the wire with a focus on the flexibility that the language offers when manipulating data. Ballerina is a great fit for your next API project. Now you know why!

Thanks for reading. I hope this was helpful. If you have any questions, feel free to leave a response.

Resources

• You can read more about network interactions in Ballerina here.
• You can also check out this service implementation to understand using record types differently, particularly for different input and output formats.
• Read why Ballerina is a language and not a framework or library.
• Visit ballerina.io to learn more about Ballerina.

Want to connect? 
Follow me on Twitter.