The Hidden Cost of Parallel Processing in GitHub Actions

Why monolithic workflows might be the better option for your short-running workflow jobs

Image source: The Blue Diamond Gallery

In my recent effort to optimize our GitHub Actions workflows, I came across a surprising fact I was unaware of. I feel it’s important to share my finding in this article.

Parallel Processing in GitHub Actions

Parallel processing in GitHub Actions workflows is a common “best practice” for the many benefits it brings:

• Faster execution: By distributing a job across multiple parallel processors, the overall execution time can be reduced significantly.
• Scalability: Parallel processing can help scale workflows to handle larger and more complex projects.
• Resource optimization: With parallel processing, it is possible to allocate resources more efficiently, allowing for better resource utilization.

If we compare a sequential workflow job to a monolithic application, parallel processing can be regarded as the light-weight microservices, with many smaller workflow jobs which focus on their specific purposes.

In the CI/CD world, using parallel processing to tackle unit tests, integration tests, security scanning…