Member-only story
Running Netflix Conductor 3 in Docker Using Elasticsearch 7 and PostgreSQL
All wrapped in an easy, repeatable, and maintainable process
About This Article
The standard implementation of Netflix Conductor is configured to use Elasticsearch 6. This article covers the changes that need to be made to use Elasticsearch 7 and PostgreSQL, as well as providing several scripts to automate building and running Conductor, both locally and in a cloud architecture.
At the time of writing, the latest released version of Netflix Conductor is 3.5.3. The information in this article is based on version 3.5.1 but should be applicable (and easily modified) to the latest version. If you run into any issues, please let me know, and I’ll do my best to help. Another great source of help is the Netflix Conductor discussion board.
I am running all the supplied code on a Windows 10 machine, from Git Bash, which comes included with Git for Windows. Our cloud architecture runs on CentOS.
What is Netflix Conductor?
Netflix Conductor is a workflow orchestration engine that was built by Netflix to “orchestrate microservices-based process flows” (source: Netflix Conductor documentation). The full feature list of Netflix Conductor is located here, but a few key features are:
- Workflow and task definitions are implemented in JSON format. This allows for versioning and easy support for custom tools to create and manage workflows and tasks.
- Tasks and Workers, i.e., the building blocks that make up the workflows and the microservices that host them, respectively, are completely language agnostic, which allows the implementation to be done in the language best suited for the tasks. Libraries exist for Java, Python, .NET (shameless plug: this one is developed by me), and others.
- The database architecture is pluggable, which means we can choose what database we want to use for Conductor. Conductor comes with numerous ready-made plugins, supporting (amongst others) Dynomite (also by Netflix), MySQL, and PostgreSQL.
- Netflix Conductor comes with an (optional) UI, which allows insight and control over workflows, like…
