The Emergence of the LLM Tech Stack
Exciting developments ahead.

The generative AI sphere is advancing at lightning speed now. I think we all miss the Javascript days at this point when something comes out, but it’s just “one more framework.” Today (April 2023), we are witnessing a stack being formed, solidifying, and taking shape. This is what is emerging on the scene:
Storage Layer
Compared to the other stacks, this is equivalent to the Database Layer of any other stack. The special particularity of this one is that we have another paradigm, the Vector Database. Without getting into much detail, this database converts the words into numbers, that are used as “nearest neighbor” indexes to assess how closely similar objects are to one another or to a search query. It's used for semantic search, nowadays as a state of art, the Pinecone database combined with OpenAI is an excellent tool for semantic search. Here is an example from the documentation.
Model Layer
This is the layer where you essentially choose the Large Language Model (LLM). The points are all agnostic here, and you always have embeddings that will use the layer below to access the data and finetuning. Currently, OpenAI dominates the scene, but in the future, open-source models from Meta or StabilityAI could take up some share. The crucial aspect here is that it needs to be plug-and-play.
Service/Chain Layer
If you want to compare anything with this layer, it would be something close to Kubernetes or Terraform, I should say. The high-level part combines all the other components.
Chain architecture, particularly the one developed by LangChain, should become the industry standard in my opinion. It’s quite new, but the idea is solid, and the integration of all concepts looks impressive. It utilizes the layers below and introduces other concepts and patterns, like Agents, to make an entire application work.
Also, it depends on how “low-end” this layer will be. This is where OpenAI plugins also exist, which will be an end-user app by itself. Depending on what you need as an end consumer, plugins like Zapier may be more than enough.
This will be the most crucial layer, regardless of future developments. If you want to use an analogy, LangChain is to OpenAI what Terraform is to AWS or Azure.
UI Layer
This layer is the final layer, the presentation if you will. It would be more closely related to Power BI or Tableau in terms of comparison.
For LangChain, we now have Flow Chain emerging, and it looks amazing; you can view the repository here.

Other contenders
There are always other solutions that tend to be “SaaS for profit” oriented, which also bring strengths, particularly in the enterprise-grade world. One of them is Microsoft, and the image below says it all:

Microsoft Graph is a dear tool for all the .NET developers, myself included. This is just a small stretch of what has been showcased in terms of Office Copilot, as well as other “one tool to conquer them all” types of approaches, like Power BI or Power Apps.
Of course, Google will attempt the same thing, but let’s see how it goes.
Some things stay at it is
The lingua franca is still Python, and other user-friendly tools remain the same, such as Jupyter Notebook and Google Colab.
Final Thoughts
The stack is emerging, and the title “prompt engineering” may not be as charming as it seems nowadays. In my humble opinion, everything is gravitating towards a “distro” of data science. And it looks amazing.