Agentic AI
Agentic AI isn’t just about LLMs and chatbots, or even using AI agents to generate code. It represents a structural shift in how we design, deploy, and operate software systems. While the use of LLMs to generate code is a really interesting topic I think that many developers are failing to think more about ‘agents’ and how these will shape how we build systems in the future. I don’t mean using agents for personal productivity tasks but rather using agents in our distributed systems, working as the logic engines. For developers and DevOps engineers, this movement is starting to look a more like the initial days of containerization which initially was a niche tool but then quickly became foundational.
This post is a very belated run through of this topic I presented back in September last year but never got around to posting. I’ll try and cover a high level view of the technical landscape behind Agentic AI, the components that matter, and how these systems will likely integrate into modern distributed architectures. I hope to expand on this more in future posts.
What Exactly Is Agentic AI?
Agentic AI refers to AI systems that don’t just generate text they cans set goals, make decisions, take actions, adapt to changing conditions, and operate over time not just per request. Think of it as moving from “ask the model a question” to “give the model a mission.” This is a huge shift for personal productivity but Agents can also be useful as an active participant in your architecture of modern distributed systems.
The Future of Software Development is AI-Native Systems
Agentic AI + Distributed Systems = a new kind of platform
Let’s imagine a world where:
- Agents monitor and scale microservices automatically
- Agents coordinate deployments, rollbacks, and incident response
- Agents manage data pipelines end to end
- Agents handle conversational, adaptive data collection instead of rigid forms
Not only can we use these forms of agents to perform these tasks, but we can also potentially out thm in the middle of our complex systems tro replace the business logic layers where the characteristics of an LLM agent would make this advantageous. Rather than building dynamic rules engines or complex business logic to handle all scenarios, instead we could delegate that to an AI Agent. Letting the agent decide on the best cause of action or handle error handling scenarios using all the context information at its disposal. Now the non-deterministic nature of LLMs means this is not a fit all solution but in the right architecture and with appropriate guard rails this could be groundbreaking.
The old “Forms over Data” era is fading, and so is the static business logic components paradigm. The future is about context, conversational inputs and outputs, and adaptive flow. Building business logic is hard so instead if we can just define the intent then AI can perhaps handle the normal logic flows but also the edge cases and error flows. Combine specialist agents together and you get even more power.
Collaborative Agentic AI: Multi-Agent Systems
One agent is useful, but multiple agents working together is where things get interesting. In a multi-agent setup each agent has a ‘role’, a ‘system message’ and an optimized LLM for their task. These agents can then collaborate toward a shared objective, whilst self-healing (through redundancy and fault tolerance) and eventually self-organising (i.e. they can fan out, specialize, and coordinate).
Some common agent orchestration patterns emerging are below (but there are many more emerging):
- Fan-out: one agent delegates tasks to multiple specialized agents
- Supervisor/Worker: a manager agent coordinates worker agents
- Sequential: agents pass work along a chain
These patterns mirror distributed systems design but with AI components instead of microservices.
Some Key ‘Context’ Concepts
Here are some key concepts in this space which I think developers need to be aware of if they’re not already. This is not a deep dive into agentic systems but more of a high level reference to some of the keys concepts.
RAG (Retrieval-Augmented Generation)**
RAG is basically how we give LLMs our own data to use. For example we might pass the LLM our product sales catalog so it can answer product questions accurately. Or more efficiently we might pre-search relevant product information based on the sales enquiry rebased request and pass that to the LLM as context. RAG is still fundamentally about passing context to the model.
Vector Databases: The Backbone of RAG
Vector DBs store embeddings which are mathematical representations of text, images, or other data which enable semantic search, making RAG fast and scalable whilst increasing accuracy of results but allowing you to retrieve the right context for the model. The workflow looks like below:
- Convert your documents into embeddings
- Store them in a vector DB
- Query based on the user’s prompt
- Pass the retrieved context into the LLM
Fine-Tuning
Fine-tuning allows us to tune the model to our specific domain or use case. This is a subject for another day but essentially we can use a few techniques of which the easiest and most accessible is using system messages and prompts to guide the LLM to behave as we desire.
MCP & Tools
MCP (Model Context Protocol) provides a standard by which LLMs can make use of tools and make API calls safely and consistently. Building a simple MCP server is easy, especially if you already have an existing API layer that you can wrap. MCP servers define what actions/tools/features are available in an LLM consumable way to increase the capability of the agent. For example the Microsoft Dev Ops MCP Server gives an agent the ability to see builds, repos and work items within your Azure DevOps organization.
System Messages & Prompts
System messages define the agent’s identity and rules, whilst the prompts are the data we pass to the LLM to trigger responses. Both can provide massive flexibility in how an LLM behaves.
It’s all about the ‘Context’
LLMs are context machines, not knowledge machines. Agents must be provided with context without our systems. This context comprises of:
- System messages
- User message
- Interaction history
- RAG context
- Tool call results
- Memory (short-term or long-term)
This creates a ‘context graph’ that the agent uses to reason. If you take one thing away from this post: LLMs are context engines. Everything is about giving them the right context.
Look at the list above and concepts we are familiar with can be seen:
- Prompt design becomes a form of policy or prompt engineering
- System messages are configuration to be handled
- History management requires state management patterns
- RAG relies heavily on data engineering
- Tool calling builds on existing API contract design and build
Building Agentic Systems
Building Agentic Systems is something I hope to cover more in the future as I continue to explore this large, rapidly evolving subject.
Conclusion
The future of distributed systems will be shaped by agents that can reason, collaborate, and act. And the developers who understand how to build, orchestrate, and control these systems will define the next generation of software.
I think Agentic AI is not just another AI trend but instead a fundamental shift in how we build software. To navigate this shift, developers need to understand not just LLMs but how to manipulate their context through system messages, prompts, RAG and tool integration (MCP), combined with the knowledge of how Agents can be tasked with a shared purpose through multi-agent orchestration patterns.
If you’re a developer or DevOps engineer, now is the time to experiment. The tools are here. The patterns are emerging. And the possibilities are enormous.