Knowledge Base Citations, Agent Versioning, and Insights on the GenAI Platform
The GenAI Platform is a powerful tool for serving and deploying LLM agents at scale on the DigitalOcean Cloud. Using the platform, it‘s easy to give some of the most powerful LLMs like Llama 3.3, GPT-4o, and Claude Sonnet 3.7 access to whatever information you need using RAG with files stored on DigitalOcean Spaces storage as a Knowledge Base.
When using agentic applications across a wide variety of contexts, there come a plethora of different meta characteristics which are critical to identify and control. Recently, several features went live on the GenAI Platform that work towards controlling and interacting with these facets of the models. Namely, these are Knowledge Base (KB) Citations, Agent Versioning, and Agent Insights. In this article, we look at each of these new features and depth, and discuss how they can help you evolve your AI workflow even further.
DigitalOcean’s GenAI Platform is our foray into creating a live-service where users can access, create, and modify LLM powered AI Agents on the cloud. Using GenAI, its possible to create anything from powerful agents for your business to workflow automation for repetitive tasks to simple chatbots for any use case. With GenAI, it’s simple to make agents that work with your own data, and can run agentic functions for your service.
In this section, let’s look at each of the new features for the GenAI Platform in a bit more detail.
Knowledge Base Citations
The most exciting new feature to be released for developers interested in building a deeper understanding of their agents are the KB Citations. These give us clear directions to follow when we want to better understand how the model is generating the response given. It does this by tracing back which documents are being consulted with Retrieval Augmented Generation (RAG) to find the result. Using this, users can see exactly how and why a certain answer is given, and follow that back to the resource used to give the response. In addition to making the inference more accurate and personalized to a required data corpus, this also functions as a helpful search methodology for your work.
To see Knowledge Base Citations on your own GenAI Platform outputs, we can use the playground for each model. Go to your Agent homepage in the GenAI section of the DigitalOcean Cloud Console, and click into your Agent. Then, generate an output in the playground. We can use the link underneath the output to view our citations and then access the files in the Knowledge Base.
Agent Versioning
The next feature we want to discuss is Agent Versioning, a sub-practice of LLM-ops versioning. Agent Versioning is essentially the practice of making saveable copies of each iteration of an agent’s development. Using this save state, it is simple to roll forwards or backwards through updates. This is particularly important with large sets of agents operating in tandem, where a small change can potentially upset the entire apparatus. By rolling back updates, we can ensure that our agents are always operating as expected during development without affecting production.
To access Agent Versioning, click the Activity tab on your Agent’s homepage in the GenAI tab of the Cloud Console. There, we can see any previous versions of our Agents if we need them.
Finally, we want to introduce Agent Insights. These are one of the most important features for production with any LLM powered Agent because it gives us the ability to directly determine how much volume, in tokens, our model is being called for at any given time. This is critical for measuring costs at scale.
To see agent insights, simply scroll down in the overview on the Agent homepage. On the left, we can find a chart indicating our token generation on different time scales. On the right, we can see advanced token metrics like average end-to-end throughput and average end-to-end latency.
To take advantage of these new features, we need to set up an agent or series of agents on the GenAI platform. For this tutorial, we used the corpus of this blog as the Knowledge Base material. Other examples of content that could be used include financial statements, notes, and other records. We recommend sourcing your data privately, or testing using a public set of data or files like those found on Kaggle or HuggingFace Datasets.
With the power of the GenAI platform, we can create a LLM powered AI Agent that can respond to our data using RAG. With the implementation of our new features, we can offer our users insights into where their responses are coming from using Knowledge Base Citations, deploy different versions of our agents at different development stages or roll them back when there are problems, and we can get direct insights into how our models are performing with regard to cost and efficacy.
The new features offered on the GenAI Platform begin to bring the platform up to par with the competition in terms of meta-capabilities and insights, while maintaining the DigitalOcean ecosystem integration that made it so powerful in the first place. With all of the best models from OpenAI and Anthropic now available on the platform, DigitalOcean is now one of the easiest places to create effective RAG solutions for your workloads at scale.
