A while ago, MongoDB purchased VoyageAI for 220 million dollars. Since then, they have released a couple of dedicated embedding models. For example, you can read their blog post on voyage-context-4.

The key premise in those sorts of models is that you can feed the model text of any size, and it will automatically handle generating embedding vectors, smart chunking, providing context, etc.

I ran into this recently and was curious to see how this can work. In particular, since RavenDB handles both embedding generation and vector search, I decided to do a full evaluation of MongoDB’s way of chunking. MongoDB built their own model to achieve this, but RavenDB’s approach to embedding generation is to rely on any embedding model you prefer to use.

Before we get into the full details, let’s talk for a second about what the point of contextual embedding is, so we are all on the same page.

Embedding models take your data and translate it into a multidimensional mathematical space based on its meaning. Similar items will be located near one another in this multidimensional space, and we can take advantage of that using vector search. That is why you can find Mozzarella & Ravioli if you want Italian food today, as in this example:

The problem is that all embedding models have a context limit. There is only so much text that you can push into the model before it will give up on you. If you want to search through a much bigger piece of text using semantic search, you need a different approach.

The industry standard approach to handling this is via chunking. In other words, you take a long piece of text, split it into separate parts called chunks, and generate an embedding for each one separately.

The easiest way to think about this is that you have a long document, and you generate a separate embedding vector for each page of text independently. Instead of having to digest a whole article, you feed a bounded chunk (page) to the model to generate an embedding vector.

Chunking is a neat trick, but it leads to its own set of problems. Assuming we have a large document that talks about new features in RavenDB, with a particular page that expounds on the details of “the database’s ACID guarantees". What would the embedding vector for that page look like?

If we just chunk the data naively, we’ll get a vector that is related to the generic concept of ACID in databases. The chunking approach loses the context of the data; it doesn’t understand that the database in question is RavenDB.

Contextual embedding allows you to bake a global perspective directly into every chunk’s embedding. In other words, the embedding for that page would know that the database that is being talked about is RavenDB.

If you are dealing with large texts and want to have high-quality search, contextual embedding is a feature you want. I guess that explains why MongoDB paid 220 million dollars for Voyage AI.

Sadly, I left that sum of money in my other pants, so RavenDB’s strategy for dealing with this scenario is quite different. We planfor models to become a commodity, so there is little benefit in trying to produce your own models at this point in time.

Instead, RavenDB takes the approach of working with all off-the-shelf models. That means that we are far more flexible, using the latest state-of-the-art models, instead of having to keep chasing them. But only some models support contextual embedding…

Luckily, we figured out that we can add this feature from RavenDB’s side, without needing to develop a custom embedding model for this. The technical announcement about it is here, with all the details. But the gist of it is that RavenDB allows you to attach context to the value you send for embedding.

The scenario below shows an example of storing litigation files using RavenDB and enabling proper semantic search over large amounts of data:

const tokenCount = 2048;
const overlap = 128;


const chunk = (field) => text.splitParagraphs(field, tokenCount, overlap);


embeddings.generate({
  FullDetails: chunk(this.FullDetails),
  CaseSummary: chunk(this.ExtractedSummary),
  PartiesInvolved: chunk(this.MetadataParties),
  Precedents: chunk(this.CitedAuthorities),
  RatioDecidendi: chunk(this.CoreLegalRules),
  ObiterDicta: chunk(this.DissentingArguments)
})
.withContextPrefix(this.Headline);

You can see that we generate embeddings for quite a few fields. For all of them, we use a chunking strategy of 2K tokens with an overlap of 128 tokens. Note the last line that adds a withContextPrefix call, where we add the Headline as part of the context for the data we’ll be embedding.

This additional context gives the embedding model enough information to contextualize the information we give it. The nice thing about this feature in RavenDB is that we don’t need to have any special support from the model. Everything is handled directly by RavenDB. That includes chunking, caching, adding the context, etc.

What to do when I don’t have pre-existing context to add?

If you have a title for an article, or a summary already written for you, that is great. But what happens when no such thing exists? You can also use GenAI tasks in RavenDB to process the data and get a proper summary (and then generate the embedding with that summary to have better queries).

I took the context prefix feature for a spin with a bunch of well-known datasets in the field of embedding and retrieval. We are using nDCG@10 — normalized Discounted Cumulative Gain at rank 10, the standard retrieval-quality metric on the public BEIR, LoCoV1, and LongEmbed benchmarks.

The underlying embedding model we use is OpenAI’s text-embedding-3-small, and we use exact() vector search in RavenDB, since we are testing purely the embedding output.

Adding context to chunked documents

For the following benchmarks, we defined two embedding tasks. One that would simply generate chunked embeddings from the raw text (with 256 tokens per chunk), and another with additional context taken from the document’s title.

embeddings.generate({ 
    ContentEmbedding: text.split(this.Content, 256)
        .withContextPrefix(this.Title) 
});

TREC-COVID        +9.2

COVID-19 literature comprising ~129K near-identical CORD-19 papers with short, keyword-like queries. The abstracts all look alike, so the paper’s title is the single most discriminating signal. You can see that this approach is able to provide better results than any of the other options.

Fair benchmarks are hard (we made it harder for us)

In the following benchmarks, the ravendb and ravendb+ctx entries are the only ones that are actually using chunking. In other words, all the other alternatives are getting the full document to work on. And indeed, you can see that the ravendb entry (which does native chunking) isn’t doing that well in this benchmark. With the added context, it reaches the top.

Chunking at 256 tokens was used because it is a reasonable chunk size (about two paragraphs of text), and at that size, you may lose the context of the overall document. This allows us to showcase how effective the additional context technique is. That is also quite useful for additional focus. Embedding quality degrades with the length of the text, so shorter chunks embed their concepts much more faithfully.

NFCorpus        +2.8

Consumer-health and nutrition queries matched against PubMed documents. Titles name the medical topic (such as a condition or a nutrient), which disambiguates heavily overlapping biomedical text; the prefix lifts us past published text-embedding-3-small results. In fact, only text-embedding-3-large is able to do better than us here (see below for benchmark results showcasing RavenDB’s approach with text-embedding-3-large).

SciFact        +1.1

Scientific-claim verification against research-paper abstracts. The title names the paper’s specific finding, nudging near-duplicate abstracts apart, but the abstract is already on-topic. The gain is modest, and we land within a point of the published te3-small score.

As you can see, in this case ravendb+ctx is doing better than ravendb. However, I wouldn’t say that it is doing well. The chief problem is that chunking to such a small size really hurts us, and just using the full document is better.

Testing additional context with text-embedding-3-large

We intentionally test this approach with a modest model (text-embedding-3-small that has ~100M - 300M parameters). Does this approach scale when we use a bigger model? The text-embedding-3-large model is estimated to be in the 1B - 2B parameter range. How does it behave when we use the same technique?

In the graph below, we are testing the Legal Case Reports dataset, which has a lot of large documents (some with > 100K tokens and many over the 8K token limit).

We tested the quality of the results with chunking of 256 and 4096 tokens.

You can see in the graph that text-embedding-3-large is indeed better than text-embedding-3-small. There is a +3.3 difference between the baseline numbers of both models.

With the context option, however, text-embedding-3-small is almost as good as text-embedding-3-large! And with context, text-embedding-3-large ismuch better.

We also tested text-embedding-3-large with a much larger chunk size of 4K, which should give it more context to draw on (but also dilutes that contextt). Even so, it wasn’t able to beat the additional context (with a much smaller chunk size).

Dealing with large documents

The previous datasets we dealt with all had documents that fit nicely within an embedding model context window. Now we are going to deal with much longer documents (5K–470K tokens each). To make things more interesting, these have no natural title to anchor a chunk.

To handle this, we use another RavenDB AI feature, GenAI Tasks, which reads the first 16K of the document and generates a short summary for it. We then use that summary as the additional context for the chunking.

Without further ado, here are our results:

Those datasets were taken from the LoCoV1 and LongEmbed datasets. They are quite large and are usually used to explicitly test handling very large documents.

You can see that this technique shows a measurable impact on most (but not all) of the datasets we have tested. It gets more interesting when you compare it head to head with the actual results of the LoCoV1 and LongEmbed papers.

The results we are showing here show us being worse on almost every level, which would typically be a Bad Thing. In this case, we are comparing RavenDB using an off-the-shelf embedding model (with chunking!) versus dedicated top-tier embedding models that process the whole document.

Across almost every task RavenDB is able to exceed the results of OpenAI Ada, Voyage-001, and E5-Mistral-7B.  Let’s take E5-Mistral-7B as a good example. It is a 7B parameters, while text-embedding-3-small has only 100M - 300M parameters, making it about 50 times smaller.

The following graph was extracted from E5-Mistral paper (arXiv 2401.00368), Table 17 and should give you a pretty good idea about how the two models compare:

On the other hand, when we use the same text-embedding-3-small and our context prefix approach, we get the following results:

The most interesting thing about this graph is what this means. There is a very clear divide between the datasets where E5-Mistral-7B is leading and those where RavenDB’s approach leads (with a much smaller model).

The whole-document 7B model dilutes long text into one vector; our approach keeps focused chunks and restores document context via the summary. In the datasets composed of short documents, Mistral wins handily (it's a 7B model, ~50× bigger).

On long documents, RavenDB’s approach flips that by large margins when the answer is spread across the document (QMSum, passage retrieval, multi-hop QA). On long documents the chunk+context prefix strategy buys much more than raw model size does.

The 7B model reads the whole document into one vector and gets diluted; the small model retrieves a focused chunk and gets its document context back from the summary.

Summary

RavenDB’s context prefix feature shows how a different architecture can get you better results and higher efficiency. RavenDB’s approach allows us to go head to head with dedicated models and still come out ahead when dealing with large documents and complex tasks.

It also works on any model. I used text-embedding-3-small specifically because it is a baseline model, not a top-tier one. The fact that this is model-agnostic means that you can tune your approach based on your dataset and your requirements. When a new (and better) model comes by, you can just move to it and still reap the benefits.

This approach won’t cost you 220,000,000 USD. I just checked, and producing this blog post cost us about $110 (most of that by generating summaries for the large documents, to be honest). Only $22 of that was spent on the actual embedding.

Pair a DGX or Mac Studio in a cupboard with Gemma 4 (another great 7B embedding model) and RavenDB’s context prefix mode. You get top-tier results for a one-time ~$4,000 USD hardware investment, with no monthly bills.

* You can find the code to reproduce the findings in this post in the following GitHub repository.

I've written before about why I think the “modern” approach of solving every problem by adding agents is a doomed path. The current instinct is to add more layers of agents, judges, reviewers, etc. - and hope the model will be smart enough to do the right thing and actually get something done.

The issue is that we already have a well-understood way to coordinate independent contributors working on a complex system. It's called software design & architecture, and we've been refining it for decades. The instinct to solve a coordination problem by adding a smarter message bus between your agents is exactly backwards.

When talking about multi-agent support in RavenDB, I want to be clear about what we did not build. We didn't build an orchestration framework. We didn't build a swarm. What we built is much more boring, and I mean that as the highest compliment I know how to give.

We built a way for you to solve problems in a predictable manner and without a lot of hassle.

If you haven't looked at RavenDB's AI Agents yet, the short version is this: an agent is a system prompt, a connection to an LLM, and a set of Query tools, which let the agent read specific data through RQL you define, and Action tools, which let it do something in your application, but only through the specific doors you've opened.

The note on “specific” is the whole point. Instead of giving the model the freedom to access any data it wants and execute anything it feels like, we place careful guardrails that it cannot escape.

The agent can only pull the levers you gave it, using the data you explicitly handed it. The database takes care of the tedious, error-prone plumbing (such as conversation state, message history, talking to the model provider, etc.) and stores every conversation as a real document in the @conversations collection. You define capabilities, and RavenDB takes care of all the rest.

Where one agent starts to hurt

A single agent like this will take you a genuinely long way. Right up until it doesn't. The issue is the slow creep of complexity. Let’s say that you start with a simple agent to deal with answering employee questions in the context of the HR department.

The next feature is to assist them in filing expense reports. Then someone wants it to handle time-off requests, and then to look up the org chart, and six months later you have one prompt trying to be an entire company, with thirty tools competing for the model's attention and a context window stuffed with things that have nothing to do with the question being asked.

That situation is problematic on multiple levels:

• Your context window is full of a huge prompt (usually not relevant to the task at hand), a lot of tool descriptions and capabilities meant to cover every possible scenario under the sun.
• Users’ questions are either squeezed into the remaining context window or you have to move to models with larger context windows, which also cost more.
• You are also stuck with a single model for everything, instead of being able to pick the right model for each scenario. You overpay to run trivial chit-chat on your best model, or you cripple the hard tasks by forcing them onto a cheap one.

This is not a new problem. Scope is how you manage complexity, and it works exactly the same whether the thing on the other end is a compiler or a language model.

Multi-agents are just agents that talk to each other

A multi-agent system in RavenDB is not a special kind of agent. It's several ordinary agents, each built exactly the way you already know how to build one, where one of them happens to know that another exists and can hand work to it.

The way you connect them is almost anticlimactic. You add a SubAgents entry to the parent agent: an identifier and a description. That's the entire wiring job. The description matters because that's what the parent reads to decide whether a given request belongs to the specialist. There is no router you write, no dispatch table, no orchestration layer.

You made the subagent available to the root, and RavenDB will invoke it for you when it is needed. RavenDB will also handle all the logistical minutiae needed to accomplish this successfully.

Those include ensuring that the scope for the root agent is shared with the called subagents, managing memory and conversation history, allowing the subagent to invoke its own queries and actions, etc.

We ship a demo for this — an HR chatbot, samples-hr on GitHub if you want to run it yourself. An employee comes back from a conference, uploads the receipt, and types "submit this expense." From their side, it's one smooth conversation.

Behind the scenes, it's two agents. The HR Assistant faces the user and handles benefits, policies, and general questions. It does not know the first thing about filing an expense. What it does know is that there's an Expense Manager specialist, and — from that one-line description — that receipts belong to it. It hands the task over, the specialist analyzes the receipt and creates the BusinessTripBills document, and the answer flows back up.

Go look in the database afterward and you'll find two conversation documents, not one — the HR Manager's log and the Expense Manager's log, each with its own scoped history. That's not an implementation detail I'm mentioning for trivia's sake. It means you can open either conversation and see exactly what that agent received and how it responded. The audit trail falls out of the design for free.

Here's the parent agent, trimmed down to the part that matters:

public static Task Create(IDocumentStore store)
{
 return store.AI.CreateAgentAsync(
  new AiAgentConfiguration
  {
   Name = "HR Assistant",
   Identifier = AgentIdentifier,
   ConnectionStringName = ConnectionStringName,
   SystemPrompt = @"You are an HR assistant.
Provide info on benefits, policies, and departments.
Do not suggest actions that are not explicitly allowed by the tools available to you.
Do NOT discuss non-HR topics. Answer only for the current employee.",


   Parameters =
   [
    new AiAgentParameter(EmployeeIdParameter,
     "Employee ID; answer only for this employee")
   ],


   // The HR agent has no idea how to file an expense.
   // It just knows a specialist exists, and when to call it.
   SubAgents =
   [
    new AiAgentToolSubAgent
    {
     Identifier = ExpenseAgentIdentifier,
     Description = "Manages business trip expenses: analyzing " +
      "receipts/bills, reporting expenses, and retrieving " +
      "monthly expense summaries."
    }
   ],


   // Its own capabilities cover only HR concerns.
   Queries =
   [
    new AiAgentToolQuery
    {
     Name = "GetEmployeeInfo",
     Description = "Retrieve employee details",
     Query = $"from Employees where id() = ${EmployeeIdParameter}",
     ParametersSampleObject = "{}"
    }
   ]
  });
}

The Expense Manager, for its part, is defined in exactly the same way as any other agent. There is nothing special about it. It's a normal agent that happens to be pointed at by a SubAgents entry.

Notice what this buys you. The HR agent never sees the finance tooling. It doesn't integrate the expense system, doesn't swallow that domain, or deal with a BusinessTripBills document. It shells the call out to the specialist and stays in its lane. If tomorrow finance wants their own policy-checking agent in the loop, you add another SubAgents entry to the Expense Manager and the HR agent is none the wiser. Each piece keeps a small & independent scope.

Each of those agents is isolated from the others, so we can have the HR Agent use a pure textual model, maybe with high levels of reasoning. The Expense Manager agent, on the other hand, needs to be multimodal (to be able to read receipt images), but it doesn’t need to be smart. You can customize each for its own needs, without having to find the lowest common denominator.

The fact that each of those agents (even if they both participate in the same conversation) will use separate @conversations documents also means their contexts are isolated from one another. The fact that you just pushed the entire set of receipts from a two-week business trip to the Expense Manager agent doesn’t weigh down the HR agent when you ask about your remaining holiday balance.

When you should not do this

This shouldn’t be your default architecture. Like any advanced technique, you need a sufficient level of complexity to justify it. If the work fits in one sentence, it probably fits in one agent.

Answering from a knowledge base, generating a document from a template, running a linear sequence with no branching — a single well-built agent handles all of that, and reaching for sub-agents just to keep things tidy or to mirror your org chart is perfectionism. It's the same mistake as premature abstraction, and it costs you real latency and real tokens for the privilege.

Multi-agents earn their keep when one of the following is true:

• You need to independently develop the agents (different teams are handling different agents).
• You will use different models for different parts of the system.
• You want to explicitly limit the sharing of context between different parts of the system.

For example, keeping with the HR agent theme, let’s say that we want to allow users to ask questions about our policies (an example of such an agent). The problem here is that we may have a lot of policies, and even when we limit ourselves to the right one, that is a lot of text.

Shoving all of the work of finding the right policy and extracting the specific elements to match the user’s question into an isolated agent can massively reduce the number of tokens that your agents will burn.

Summary

RavenDB’s multi-agents aren’t about orchestrating a robot army or an independent swarm of self-coordinating agents. It is far more prosaic than that. You're composing independent components with clear boundaries and letting each one own a scoped conversation, its own tools, and the model that fits its job.

That's not a new idea we invented for the age of AI. It’s bringing back the notion of independent components that are greater than the sum of their parts, in a way that is manageable, consistent, and hassle-free.

If you want to try it, grab a free Developer license or spin up a free Cloud database, and the multi-agent guide walks through the demo end to end.