Over 90% answer accuracy: MongoDB’s reference-Q&A approach to LLM optimization

An AI answer can mention your company and still leave customers stuck with outdated instructions. MongoDB’s case concerns that gap between being visible and being described accurately.

The company reportedly achieved more than 90% answer accuracy for MongoDB-related questions. Profound’s case study

The practical LLM optimization workflow is to compare answers with your verified reference and feed corrections back into the source material.

MongoDB’s problem: missing setup steps and outdated documentation

The case describes AI omitting necessary post-registration steps and referring users to outdated documentation. Profound’s case study

Illustration: Illustrating how outdated documentation and missing steps can impair understanding.

In an illustrative scenario, saying a service is ready immediately after registration would mislead someone who still needs to configure a connection. A mention-only check misses that problem.

Read answers about what customers must do after signing up, as well as pricing and capabilities. The right product recommendation does not guarantee the right instructions.

Compare answers with a verified Q&A and return errors to the source content

MongoDB compared AI responses with its own reference Q&A, investigated errors and citations, and used those findings to improve source content. Profound’s case study

Illustration: Compare the AI answer with a verified reference, then check differences against the source.

Start by establishing the correct answer from current specifications or operating documentation rather than memory.

Identify the incorrect sentence or missing condition in the AI response. If the cited source contains the same problem, it becomes a candidate for correction.

For outdated external pages, record a correction request and the official URL supporting it. Your team cannot directly rewrite another publisher’s content.

Citation aggregation and schema suggestions reduced release-update work by about 30%

The story describes agents for citation aggregation and schema suggestions, with approximately 30% less time spent on release-update work. Profound’s case study

That is not a 30% reduction in all company work. Schema provides structured information about page content; proposed markup still needs to match the actual page and product.

What the 50% visibility increase and 90% answer accuracy actually describe

The 50% visibility increase concerns presence in monitored answers. Accuracy above 90% concerns MongoDB-related questions, not the overall accuracy of AI systems. Profound’s case study

Illustration: Check whether the company appears and whether the explanation is accurate separately.

The complete question set and detailed scoring rules are not disclosed. For your own program, define the errors you want to reduce before adopting a percentage target.

Correct setup instructions and accurate pricing are separate checks. Recording them separately helps identify which documentation needs attention.

Try it yourself: build a reference table for recurring customer questions

Choose three recurring support questions and record the correct answers with supporting URLs. This is our suggested exercise.

Illustration: Compare responses against a record of questions, verified answers, sources, and dates.

Example question	Reference information	What to inspect in AI answers
What happens after registration?	Required steps and sequence	Missing or reordered steps
Is this included in my plan?	Capability and conditions	Outdated plans or unsupported features
Can I migrate my data?	Supported fields and limitations	Claims that unsupported data can be moved

Ask the same questions of the chosen AI service and retain the response, date, and cited URLs. Compare errors with your reference and choose one source page to correct.

Check again under comparable conditions after publication. If the answer does not change immediately, preserve the edit date and observations instead of layering unrelated changes.

Update the reference’s verification date when specifications change. The document used to evaluate AI must stay current too.