AI analytics platform

Original problem

The agent retrieved chunks and called the model inside the HTTP request. Long runs were killed at the function timeout, the stream died, and retrieval was not limited to the caller’s documents.

Technical environment

Next.js UI, a Node.js API, a Python retrieval worker, PostgreSQL for documents and run state.

Engineering decisions

Move the agent off the request path. Keep the existing prompts and screens. Refuse an answer that has no cited passage.

Implemented fixes

Node.js accepts the run and returns immediately. A Python worker retrieves with the tenant id in the query, reranks, and stores citations. The client resumes from the last acknowledged chunk instead of marking a dead stream complete.

Verification

A forced timeout in staging resumed the same run. A second tenant’s documents did not appear in the retrieved set. Answers with an empty corpus returned a refusal.

Identified issues

• No queue between the request and the agent run.
• Retrieval filtered in the UI, not in the query.
• No citation when the corpus did not contain the answer.
• The client treated a dropped socket as a finished run.