When compact tool outputs became enormous model inputs

One of the harder problems I've worked on recently came from building Drishti, the Jyotish agent inside Vedic AI.

Drishti needs to answer questions using a person's actual birth chart. Depending on the question, it may calculate chart information, inspect planetary periods, retrieve classical references, and bring those findings together in an explanation.

That means information has to move between multiple tools and reasoning steps. The problem I ran into wasn't a lack of information. It was that we were giving the model much more than it needed.

Where the context was growing

Individual tool responses had been designed to be relatively compact. But the total input reaching the model during a run could still become very large.

We found repeated chart facts appearing in nested tool responses. A calculation produced structured information; later steps carried portions of it forward alongside new results. The same facts could reappear in several forms.

Each output looked reasonable when examined alone. Together, the outputs created a much larger context than the question required. That added token usage and material the model had to process without necessarily adding useful evidence.

Choosing a more capable model wouldn't remove the duplication. The waste was being created by the system around the model.

The tempting fix was incomplete

It's easy to say, “make the tool responses smaller.” But a Jyotish interpretation can depend on several conditions together. Something that looks unnecessary at one step can turn out to matter when a later rule introduces an exception or modifier.

Remove too much and the model loses necessary evidence. Preserve everything and you pay for repeated input, higher latency, and a harder system to inspect.

What does the model need to know at this step, and what only needs to remain available elsewhere?

That was a more useful question than setting an arbitrary token limit.

Separating two kinds of information

We traced where repeated chart information entered the agent's context and narrowed the evidence passed forward at each step.

We also kept fuller audit records separate from the narrower material sent to the model. The system still needs detailed information to debug a run, inspect an intermediate decision, or investigate a bad answer. But the model doesn't need the entire audit history included in every input.

This is a conceptual view of the change, not a measured benchmark or a claim that every run now follows exactly the same path.

Why this matters beyond cost

The first visible symptom was input size. The deeper issue was who was responsible for deciding what mattered.

Giving the model everything can feel safer than filtering information. But duplicated, weakly related, or competing details can also make the task harder. A system should select relevant calculations and evidence deliberately, rather than use maximum context as a substitute for architecture.

Reducing context still needs evaluation. An optimization fails if it drops a condition that changes the substance of the interpretation. Lower token usage alone isn't enough to establish that the resulting answer is better.

What changed in how I build

I used to focus primarily on whether the model had enough information to answer. Now I pay more attention to the difference between information the system possesses and information the model needs at this moment.

I'm still working on the balance between useful evidence, latency, cost, and being able to inspect what happened. These changes addressed one kind of context growth; they didn't solve every failure an agent can have.

The lesson I keep returning to is that another tool or another reasoning step can sound worthwhile in isolation. The harder question is whether it helps the whole system enough to deserve its complexity.