Interchange: change here between the Markets and ML lines.
Market query agent
Why
An analyst should be able to ask a question about a stock without knowing the database. The hard part is not answering; it is knowing how often the answer is wrong, and in which ways.
How
I built an agent that turns a plain question into a database query over daily prices for a basket of Indian stocks, computes the metric, and answers. Next to it sits the part I care about: a benchmark that scores it, and a list of every way it failed, by name.
What came out
93.8%, on a 48-question benchmark, every miss classified.
What broke
All three misses landed in the one tier written to catch them: questions the data cannot answer. Twice it picked one reading of an ambiguous question without saying so, and once it made up an answer after a query came back empty. The benchmark is also small, was run once per configuration, and was written by the same person who wrote the agent, all of which the README says out loud.