MTS unveiled a trust controller for LLM agent memory — a breakdown of the MyCity chatbot failure

MTS has unveiled a trust controller for LLM agent memory

🧠

Here's where agents get burned: the knowledge base holds two fragments — one correct and one outdated. Both are at hand, both look like fact. The model doesn't distinguish between them and confidently outputs the wrong one.

MTS engineers have introduced a trust controller for LLM agent memory — a mechanism that, before answering, chooses between conflicting knowledge fragments. The breakdown is based on the failure of the New York chatbot MyCity.

⚠️ How it broke in real life: in 2024, the New York City government launched MyCity — a digital assistant for small business owners, conceived as a reference guide to city rules and laws. The bot advised entrepreneurs to:

— not pay employees their tips in full;

— fire a person for complaining about harassment.

Such advice exposed entrepreneurs to fines and lawsuits. The correct information was sitting in the bot's own knowledge base — right alongside the outdated information. The cause of the failure: the bot couldn't choose between the correct and incorrect option when both were at hand.

MTS's approach is to choose between conflicting fragments before generating a response. In the breakdown on Habr — there's a concrete architecture, not just general words