Filter the LLM's tool list by the user's permissions, not its output
The requirement was a natural-language command interface over an existing business application. Ask for the sites available next month; log an expense against a location; find out what a client owes. Not a chatbot — a way to drive the application’s own operations by typing a sentence.
The security question arrives immediately. The application has a permission model. Some users can see financial figures and some cannot. If a language model is now sitting in front of the API, what stops it doing something the person asking is not allowed to do?
The wrong answer, and the common one, is to put the rules in the prompt. “Only answer questions about X. Do not reveal financial information to users who are not finance staff.” Prompt instructions are guidance, not enforcement, and treating them as a security boundary is a category error.
The approach taken here has the authorisation in the structure instead, in four layers.
Layer one: filter the tool catalogue per request
The tool list handed to the model is assembled per request from the caller’s own permission set. A user without permission to view invoices is not sent the invoice tools.
The model literally cannot see a tool it is not permitted to use. It cannot be persuaded to call it, cannot be tricked into calling it, and cannot hallucinate it into existence, because a tool call naming something not in the catalogue is simply an error.
This is the load-bearing layer, and its strength comes from the fact that it does not rely on the model’s behaviour at all. Prompt injection works by changing what the model decides to do. Filtering the catalogue changes what there is to decide between. The attack surface for “convince the model to call a forbidden tool” is empty when the forbidden tool is not in the list.
Layer two: the tools call the real code path
Tool execution invokes the application’s actual controller in-process, through a synthetic request and response object, rather than reaching for the database directly or duplicating the query logic.
Two consequences. The permission check that lives in the controller runs again — so even if the catalogue filter were wrong, the underlying handler still refuses. And there is exactly one implementation of every operation. A tool cannot drift from the endpoint it mirrors, because it is the endpoint.
This is worth insisting on. The tempting shortcut when building an agent layer is to write lightweight query functions for the model, because the real handlers return more than you want and take parameters you have to construct. Every one of those lightweight functions is a second implementation of an operation, and second implementations are where authorisation gaps live.
Layer three: writes never execute inline
A tool that would change data does not perform the change. It returns a pending action: the tool name, the arguments, and a human-readable summary of what would happen. The user then confirms through a separate, ordinary, authenticated endpoint.
So a mistaken interpretation of an ambiguous sentence costs a confirmation dialogue, not a corrupted record. The model proposes; the person disposes. And the confirmation endpoint is a normal part of the application with its own permission check, which means the write path does not depend on the agent layer being correct at all.
The general principle: let the model read freely within the user’s scope, and require an explicit human confirmation for anything that writes. The asymmetry is right because the cost of the two error types is wildly different. A wrong read wastes a moment. A wrong write has to be found and undone, and might not be found.
Layer four: keep the domain closed
The system prompt is explicit that the assistant acts only through the provided tools, does not answer general questions, and does not invent data. It also supplies the things a model cannot know and will otherwise guess about — that amounts are in a particular currency, and what today’s date is.
This layer is the weakest of the four and it is deliberately not carrying any security weight. It improves behaviour; it does not enforce anything. Everything that must not happen is prevented by the layers above.
Entity resolution without a vector database
The other design decision worth extracting is how the assistant turns a phrase in a sentence into an identifier.
A user says “the site on the main road”. The model needs an identifier to pass to the next tool. The fashionable approach is embeddings and a vector store.
The approach here is a search tool. The model calls a search over the existing structured data, receives matching records with their codes, and uses the code in the next call. Structured data resolves itself, always fresh. A vector index would need building, updating and keeping in sync with records that change every day, and it would answer approximately when the database answers exactly.
This is a good default to reach for first. Vector search earns its place when the target is unstructured text — documents, notes, conversations — where there is no exact match to find. For resolving a name to a row that exists in a table, you already have an index, and it is the right one.
Cost control, which turned out to be a correctness issue
Token cost was reduced by having tools return a compact envelope — a total count, the number returned, and a small number of trimmed rows — rather than full records, with a cap on conversation history.
That change fixed an accuracy bug as a side effect. The model had been reporting the page size as the total number of records, because it received a page of results and no indication that more existed. It answered with a count that was wrong by a large factor, confidently, because nothing in its input told it otherwise.
A model will report what it can see as though it were everything, because from its position it is. Any tool returning a subset must say so explicitly in the response — a total, a truncation flag, something. This is not a prompt problem; it is missing information in the tool’s output contract.
The design test
If you are building something like this, the question to answer is: what would have to be true for a user to perform an action they are not permitted to perform?
If the answer includes “the model would have to ignore an instruction”, the design is not finished. Models ignore instructions. They are talked out of them by users who are trying, and occasionally by users who are not.
If the answer is “the tool would have to be present in a catalogue built from that user’s own permissions, and the controller behind it would have to skip its own check, and the user would have to confirm an action described to them in plain language” — then the model’s behaviour is no longer the thing standing between a user and an operation, which is exactly where it should not be.
The model’s job is to interpret a sentence. It should not also be the last line of defence.