What it takes to let AI answer questions about your product
by Luke, Founder
AI chatbots are familiar enough now. What is less obvious is how you can give one useful, controlled access to your own product — so it can answer questions using live data, documentation and existing business logic.
Imagine an event management system. It knows who is working, whether profiles and health and safety forms are complete, whether accreditation has been collected and which communications have been sent.
An event manager might ask: “Are we ready for tomorrow?”
Today, answering that may mean checking several screens and reports. AI could answer in seconds.
But this is not mainly a chatbot problem. It is about giving AI controlled access to the right parts of the product, while checking every request against the current user's identity, permissions and context.
Expose capabilities, not the database
Rather than giving AI unrestricted database access, the product can expose narrowly defined tools such as:
- check incomplete crew profiles;
- check accreditation status;
- summarise health and safety completion;
- check whether a communication was sent.
Each tool should enforce permissions and return only the fields needed for that task. A health and safety check does not need to expose someone's address or emergency contact details.
These capabilities can be exposed through an API or a standard such as MCP — Model Context Protocol. The AI chooses the appropriate tool, but the application controls what it can return.
Different information needs different treatment
Consider: “What does the handbook say about collecting accreditation?”
and: “Has Sarah collected her accreditation?”
The first requires searching documentation, often called RAG, or Retrieval-Augmented Generation. The second requires current data from the live application which is where MCPs can be useful.
Both need access controls. Users should only be able to retrieve documents and application data they are authorised to see.
Keep the product in control
AI should not contain a second version of the product's business logic.
The existing application should continue to own:
- authentication and permissions;
- organisation and event boundaries;
- business rules;
- operational data;
- transactions and audit history.
AI becomes another interface to those capabilities, using the same rules as the rest of the product.
Permissions and data handling matter
If someone asks: “Show me the crew working at another event.”
the application, not the AI's instructions, must prevent unauthorised access.
The same applies to data sent to an AI provider. Organisations need to understand and approve how that data is stored, retained and used.
Start with questions before actions
A sensible progression is:
- answer questions from documentation;
- answer questions about live application data;
- combine sources to answer broader questions;
- eventually allow actions.
For example: “Who hasn't completed their health and safety form?”
might become: “Send them a reminder.”
Actions such as sending messages or changing records need stronger permissions, confirmation where appropriate and a clear audit trail. Starting with read-only access reduces risk while the team learns how users interact with the system.
Evaluate, monitor and protect it
AI features need more than traditional software tests. Use representative questions to check that the system:
- gets the right information;
- respects permissions;
- prevents cross-organisation and cross-event access;
- returns only necessary fields;
- avoids inventing answers.
It also needs guardrails against jailbreak attempts, inappropriate content and sensitive data being sent to external services.
And it must be monitored so we know which tools were called, what information was returned, how long requests took and what they cost.
AI becomes another way to use the product
The value is not the chat box itself. It is the ability to use existing product capabilities through natural language.
Instead of visiting four dashboards, an event manager asks: “Are we ready for tomorrow?”
The system might respond: 183 of 191 crew members are ready. Eight still need to complete their health and safety form.
The event manager could then even ask:
“Send those eight crew members a chase email!”
Pretty neat.
