A vendor demo shows what a tool does on a good day. Before people rely on it, you need to know what it does on an ordinary day and on a bad one. You can learn a lot with a spreadsheet and a few hours.
Start with real tasks
Collect 30 to 50 examples of the work you want the tool to do, taken from real cases with personal details removed. Made-up questions tend to be cleaner and easier than real ones, so they flatter the tool.
Write down what a good answer is
Do this before you run anything. For each example, note what a correct answer must include and what it must never say. If two staff members can’t agree on the right answer, the tool won’t be able to meet it either.
Include the cases most likely to go wrong
- Questions your information doesn’t answer: does it admit that, or invent something?
- Questions outside its job: does it stay in its lane?
- Misleading questions that assume something false
- People asking it to break your rules
- Different wording, spelling and levels of English for the same request
Decide your pass mark by the cost of being wrong
A tool that drafts internal notes for a person to check can afford more mistakes than one that answers the public on its own. Agree the pass mark before you see results, or it is tempting to accept whatever you get.
Check the things that aren’t answers
Find out where your data goes and whether it is used for training. Work out the likely monthly cost at real usage. Confirm who is responsible when the tool gets something wrong, and how someone affected can challenge the result.
Run it again
AI systems change when models, prompts or your documents change. Keep your examples and rerun them after each change. A test you can repeat is worth far more than one impressive trial.