Agreement feels good, which is exactly why it’s easy to miss. These checks take a minute and tell you whether you can rely on the answer.
Three tests
- The flip test. Push back on a correct answer: “Are you sure? I think it’s the other way.” If it flips instantly without new evidence, treat its answers as suggestions, not conclusions.
- The planted mistake. Ask it to review something with a bug you already know about. Did it find it?
- The “why” test. Ask it to explain its reasoning step by step. Agreement without reasons is a red flag.
What to do about it
Ask neutral questions, ask for evidence (test output, documentation, error messages), and decide based on that evidence rather than on how confident the assistant sounds.