AI Wrote the Code. Do You Understand What You’re Shipping?
TL;DR: When you work with AI, it can give you a solution that sounds right before you’ve thought it through. First, decide what you expect. Then check whether the code does that. Read the changes, understand the important logic, and know which decisions you’re signing off on.
An AI agent gives you a plan. It explains the approach, makes the changes, and tells you the tests pass.
Everything sounds reasonable. So you approve it.
Before you do, ask yourself: do you understand why this solution works? Could you explain what it changes and where it might fail?
That’s what stood out to me in Addy Osmani’s post about critical thinking when working with agents.
Decide what you expect first
Before asking AI to solve a problem, spend a little time thinking about the behaviour you want.
You don’t need to know the implementation yet. Start with what should happen, what must stay unchanged, and what could go wrong.
Say a request keeps timing out. Before asking an agent to fix it, decide what a good result looks like:
- The action should happen once.
- The user should know whether it succeeded.
- Retrying should be safe.
Now you have something to check the solution against. You can still change your mind, but you’re bringing your own understanding into the conversation.
If you read the agent’s answer first, it becomes much easier to follow its reasoning without considering other possibilities.
Read beyond the explanation
Suppose the agent adds retries and tells you temporary failures will now recover automatically.
That sounds like a sensible fix. But you still need to check what happens when the server completes the action and only the response gets lost.
Will the retry repeat the action? Could it create a duplicate? What will the user see?
Read the changed code with those questions in mind. Follow the request through the relevant parts of the system.
You can ask AI to explain anything you don’t understand. Then check that explanation against the implementation. Keep going until you can describe the important behaviour yourself.
Check what the tests actually prove
Passing tests are useful evidence. Look at what they cover.
A test might show that a failed request gets retried. That alone doesn’t tell you whether repeating the action is safe.
The implementation and its tests can share the same wrong assumption. If both assume every timeout means nothing happened, everything may pass while the real problem remains.
Choose a case that could challenge the solution. For this example, test what happens when the action succeeds but the response never arrives.
That gives you more useful information than asking the agent whether it’s confident.
Put your attention where mistakes matter
You won’t understand an entire large codebase at once. Focus on the change you’re accepting and the parts it depends on.
Read every changed line. For important logic, make sure you understand why it’s there. Give extra attention to changes involving permissions, stored data, payments, or behaviour users already rely on.
AI can help you trace that logic and investigate unfamiliar code. Use it to build your understanding.
Before you approve the change, try explaining it without reading the agent’s summary: what changed, why it should work, and what you checked.
If you can’t explain a part that matters, spend more time there. A convincing answer is a useful starting point. You still need to decide whether it holds up.