Software engineers are used to things being repeatable. Same input, same output. That's how we know we can trust our tests, builds, and deployments. When something behaves differently twice in a row, we treat it as a bug and fix it.
LLMs don't work that way. The same prompt can give you different answers. Sometimes the difference is small. Sometimes it's not. This can feel uncomfortable because it goes against what we're used to.
But it's not automatically bad. That variability can be useful when you're exploring ideas or looking for new approaches. The problem is when you treat an LLM's output the same way you'd treat code you wrote yourself or reviewed carefully.
Before LLMs, we already learned from imperfect sources — documentation, forums, blogs, coworkers. LLMs are just another source, faster and broader. But there is a difference worth paying attention to: a human author has context, intent, and accountability. An LLM has none of that. It generates text based on patterns. It doesn't know what's true or what matters. That doesn't make it useless. It just means you have to treat it differently.
Example: I was working on a Kubernetes deployment for a small cron job that writes to BigQuery. I needed to add a dry-run option. The LLM gave me YAML and code changes that looked fine at first glance. But it changed how some parameters were handled — strings became literals — and the deployment broke because it was looking for values that no longer existed.
That same week, the same model helped me simplify how we handle environment variables across deployments. We removed some messy custom logic and let Kubernetes do more of the work. It worked well.
Same tool. One fix, one new bug.
The takeaway:
- Treat LLM output as a suggestion, not a final answer.
- Syntax and formatting can be correct while the logic is wrong.
- Tests and linters help, but they don't catch everything — especially when the change looks reasonable.
- The engineer is still responsible for understanding what the code actually does and whether it fits the system.
Use LLMs to generate ideas and surface possibilities. But when you move from thinking to shipping, apply the same careful review you would to any other change. The LLM suggests. You decide. And you verify.