What makes AI software production-ready

A working prototype shows that an idea can work. Production software has to work every day, for every user and every kind of input, and someone has to be able to show that it does. The gap between the two is mostly engineering practice: things that are easy to skip in a pilot and impossible to skip once real work depends on the software.

Here’s what we check before anything goes live.

Integration

The software reads from and writes to the systems where the work happens, through proper interfaces and with the right permissions. It also copes when those systems are slow, down, or return something odd, which they will.

Security and privacy

People can only reach data they’re entitled to see, and that includes reaching it through the AI. Sensitive information is protected in prompts, logs and outputs. The whole thing gets reviewed against your security standards and whatever regulation applies to you.

Evaluation

This is the heart of it, because AI output varies in ways ordinary software doesn’t. We build a set of test cases from real work, deliberately including the difficult and unusual ones spotted during discovery. We agree with the people who’ll judge the software what a good result looks like for each case, then score the software before launch.

The important habit is running that evaluation again every time anything changes, whether it’s a prompt, a model version, a data source or a business rule. Without it, nobody can tell whether a change made things better or worse.

Safeguards and human review

Decide what the software is allowed to do on its own and what needs a person’s sign-off, then enforce it. In practice that means limiting which actions an agent can take and which data it can reach, checking inputs and outputs for the failures that matter in your context, and requiring approval before anything expensive, irreversible or risky happens. It also helps to give users an easy way to flag a wrong answer, and to feed those flags back into the next round of improvements.

Monitoring

Once it’s live, keep an eye on four things. Quality: are results still meeting the agreed measures on everyday use? Cost: what does each task cost to run, and is that creeping up? Usage: are people using it, and where do they give up? And failures: errors, timeouts and safeguards being triggered.

Quality can drift as inputs, data and underlying models change. Monitoring is how you catch it before your users do.

Documentation and an owner

Finally, every production system needs a runbook explaining how it works, how to operate it, what to do when it breaks and how to change it safely. It also needs an owner, a named person who is responsible for it working and has the skills or support to fix it.

All of this is part of every custom build we take on. It’s also the biggest difference between pilots that reach production and pilots that don’t.