AI-generated code security review

Risk one: the code, and the controls it stresses

Generated code does not introduce a new vulnerability class. It changes the rate and the shape of the old ones, and it does that faster than the controls around it were tuned for. Four controls take the strain first:

• Human review depth. Reviewers read a large, fluent, plausible diff differently from a small handwritten one. Review that was calibrated on the second does not transfer to the first, and approval rates rise while attention falls.
• Test adequacy. Tests generated alongside the code tend to assert the behaviour that was written rather than the behaviour that was required, so coverage rises without confidence rising with it.
• Dependency introduction. A suggested import is a supply chain decision made silently. The review looks at whether a new dependency can reach a merge without anyone deciding to take it on, and whether a package that does not exist can be installed on a typo.
• Secrets in generated code. Assistants produce plausible configuration, and plausible configuration contains plausible credentials. Real ones arrive when context includes them.

The practice set for this is NIST SP 800-218A, Secure Software Development Practices for Generative AI and Dual-Use Foundation Models, which extends the Secure Software Development Framework to generative AI, and the relevant failure category when generated output is consumed without validation is OWASP LLM05 Improper Output Handling.

Hardening the agent

• Repository scope. An explicit allowlist, not organization-wide access, and a separate machine identity rather than a developer’s personal token.
• CI credentials. Workflows triggered by agent-authored branches run with the least privilege available and no access to deployment secrets; secrets are exposed only to workflows that a human approval has released.
• Merge rights. None. Branch protection requires review by an identity that is not the author, and the agent is an author like any other.
• Branch protection. Enforced for administrators too, since the exception that exists for emergencies is the one an agent inherits when it runs under an administrator’s token.

A coding agent should hold the permissions of a new contractor on their first day: read and write on a working branch in the repositories it was scoped to, no merge rights, no production credentials, and no access to the secrets its own pipeline uses. The test of the model is simple: if the agent were fully compromised tomorrow, what would it reach, and which of that would anybody notice?

What reviewers should look for

Reviewer guidance is part of the deliverable because the reviewers are the control that is actually under load. The patterns worth naming: authorization checks that are present on the endpoint but not on the object; input validation that checks shape but not authority; error handling that swallows the failure path the test never exercises; a plausible-looking dependency nobody chose; configuration defaults that are convenient rather than safe; and code that consumes model output or external data and treats it as trusted.

An optional sample review grounds all of this in your actual output: we read a set of recent merged changes you select and report what the patterns are in your codebase rather than in general. That is a review of source you supply, read-only, and it is not testing of a running system — a distinction that matters for how it is scoped and what it can claim.

How the work is bounded

The scope is agreed in writing before work starts, and the engagement is quoted in writing with it.

SecHB does not issue certifications, attestations or audit opinions: those come from accredited certification bodies, CPA firms and QSAs. The work here is what an organization does to be ready for them.

Nothing here is legal advice. Where a question turns on the law, the work is done alongside the client’s counsel, not instead of them.

Questions we are asked

Should we stop using coding assistants?

Stopping is not the recommendation and would not hold anyway. The recommendation is to treat the assistant as what it is — a very fast contributor with no accountability — and to make the controls that catch a fast contributor’s mistakes strong enough to matter.

Does this replace a secure code review?

This is a review of the controls around generated code, not a line-by-line review of your application. A secure code review of a specific service is a separate engagement, and the two answer different questions.

What permissions should a coding agent actually have?

A coding agent should hold the permissions of a new contractor on their first day: read and write on a working branch in the repositories it was scoped to, no merge rights, no production credentials, and no access to the secrets its own pipeline uses.

Do you need repository access?

Repository access is not required for the control review, which runs on configuration, policy and pipeline definitions. It is required only for the optional sample review, and then read-only, on repositories you name.