An AI agent that can probe your applications around the clock sounds like exactly what an overstretched security team needs. It also raises the obvious question every CISO asks first: what stops it from doing something you didn't intend, on a system you can't afford to break?
The honest answer is that autonomy has to be bounded. In Plainsea, the agent works inside limits you set before the run, at a pace you choose, and – if you turn on operator approval – it stops before every action and waits for your decision. This article walks through those controls, in the order you meet them.
What Is human-in-the-loop AI pentesting?
Human-in-the-loop AI pentesting is penetration testing in which an AI agent does the testing, while people set its limits and can approve or deny each action before it runs.
That approval is a choice, not a requirement. With it switched off, the agent tests on its own – within the scope, mode and pace you've defined. For critical systems, where you want a person to agree to every step, you switch it on.
In Plainsea that comes down to four layers: a defined scope with explicit exclusions, a testing mode and pace suited to the target, optional per-action operator approval, and the ability to pause or stop a run at any moment.
Before the run: decide where the agent may go
Targets come from your asset inventory
In the Pentest Scope Builder you choose exactly what the agent tests: a single target from your asset inventory, or a saved scope – a group of assets you have defined. Only licensed assets can be targeted, and a scope can only be selected when it is active, which requires all of its assets to be licensed and verified.
Ownership verification is built into the asset inventory, with four methods: a DNS TXT record, an HTML meta tag, an HTTP response header or a well-known JSON file.
Exclude what must not be touched
Anything you add to the out-of-scope list is excluded from the test – either an asset from the inventory or a host you type in by hand, such as admin.example.com.
Give the agent only the access you intend
Authentication roles are optional. Without them, the agent tests only what an anonymous visitor can reach. Add a regular user and an admin role, and it can test access control across privilege levels – for example, replaying one role's requests with another's session to find IDOR and BOLA issues, or trying admin-only actions as a regular user.
Choose how hard the agent pushes
Testing mode
Two modes calibrate the agent's behaviour to the environment:
- Regular – conservative pacing, high-confidence payloads only, non-disruptive. Intended for production.
- Aggressive – high-velocity discovery for finding deep logic flaws. Intended for staging and test environments.
Testing pace
Throttling controls how much the agent slows itself down between actions, so its traffic fits what the target can handle:
| Preset | Minimum delay between tool calls | Max calls per minute per target | Burst |
|---|---|---|---|
| Stealth | 3 seconds | 10 | 2 |
| Normal | 0.5 seconds | 60 | 10 |
| Fast | 0.1 seconds | 300 | 20 |
| None | – | Unlimited | – |
Stealth is designed to stay under rate-limiting and detection systems and suits fragile production systems; Normal is the right default for most engagements. If no preset fits, you can set the delay, rate and burst yourself.
Timing
A test can run immediately, overnight at 02:00 to stay out of peak traffic, at 08:00 on the next business day so your team is on hand to respond, or on a custom schedule – once, daily, weekly or monthly.
A final review before launch
Nothing starts until you've seen the whole plan in one place: targets and exclusions, authentication roles, schedule and agent settings. Operator approval and automatic publishing can still be switched with a single click on this screen.
Launching also means confirming that you are authorized to test these targets. Testing only systems you have permission to test is a basic rule of professional pentesting – the platform simply makes it part of every launch.
During the run: approve actions before they run
How closely you supervise the agent is your choice, test by test. By default it runs autonomously within the scope, mode and pace you've set. Turn on Operator Approval – before launch or at any point during a run – and it pauses before each tool call and waits for your decision.
For every pending action you see:
- the tool the agent wants to use,
- the tool group it belongs to,
- the exact arguments it will run with.
You then choose:
- Allow – run this call.
- Allow for Run – run it and auto-approve matching calls for the rest of this run, so you aren't asked about the same thing twice.
- Deny – the call doesn't run.
Pending approvals appear at the top of the run's log, with a counter on the Logs tab so they're hard to miss.
When to turn it on – and when not to
Operator approval is most useful where a mistake would be costly: production systems, a target you are testing for the first time, or a new configuration you want to watch in action. On staging and lab environments, or on targets you test regularly with settings you already trust, autonomous mode gets you full coverage without anyone having to sit with the run.
You don't have to decide once and for all. Switch it on or off while a run is in progress – for example, approve the first actions by hand, then hand over to auto mode once you've seen enough.
Pause, resume or stop
A running test can be paused and resumed at any point. Stopping is final and asks you to type STOP to confirm – you keep the partial results, but a stopped run can't be resumed.
See what the agent is doing
While a run is in progress you can follow its progress, the endpoints it has discovered, its current intent, an attack diagram and the full log. Findings it is still working on are shown as working hypotheses – they aren't treated as confirmed until the agent completes the full test cycle.
After the run: review before anything is published
Findings can be published automatically or held for manual review. Turn automatic publishing off and nothing reaches the wider team until someone has looked at it.
Each run also keeps a record of the configuration it was started with – complexity, risk level, pace and environment – so you can always tell exactly how a result was produced, even if the test's settings have changed since.
Recommended Settings for Production Systems
| Setting | Recommendation |
|---|---|
| Testing mode | Regular |
| Testing pace | Stealth for fragile systems, Normal otherwise |
| Operator Approval | On, at least for the first runs |
| Out of scope | Every host that must not be touched |
| Schedule | Next business day 08:00, so your team can respond |
| Automatic publishing | Off, so findings are reviewed first |
For staging and lab environments, Aggressive mode, a faster pace and autonomous mode get you more coverage in less time.
Frequently asked questions
Can an AI pentest agent be used safely on production systems?
That depends on the controls around it. Restrict the scope, exclude sensitive hosts, choose a conservative mode and pace, and turn on operator approval so nothing runs without a person agreeing to it.
Do I have to approve every single action?
No. Operator approval is optional and can be switched on or off during a run. "Allow for Run" also approves matching calls for the rest of that run.
Does the agent replace human pentesters?
No. It takes over repetitive testing at scale, while people decide what is tested, approve sensitive actions and review the findings. For more on combining human expertise with automation, see Rethinking Pentesting: Why Modern Security Teams Need an Augmented, Continuous Approach.
Autonomy you can trust
An autonomous agent is only as useful as the trust you can place in it. Bounded scope, deliberate pacing and optional per-action approval turn AI pentesting from a risk you have to accept into a capability you control.
See how it works on the Plainsea platform, or book a demo to try it on your own environment.
