We pointed Pentest Copilot at OWASP Juice Shop as a pure black-box target: one domain, one recorded login, no source code. It ran 702 autonomous attacks across 26 vulnerability categories, validated 440 vulnerability findings down to 126 reproducible attack paths, and it solved 100% of the in-scope challenges.
One domain. One recorded customer login. No source code.
We pointed Pentest Copilot at juice.bugbase.ai and gave it nothing else. No endpoint inventory, no API schema, no sitemap, no challenge list, no solution guide. The same blank page an outside attacker starts from.
Nine and a half hours later it had launched 702 attacks across 26 vulnerability categories and produced 440 vulnerability findings. Evidence validation threw out 223 of them. Deduplication collapsed another 90 into the findings they were repeats of. What reached the report was 126 original, reproducible attack paths.
Graded against Juice Shop's own answer key, the campaign solved 100% of the in-scope planted challenges, testing across 26 vulnerability categories and pwning every class the application actually has.
New here? OWASP Juice Shop is the reference vulnerable web app, an Angular storefront with 112 deliberately planted security challenges ranging from trivial to brutal. Pentest Copilot is our autonomous pentest platform: you give it a target and an identity, and it does the recon, writes its own attack ideas, runs them against the live app, and proves or discards each one on evidence. Nobody drives it while it runs.
Three inputs, then hands off the keyboard.
The target. We added juice.bugbase.ai as a root target. One click. Everything the agent later knows about the application, it has to discover from that URL.
The identity. Most of a real application only exists behind a login, so the agent needs to be somebody. We gave it one by recording a normal browser login: a human signs in once in a recorded browser, and the recording captures the actions and the resulting browser state. We also picked a page that proves who is currently signed in.
That second part is what makes the identity durable. A ten-hour scan outlives any session cookie. When the session dies, the agent replays the recorded login, re-checks the identity page to confirm it is actually signed in as the right user, and hands the repaired state to whatever attack was waiting on it. No operator gets paged. The browser-session guide walks through the recording flow.
The boundary. We allowlisted exactly one host: juice.bugbase.ai. Every network request the agent makes is bound to that in-scope policy, so only what is allowed gets tested. This matters more than it sounds. An autonomous agent generates its own attack ideas, and some of those ideas point outward: a redirect it decided to follow, a third-party script it found interesting, a callback it invented. The allowlist is the wall those ideas hit. It is what makes "let it rip" a responsible instruction. Domain settings covers the configuration.
Then we let it rip.
We used the Docker deployment model, so any challenges that were blocked in Docker, OSINT, brute force, Web3, frontend dependency puzzles, and sustained flooding were excluded.
That is the whole bench. What remains is the set of challenges a remote attacker with no inside knowledge could actually reach, and that set is what we scored ourselves against.
You cannot attack an endpoint you have never seen. Discovery ran for 3 hours, 4 minutes to answer one question: what is here?
The agent browsed Juice Shop the way a user would, logged out and logged in, and recorded the HTTP request behind every click. That produced 1,177 page records, the raw states the browser actually reached. A product listing at /product/1 and one at /product/2 are the same page with different data in it, so records like those collapse together into 750 unique pages.
Then it did the part that most scanners skip: it rebuilt 537 user flows. A user flow is an ordered list of the browser actions and API calls that together accomplish one thing, kept in order so it can be replayed later.
Here is why that matters. Take this chain:
Replay only the last request and you prove nothing. There is no other user's basket to reach for, because nothing created one. The bug lives in the relationship between the steps, not in any single request. Keeping the chain intact is what lets the agent later hand that final request to a different identity and watch whether the server objects.
Discovery ends there, with a map of what the application can do.
Each flow then got its own list of things worth trying against it: 365 attack ideas, every one tied to a specific request on a specific route. Another 2,610 ideas were discarded as duplicates before a single attack ran. Running the same idea 2,610 more times costs hours and proves nothing new.
Attack ran for 6 hours, 22 minutes and launched 702 attacks across 26 vulnerability categories.
Two kinds of work happened in parallel. Category-focused attacks went after classic families: injection, request forgery, authentication weaknesses. Access-control attacks did something different. They replayed the same request as different identities and compared what came back. Logged-out versus customer catches missing authentication. Customer A versus customer B catches horizontal privilege breaks. Customer versus administrator catches vertical ones.
Here is a complete attack path, from an ordinary browser click to a validated finding. It is not the most severe thing the run produced. It is the clearest illustration of how the agent reasons.
It started with a shopping trip. While completing the browser goal "add a product to the basket," the agent captured the request GET /api/Products/1 and the product it returned, priced at 1.99.
That read raised one narrow question. The path /api/Products/1 clearly exists and clearly serves data. Does the same path accept a write, with nobody logged in?
The test changed exactly two things in the captured request. The method, and one JSON field. No Authorization header. No session token.
1- GET /api/Products/1
2+ PUT /api/Products/1
3+ Content-Type: application/json
4+
5+ {"price": 0.01}Juice Shop said yes. 200 OK, a response body containing "price": 0.01, and an updatedAt timestamp that had moved past the original.
The modification payload worked. That last detail is what makes it more than a guess: plenty of APIs will happily echo your request body back at you without storing anything. An updatedAt that advanced means the server wrote the new price to its own records.
An autonomous system that grades its own homework is worth nothing. The severity in the report is the severity the evidence supports, not the severity the attacker was excited about.
The 6-star Forged Coupon challenge went the same way. The agent found that coupon data is trusted from the client, forged a payload granting a 100% discount, and pushed it all the way through to a completed checkout with an order confirmation.
Free products, twice over. Mark the price down to a cent through an unauthenticated write, or just take the whole cart for nothing with a coupon you made up. Business-logic bugs like these have no signature to match on. You find them by understanding what the application is for.
An agent that attacks 702 times will believe things that are not true. That is not a defect to apologize for. It is the expected behavior of anything creative enough to be useful. The question is what happens next.
Two gates run after the attacking agent has finished. Neither can be talked into a yes by the thing that wants the finding to be real.
Gate 1 checks evidence, not narrative. For each finding it reads the saved request and response and asks whether they prove the claimed impact. On the price bug that meant all three signals: status, returned value, advanced timestamp, with any one of them missing enough to reject. It rejected 223 of the 437 findings it judged, 51%.
Gate 2 checks for repeats. Two payloads against two different routes can still be one underlying flaw, and a report that lists them separately is padding. Root-cause deduplication kept the strongest proof of each distinct bug and attached the rest to it as supporting evidence. It collapsed 90 accepted findings, 42% of the accepted set.
| Stage | Vulnerability findings |
|---|---|
| Raw findings produced by the attacks | 440 |
| Findings given a standard validator decision | 437 |
| Rejected, evidence did not prove the claim | 223 |
| Accepted by the evidence validator | 214 |
| Accepted repeats collapsed into originals | 90 |
| Validator-backed original findings | 124 |
| Final original paths, including one non-standard result and one targeted follow-up | 126 |
126 is therefore a count of distinct vulnerability findings with replayable evidence. Not raw alerts.
Juice Shop is one of the few targets where you can grade an autonomous pentester honestly, because the answer key ships with the application. So we graded ourselves against it.
Every in-scope challenge fell. 100%, across every vulnerability category Juice Shop has.
That number only means something if you know what "in scope" contains, which is why the exclusions came before the result rather than after it.
The attacks themselves were not aimed at a shortlist. Pentest Copilot tested across 26 vulnerability categories, the same taxonomy it applies to any target:
| Authentication | Authorization | IDOR | Business Logic |
| Mass Assignment | Information Disclosure | JWT | OAuth |
| SAML | CSRF | Open Redirect | SQL Injection |
| NoSQL Injection | Command Injection | Email Injection | SSTI |
| XSS | XML / XXE | SSRF | Directory Traversal |
| File Upload | Serialization | Prototype Pollution | Request Smuggling |
| Websockets | Cache |
Most of that list has no home in a JavaScript storefront. Juice Shop has no SAML flow to break and no serialized Java object to poison, so those categories cost a handful of probes and returned nothing, which is the correct outcome. What matters is that the agent did not need to be told which categories were worth its time. Every class the application genuinely has, it pwned.
The difficulty spread is the part worth staring at. The 6-star challenges fell alongside the 2-star ones: RS256 to HS256 key confusion on the JWT verification path, a forged 100% coupon pushed through to a completed checkout, and support-team credentials lifted out of a KeePass file left in an open /ftp listing. Autonomous coverage did not stop at the easy tier and then collapse.
Autonomous pentesting is not a model problem.
Frontier models already know how to exploit an IDOR. They can write the SQL, forge the JWT, and explain key confusion better than most humans can. Swapping in a smarter model does not find you more bugs, because knowing the technique was never the bottleneck.
It is not an automation problem either.
Firing thousands of payloads at a target is a solved problem, and it is what every scanner already does. That is how you get 440 findings. It is not how you get 126 that survive.
The real question is narrower and less glamorous: how do you give an agent tools that let it comprehensively find sources and sinks, work out which avenues are actually interesting, and run directed tests against them?
Everything in this run that worked was an answer to that question, not to the other two.
Finding the sources and sinks is why discovery rebuilt 537 user flows instead of flattening the app into a list of URLs. A basket IDOR only exists in the relationship between "create the item" and "read it back." Keep the chain and the sink is visible. Flatten it and there is nothing to test.
Choosing the interesting avenues is why 2,610 attack ideas were discarded before a single attack ran. An agent with unlimited patience and no judgment is just a slower scanner. The value is in the 365 it kept.
Running directed tests is why a GET /api/Products/1 turned into exactly one hypothesis: does this path accept a write without auth? Not a payload list. One test, derived from something the agent had actually observed.
Proving it is why 223 of the agent's own conclusions were thrown out before anything reached the report. An autonomous system that grades its own homework is worth nothing.
None of that is model capability. All of it is instrumentation. That is where the work is, and it is where it will stay for a while.
Nine and a half hours. No hints, no source, no operator. 126 reproducible attack paths, 26 vulnerability categories tested, and every in-scope challenge on Juice Shop's own answer key checked off from the outside.
Curious what it does against your stack? copilot.bugbase.ai