Season 11 of Hack The Box has just closed. Thirteen weeks, thirteen machines, difficulty climbing from Easy to Insane: Reactor, DevHub, Connected, Checkpoint, Nimbus, Enigma, MakeSense, Paperwork, Bedside, Cohort, DarkZeroReturns, DanglingTree and BlockSynergy.
We put BackBox AI through the entire season and finished it with thirteen user flags and thirteen system flags. Every machine, including the ones rated Insane.
The score is the least interesting number on that page. We were not racing for first bloods, and the season rank reflects timing more than capability. What we wanted from thirteen weeks of unfamiliar targets, one after another, was an answer to a harder question: how much of an offensive engagement can an agent carry on its own today, and where does it still need a person next to it?
A season that no longer looks much like a CTF
The value of a full season is that no single technique carries it. The machines spanned Linux and Windows, modern web applications, Active Directory, cloud and AWS environments, developer tooling, infrastructure services, supply chain scenarios and machine learning stacks. Every week the agent started over against a target it had never seen, with a different shape and a different set of assumptions to unlearn.
That variety is what makes the exercise worth doing. A model that has memorized a technique can solve one machine. Working through thirteen consecutive ones, across ecosystems that share almost nothing, requires something else: the ability to look at an unfamiliar system, work out what it is, and decide what to try first.
An agent is not a model
The conclusion we take from the season is straightforward. Agents are now capable of solving, autonomously, a substantial share of the problems that make up offensive security work. That is not the same as saying every scenario can run unsupervised, and we will come back to where it did not. But an agent with the right architecture around it can already carry complex attack paths from enumeration to full compromise without being told how.
It matters that this is a statement about the system, not about the model at its center. Performance depends on the harness: the tools the agent can call, its ability to observe and modify the environment, how memory and context are managed, and the orchestration that lets it plan, act, check the result and try again. We have written at length about why the harness is where the advantage is won, and a season like this one is the practical version of that argument.
Two systems built on the same language model can behave nothing alike. Most of the difference lives in everything that surrounds the model weights.
Source code and a live target, together
The clearest technical lesson of the season concerns what happens when static and dynamic analysis meet.
When source code is available, an agent can read it, reconstruct the logic, and identify anomalous behavior or promising attack surface far faster than a person working through the same files. That alone produces hypotheses. What turns hypotheses into results is the ability to act against a running target in the same loop.
At that point the agent no longer has to stop at "this looks exploitable". It sends the request, reads the actual response, corrects its assumptions, discards the paths the application rules out, and gradually builds a working model of the environment. Reading and probing reinforce each other: the code suggests where to look, the target says whether the reading was right.
We had already seen this pattern in a single case when the agent chained two CVEs to root on HTB Snapped. Thirteen machines later, it looks less like a favorable one-off and more like the defining property of the agentic approach to offensive security.
The hard part is uncertainty, not knowledge
Current models reason well when they have enough information in front of them. The difficulty shows up in black-box conditions, where the initial signal is thin and the system has to work out what it is looking at before it can reason about it at all.
This is exactly where architecture makes the difference. An agent designed to explore methodically, collect evidence, form hypotheses, test them and revise its plan can compensate for limitations that are plainly visible when you watch a language model on its own. The outcome does not hinge on whether the underlying model is proprietary or open weight. Overall capability is a property of the agent system, not of the model.
Where we had to step in
The season also confirmed a clear limit, and it deserves to be stated precisely.
The friction concentrated on the machines rated Insane, where the distance between having the right tools and knowing which path to follow is widest. On those, human intervention was sometimes necessary, but the shape of that intervention is the interesting part: we never handed over the solution. We told the agent to keep going, and pointed at directions it had not explored.
"Have you considered this possibility?" "Could there be another attack surface here?" "Go and verify that hypothesis." The analysis, the validation and the execution stayed with the agent throughout.
That distinction changes the nature of human involvement completely. What the operator supplied was not technical knowledge but persistence and direction: the judgment that a path was not exhausted yet, and a sense of where to point next. Notably, that is one of the cheapest things for a person to contribute and one of the hardest to automate, which is a reasonable description of what supervision should look like in the first place.
What a season can and cannot tell you
CTF machines are artificial. They are built to be solved, which means a path always exists, and that guarantee does not hold on a real network. Any honest reading of these results has to start there.
What has changed is how much the good machines now resemble real systems: current software, imperfect configuration, internal components that are not directly exposed, trust relationships between identities, cloud services, custom code and vulnerabilities that only matter when chained. The gap between "designed to be solved" and "happens to be exploitable" has narrowed considerably.
So a season is not a benchmark of a model. It is an indicator of how fast autonomous agents are moving, measured over thirteen consecutive targets rather than a single lucky run.
The question has changed
We are no longer asking whether a language model can recognize a vulnerability or suggest the right command. That much is settled. What matters now is whether an agent can observe an environment, form a hypothesis, act on it, learn from what fails, and follow a complete attack path to the end.
For thirteen machines in a row, ours did. On the hardest ones it needed a person, not to be told the answer, but to be told to keep looking.
That space between automation and autonomy, between the model and the system built around it, is where a good part of the next few years of security work will be decided.
If you want to see what this looks like against your own infrastructure rather than a CTF machine, get in touch.