A team running an early frontier model walked away from DEF CON’s flagship Capture The Flag contest in 2024 with a silver medal. Organizers and competitors were shocked. Two years later, security researchers have published analysis arguing that frontier AI systems now solve CTF challenges far faster than most human competitors, a shift that is forcing a fundamental redesign of the competitions that trained generations of security engineers.
The analysis, posted by a researcher who goes by Kabir and discussed widely on Hacker News, asks a question the community has been avoiding: what is a security competition for, when machines win it?
The speed gap
Kabir’s analysis centers on a simple observation: frontier models, particularly the reasoning-focused systems released over the past two years, solve CTF challenges in minutes what human teams take hours to crack, and they do it without the years of training that the best human competitors accumulate. The finding matches what has been visible at major competitions for a while. At DEF CON’s flagship CTF in 2024, a team running an early frontier model finished with a silver medal, a result that shocked organizers and competitors alike. Since then, autonomous hacking agents built on the same models have been demonstrated by major vendors, including Microsoft’s SA-1 system unveiled at DEF CON in 2025, which was designed to work unattended against real-world vulnerabilities.
The implications go beyond bragging rights. CTF challenges have always been a proxy for real security skill – the ability to read code, find flaws, and exploit them under time pressure. When a model can do that faster than a human, the proxy breaks. Recruitment teams that used CTF results to identify talent can no longer distinguish a human expert from a human running the right model, and competitors face the same ambiguity. The competitions were, in effect, a form of benchmark, and benchmarks fail the moment the test set leaks into the training data – except here the “leak” is the entire public history of the sport. The distinction matters for how the competitions are judged: in the past, a team that solved a challenge in minutes had clearly demonstrated skill, while today the same result could be produced by a competent engineer prompting the right model, and organizers cannot tell the difference.
What CTF was for
CTF competitions grew into the security industry’s most reliable talent pipeline. Universities field teams, companies sponsor them, and top finishers have historically walked into job offers at the best security firms. The format rewarded exactly the skills that defensive and offensive security require: deep systems knowledge, creativity under pressure, and the discipline to work in teams.
The rise of AI does not erase those skills, but it changes their relative value. If machines can solve the mechanical parts of a challenge – the pattern matching, the library lookup, the standard exploit chains – then the human contribution shifts toward what models cannot do: designing novel attacks, understanding business logic, and knowing when a system is worth attacking at all. Competitions that do not adapt risk becoming a measurement of who has the best model, which is a very different competition from the one that built the community.
Where the competitions go next
The community is starting to adapt, and the analysis has accelerated the debate. Three directions are emerging. The first is AI-versus-AI divisions, where models compete against models and humans judge the results – a format that tests model capability directly and keeps the sport’s spectacle intact. The second is a human-division model, with rules that restrict or flag AI assistance, which preserves the traditional format but invites constant enforcement problems. The third is a shift in what gets tested: challenges built around novel vulnerabilities and real-world systems, where the frontier models have less training data to lean on.
None of these is clean. The Kabir analysis, and the Hacker News discussion that followed, made the underlying tension explicit: the moment a competition’s challenges become solvable by a model, the competition has to change, and the change has to happen faster than the models improve. The stakes are not confined to the sport itself: the same models that solve CTF challenges are being pointed at real networks, and the way the competitions evolve will influence how the industry teaches the next generation of defenders, and how it evaluates the tools those defenders will use. The security industry has never had to update its talent pipeline on this kind of schedule.
Frontier AI has ended the CTF competition in its current form – not by banning it, but by winning it. The community that built the sport now has to decide what it is for: a test of humans, a test of machines, or something new that tests the collaboration between the two. The answer will determine how the next generation of security talent is found. The clock is running.
