This text represents my personal opinion, not that of the NorthSec CTF volunteers as a whole.
Folks, we need to talk about AI agents and CTFs.
In the second half of 2025, AI models and tooling got a lot better. To simplify the writing, “AI agents” here will mean agents that can run commands and/or connect via MCP. It’s now possible for these AI agents to solve many CTF challenges semi- to fully autonomously.
Not just easy challenges, nosebleed challenges from the DEF CON CTF finals too. Today it’s possible and easy to set up Agent Swarms that run N AI agents tasked with solving challenges autonomously. In March, at Concordia’s @hackCTF, this was used “successfully” (not everyone appreciated it). Several people online say LLMs have “killed” CTFs.
That wasn’t a problem at NorthSec CTF 2025. The winning team didn’t rely on these agents. But this year, they confirmed to me that they’re strongly considering it, even if not everyone on the team is thrilled about it.
We need to seriously think about how to adapt the CTF to this new reality. Before getting into possible solutions, here are the variables I’m trying to optimize for: participants’ fun, and challenge designers’ fun.
Participants’ fun
For some, using AI agents is fun. AI agent finds flag, brain gives dopamine.
Image of a CTF player who just submitted a valid flag
For others, it takes the fun out of doing CTFs. Even so, it would be naive to think that because the fun is reduced, these people would decide not to use these tools. In the end, competitive people (~27% of participants) will use these tools, because otherwise they won’t stay competitive.
Data from the 2025 NorthSec CTF survey
For people who are mainly there to learn or to chill, it’s harder to predict. My intuition is that few will run several agents in the background, but that many will still use them in a semi-autonomous mode.
I worry that not restricting these agents will reduce the fun that many people have at the CTF, and that over the years, these people will stop coming back instead of simply not using these tools. On the other hand, restricting them would probably frustrate others.
Challenge designers’ fun
This is the harder part. Challenge designers are volunteers who put in hours and hours every year to build challenges meant to be solved by humans. This part is crucial, because challenge designers’ fun comes from watching humans rack their brains on their challenge, watching them learn new concepts, watching them suddenly realize that some obscure part of an RFC actually matters, and getting feedback from players complimenting their work.
We lose all of that if an LLM solved the challenge 90% or 100% of the way.
I’ve gotten comments from challenge designers at NorthSec and elsewhere questioning the point or their motivation to keep creating challenges for these reasons.
That’s serious, because if challenge designers have less fun making challenges, we’ll end up with fewer challenges. Fewer challenges means less variety, which in turn makes the CTF less attractive to participants.
I don’t know how to address that without restricting AI agents.
It would be easy to think that we just need to make challenges “AI-proof,” but that isn’t particularly fun for challenge designers to do either, and in the end, if it takes longer to build a flag, we’ll end up with fewer flags.
Possible directions
In no particular order, here are the ideas proposed so far:
- Ban all LLMs
- Ban AI agents
- Ban Agent Swarms
- Do nothing
- Limit which models are allowed
- Have two scoreboards, one for people using AI agents and one for people who don’t
- Do dynamic scoring so that “sloppable” challenges are worth zero points
- Adapt challenges so they work really poorly with LLMs
- Stop making easy challenges
- Make fewer “sloppable” challenges and more physical challenges
- Require write-ups for every challenge
I probably don’t need to say that I don’t like any of these solutions. Whatever the “solution,” it’s going to upset people or fail to address the issues mentioned above.
So what do we do?
For NorthSec CTF 2026, no action will be taken to restrict the use of AI agents, except one: we’re going to add ways to askgod to self-declare (manually or automatically) flags found by AI agents:
- A “Flag found mostly or entirely with an AI Agent” checkbox in askgod web
- A new
--agentCLI flag in the askgod client - Auto-detection of some common AI agents via environment variable (for example
CLAUDE_CODE=1) in the askgod client (opt-out available) - An MCP server to submit flags to askgod.
We’ll ask people using these AI agents to use these features on a voluntary basis.
The one and only goal is to gather telemetry to inform our decisions for 2027. With these additions, we hope to get accurate stats on which flags were solved mostly or entirely by AI, when, and by how many people. We’ll be able to easily figure out the AI/human ratio for a given flag and evaluate that ratio against its difficulty and the teams’ position on the scoreboard.
We’ll present this data at the CTF’s closing ceremony and consult the community for your opinion.
After all that, we’ll make a decision, and I’ll write a new blogpost explaining what we’re doing.
In the meantime, I invite you all to think about this new reality and share your opinions with me. We have an amazing community and I don’t want to lose it — I want to grow it 🩵
NorthSec CTF 2026 is going to be a great edition, and if you haven’t bought your tickets yet, you can do so here:)