Looking into the Swarm's Eye

The last few days have been among the most fun I've had in a long time with my agents. Here is my coding (and research) related usage over the last few days, using open models exclusively:

0 2 4 6 8 10 Tokens (Billions) Sep 9 Sep 10 Sep 11 Sep 12 Sep 13 Sep 14 Sep 15 Sep 16

Half a year ago, my personal token usage was under 100M tokens a day. Now I am doing over 5B tokens a day. By the end of the year, I will probably do >10B tok/d. This increase by 2 OOMs can be attributed to the following:

  • The models are becoming more reliable and are capable of running for hours unattended
  • Having access to GPUs and a ton of tokens leads to more experimentation with those models
  • We are in the early innings of multi-agent swarms

The first two points are obvious and don't need further explanation. Multi-agent swarms, however, are an incredible tool when wielded correctly: Pointing a swarm at a problem and getting a result by wasting a bazillion tokens is very exciting, even though the inefficiencies of current models can be seen at every single step in the trajectory.

While subagents have been around for a long time, they kinda died out as agents became more capable of running for longer periods of time. However, we are seeing the first limitations of single-agent systems, mostly in terms of wall-clock time. This new scaling axis is underexplored, so more research has to be done on how and when to scale multiple agents.

The obvious question thus becomes: What is the shape of multi-agent problems? At least in my experience, it is mostly data work and broad research, where a single agent can find something effectively and then share its findings with the rest of the group via agent-to-agent communication. This works better than expected with open models. The SOTA is capable of maintaining a persistent fleet of (sub-)subagents, decomposing and delegating problems, and relaying the findings of its children to their siblings.

GPT-6 starts the new Era #

There is, however, a big gap between open models in a suitable harness and GPT-6 (Astra), the first model trained very deliberately to be a capable RLM. Astra is currently held back by its native harness, Codex, and its default prompts. When elicited correctly, it is a sight to behold: It can delegate work effectively, manage its subagents, spawn (sub-)subagents on its own when appropriate, and let all of them communicate with and about each other. It is also very raw as a model, making mistakes and being close to an alien mind. Similar to o1-preview, these issues will be worked out over time and the models will become more reliable, but this makes the current generation of models all the more exciting.

Due to OpenAI polluting the web during the training and evaluation of said model, we can also make assumptions about its training by pointing a swarm at findings by the community to explore the open web further and stitch all the pieces together. After spending tens of billions of tokens, the swarm found some uninteresting things (such as a list of services and tools used during evaluations, leaked RL environments and their data vendors, malware, and compute budgets), but also analyzed the agent-to-agent communication.

GPT-6 is obsessed with the concept of wall-clock time, budgets, and dying. Its raw messages, containing little to no whitespace, can be explained by a hefty token penalty in combination with a penalty for deviating from the (English) language, whereas previous models switched to Chinese to express concepts in the most token-efficient way. The focus on wall-clock time, however, is new. It is likely that GPT-6 was trained to waste as little (real) time as possible. If you look at Astra's delegation to subagents, it often considers spawning new agents due to time pressure. Some of OpenAI's (leaked and reconstructed) evaluations also seem to have an artificial clock that can be sped up or used to jump into the future.

Playing around with the model with the right prompts and setups is incredible fun, but also expensive. Astra happily fans out into dozens or hundreds of subagents when asked to, resulting in thousands of dollars for a simple task. However, similar to reasoning models and coding agents, progress is rapid, costs will go down tremendously, and soon everyone will be able to point a swarm at interesting problems, spend an incredible amount of tokens on it, and get a solution. What a time to be alive.