Blog

Write-ups on recent work from our group on multi-agent large language models, and read-throughs of papers we found worth the time.

Qwen3.5-Omni: An Omni Model That Acts in Real Time

2026

Paper read-through — streaming speech, 256k context, and a native omni agent

Most omni models are perception-response machines: hand them audio, get text back. Qwen3.5-Omni is built to act instead — calling tools mid-conversation and speaking while it thinks. The interesting engineering is what real time costs: ARIA fixes the clock mismatch between text and speech tokenizers, timestamps become plain text the model reads, and the answer you get by voice is distilled from the one you'd get in writing.

Paper ReadingMultimodalSpeechAgentsStreaming

Kimi K3: Scaling Both Axes at Once

2026

Paper read-through — a 2.8T open MoE with a 1M-token context

Open models have been scaling test-time compute while their pre-trained foundations sat still. Kimi K3 pushes both at once: 2.8T parameters, native vision, and a million-token context window. The architecture is elegant — linear attention with a bounded decay gate, attention over depth, 896 experts balanced by quantile matching — but the real story is the infrastructure that makes million-token agentic RL affordable.

Paper ReadingLLMMixture-of-ExpertsLong ContextSystems

DeepSeek-V4: Making a Million Tokens Affordable

2026

Paper read-through — compressing and sparsifying attention to the frontier

Test-time scaling runs straight into attention's quadratic cost. DeepSeek-V4 interleaves two new attention mechanisms — one that compresses then sparsifies, one that compresses far harder but stays dense — to serve a million-token context at 27% of the FLOPs and 10% of the KV cache of its predecessor, plus an unusually honest account of the loss spikes along the way.

Paper ReadingLLMAttentionLong ContextEfficiency

AURORA: Teaching LLM Agents When to Hit Undo

2026

Attention-based dual verification for reliable rollback

When an LLM agent takes a wrong step on a long task, the fix is to roll back — but knowing when is the hard part. AURORA reads attention patterns over the agent's trajectory to produce two signals, physical validity and overall coherence, and triggers a rollback only when one of them fires — recovering more tasks with fewer interventions.

LLM AgentsEmbodied AIVerificationReasoning

LangMARL: Giving Each LLM Agent Its Own Performance Review

2026

Credit assignment for teams of LLM agents

When a team of LLM agents fails, who should change what? LangMARL ports credit assignment from multi-agent reinforcement learning into language space: a centralized critic reads the whole episode and writes agent-specific feedback, driving a language-form actor–critic loop that improves coordination, scales to large teams, and grows complementary roles nobody assigned.

LLM AgentsMulti-AgentReinforcement LearningCredit Assignment

CoMAL: Can LLM Agents Talk Their Way Out of a Traffic Jam?

2026

Multi-agent LLMs for mixed-autonomy traffic

A handful of well-behaved autonomous vehicles can smooth out stop-and-go waves for everyone. CoMAL builds each vehicle as an LLM agent that perceives the road, negotiates roles in a shared message pool, and drives through a rule-based controller — beating human drivers and rivaling reinforcement learning, while staying interpretable.

LLM AgentsAutonomous DrivingMulti-AgentReinforcement Learning

Instructional Agents: A Faculty Meeting Inside an LLM

2026

Multi-agent LLMs for automated course design

Preparing a coherent course — objectives, syllabus, slides, scripts, assessments — usually takes a whole team. Instructional Agents simulates that team with role-specialized LLM agents working through the ADDIE framework, producing classroom-ready material for under a dollar, with a dial for how much human oversight you want.

LLM AgentsEducationMulti-AgentAI for Education