Benchmarked models

SWE-2 High, explicitly selected from the authenticated Devin CLI model catalog. Catalog lists this route as free at the September 2026 run.

11/11 done

DeepSeek V4 Flash free tier, run headlessly via the local opencode CLI (`opencode run -m opencode/deepseek-v4-flash-free`) — the same CLI this report is authored with.

7/11 done

Qwen3 8B Q4_K_M — local on M5 Max 128GB via Ollama Metal (81 tok/s, 26ms TTFT).

7/11 done

Qwen3 14B Q4_K_M — mid-size dense on M5 Max (46 tok/s).

7/11 done

Qwen3.8 27B MLX — native MLX on M5 Max, 94.7 avg with thinking (24 tok/s), 69.1 without — 256K ctx, multimodal, best local quality.

7/11 done

Gemma 3 4B Q4_K_M — fastest single-GPU local (132 tok/s on M5 Max).

7/11 done

Gemma 3 12B Q4_K_M — sweet spot reasoning on M5 Max (55 tok/s).

7/11 done

Gemma 3 27B Q4_K_M — most capable Gemma on a single GPU (24 tok/s).

7/11 done

Google DeepMind instruction-tuned MoE with near-31B quality while activating only 3.8B per token.

7/11 done

Google DeepMind 30.7B dense multimodal model with 256K context and configurable thinking mode.

3/11 done

Ornith 1.5 35B — 112 tok/s on M5 Max, 256K ctx, self-improving foundation model, Text+Vision.

7/11 done

Open frontier reasoning and orchestration MoE on a hybrid Transformer-Mamba architecture.

7/11 done

120B-parameter open hybrid MoE that activates 12B per token for complex multi-agent work.

7/11 done

Small, compute-efficient MoE for building specialized agentic AI systems.

7/11 done

Open multimodal 30B-A3B model designed as a perception and context sub-agent.

7/11 done

12B open multimodal reasoning model for video understanding and document intelligence.

8/11 done

9B dense LLM trained from scratch as a unified model for reasoning and non-reasoning tasks.

7/11 done

Open-weight 21B MoE released under Apache 2.0 with 3.6B active parameters per forward pass.

7/11 done

Cohere’s first agentic coding model, optimized for code generation and repo-level work.

7/11 done

MoE from inclusionAI for responsive agents, instruction following, and multi-turn conversation.

7/11 done

Latest Poolside coding agent model, scoring 70.2% on Terminal-Bench 2.1.

7/11 done

33B-A3B coding agent model from Poolside, a step forward from Laguna XS.2.

7/11 done