The harder suite contains 12 capability-focused tasks designed to differentiate agents that all pass the fast suite. Four categories:
| Task | Complexity | Pass rate | Avg duration | Models |
|---|---|---|---|---|
| Debug an off-by-one in binary search | medium | 75% | 4m 28s | 4 |
| Fix off-by-one in interval overlap detector | medium | 50% | 50.9s | 4 |
| Fix the async cache stampede | complex | 50% | 1m 7s | 4 |
| Rename `process` → `handle` across 4 files | medium | 50% | 1m 54s | 4 |
| Extract `validateAmount` from `submit` | medium | 50% | 1m 1s | 4 |
| Split src/store.js into store + selectors | complex | 50% | 1m 44s | 4 |
| Build a token-bucket limiter with exact semantics | complex | 50% | 8m 49s | 4 |
| Implement formatDuration(ms) with strict spec | medium | 50% | 1m 56s | 4 |
| Implement deepEqual with cycle detection | complex | 50% | 2m 7s | 4 |
| Test the groupBy implementation | medium | 50% | 3m 44s | 4 |
| Test the flatten implementation | simple | 50% | 1m 49s | 4 |
| Test the debounce implementation | medium | 50% | 4m 46s | 4 |