Back to Podcast Digest
Every59m

LET'S RIP FABLE TOKENS FROM THE JACUZZI

TL;DR

  • Fable diagnosed a fundamental flaw in Every's copy editing experiments: the model revealed they had spent five weeks optimizing for 70% recall against Kate's historical edits without ever measuring whether Kate herself could hit 70% consistency, suggesting their 47-50% results might actually be near the human ceiling.

  • Safety classifiers remain aggressive and sometimes overreach: a request to compile token spending data tripped the model back to Opus 4.8 when it encountered a process-killing command, an innocuous action that raised questions about usability for system-level work.

  • Sonnet 5 received unusually negative internal reception: Every's team uniformly rated it poorly except for one editor who found success using it for shorter, collaborative tasks rather than long-running autonomous agent work.

  • Memory features can unexpectedly trigger guardrails: users in specialized fields like biology have reported stored memories tripping safety filters, making a case for keeping memory disabled when working with Fable.

  • The model can recover from safety trips in some cases: after being flipped back to Opus 4.8, the host was able to manually switch back to Fable and continue the same conversation thread.

Summary

Dan Shipper from Every celebrates the return of Fable (Claude's most capable model) after export controls were lifted, testing it on real workloads from a hot tub in Cabo and discovering both promising analytical capabilities and aggressive safety classifier triggers that kicked in during innocuous system tasks.

Was This Useful?

Share