Totally agree. We’ve seen similar weirdness when trying to build deterministic behaviors around LLMs. It’s fun at first…. Until you’re debugging something that just needed a if/else.
We’re now mixing prompts with conventional logic exactly for that reason, LLMs are powerful, but not magical.
I would think someone working for Anthropic would be quite aware of this too.
Either fix the prompt until it behaves consistently, or add conventional logic to ensure desired orchestration.