Counterfactual SQL rewards

A SQL query can match the answer on the one database an agent explored and still be wrong. Counterfactual databases, never shown to the agent, catch it. Pick a test question and a recorded run of Tiny Aya Global (base / SFT / GRPO), or edit the SQL, and see the checks and the reward each function gives. Questions marked ★ have a run that the single-DB reward accepts but a counterfactual rejects.

Loading Python in your browser (about 20–60 s on first visit)…