Counterfactual SQL rewards
A SQL query can match the answer on the one database an agent explored and still be wrong. Counterfactual databases, never shown to the agent, catch it. Pick a test question and a recorded run of Tiny Aya Global (base / SFT / GRPO), or edit the SQL, and see the checks and the reward each function gives. Questions marked ★ have a run that the single-DB reward accepts but a counterfactual rejects.
Loading Python in your browser (about 20–60 s on first visit)…
| Database | Gold | This query | Check |
|---|