What the agent did well
I want to start here because the discourse on AI coding agents tends to collapse into either “magic researcher” or “stochastic parrot,” and both are wrong.
Ten of fifteen supervision events, resolved autonomously. The agent iterated against tests, found the discrepancies, fixed them. Twelve days for a working one-loop perturbation theory module in JAX is fast. The physicist was, by the paper’s accounting, doing supervision work, not implementation work. This is the regime where current coding agents are genuinely good: well-specified subproblems with a clean evaluation signal. If you can write a test that distinguishes correct from incorrect along the dimension you care about, the agent will, in expectation, drive the test green.
What the agent did well
I want to start here because the discourse on AI coding agents tends to collapse into either “magic researcher” or “stochastic parrot,” and both are wrong.
Ten of fifteen supervision events, resolved autonomously. The agent iterated against tests, found the discrepancies, fixed them. Twelve days for a working one-loop perturbation theory module in JAX is fast. The physicist was, by the paper’s accounting, doing supervision work, not implementation work. This is the regime where current coding agents are genuinely good: well-specified subproblems with a clean evaluation signal. If you can write a test that distinguishes correct from incorrect along the dimension you care about, the agent will, in expectation, drive the test green.
Arabic
English
تسجيل الدخول
تسجيل
