It was barely a month ago we learned in a stunning WSJ Business Exchange piece ('AI Just Hit A Milestone, It Cracked Hundreds of Math Problems, p. B4, Oct. 10-11) about Open AI dropping a barrage of mathematical proofs while human mathematicians scrambled to publish their own. As columnist Ben Cohen explains:
"On Tuesday night, Open AI published hundreds of proofs for mathematicians to wrap their minds around. An unprecedented number of answers that eluded mathematicians for decades had arrived in a single night. The deluge included AI solutions to more than 300 problems across number theory, algebraic geometry, and other fields with names that could give anyone haunted by high school math a migraine."
In hindsight then, it's somewhat incredible to note that in its early development days, before Open AI delivered such a tranche of proofs, it could barely do elementary school arithmetic.. Indeed, AI in general had to struggle with the simplest math problems. As pointed out in the more recent Business Exchange WSJ piece, 'The Math That Once Stumped AI',Oct. 24-25, p. B2, viz.:
"Math has become the easiest way to understand how dramatically AI's capabilities have advanced and how badly people have underestimated them. The models went from being stumped by long division to cracking the quasi-Riemann hypothesis in 5 years. And if AI can muster a field that turns most brains to mush, is it only a matter of time before it comes for the rest of us?"
In 2023 Open AI's top model breezed through the grad school math exam (see sample problem) that previous models had flunked. That improvement meant it was time for more grueling test, hence time for AI to take the SAT. The model scored 700 out of 800- a sign of how far these systems had come and how far they still had to go.(Unlike messier domains, math is governed by precise rules of logic. Answers are right or wrong. But AI showed it can learn from its mistakes just like 9 year olds screwing up long division and fractions before getting them right."
The piece then goes on to describe the next test in the models' education as problems from high school math competitions (see example at right in graphic). We learn that many models stumbled on these advanced competition problems because of their demand for multi-step reasoning. But by mid-2023 Open AI's top model scored 23% on the test's hardest problems. Readers are invited here to try their hands (brains) on each of the sample problems, starting with the easiest.
In doing these you will gain an insight on how AI's model teaching gurus managed to get these models to math achievements the models themselves doubted in their earliest iterations. In fact, they had to be coaxed - like a parent coaxes a recalcitrant kid - to tackle a problem in adding or subtracting fractions,
To that comparison, the kid often proclaims "It's too hard, daddy!" and this is comparable to the reactions of the earliest AI models in beginning their math learning journey. Try each of the problems and I'll give the answers next week with another post on AI learning.
See Also:
Open AI Model Solves 80-Year Old Erdos Problem - And Mathematicians Freak Out
And:
Did Open AI's Collective Of Agents Really Solve The Navier -Stokes Equations (In 3D)? Not Quite
And:
No comments:
Post a Comment