Search

Epoch AI Launches FrontierMath AI Benchmark to Test Capabilities of AI Models

FrontierMath is a benchmark for evaluating advanced mathematical reasoning in AI.

Advertisement
Highlights
  • FrontierMath was created in collaboration with over 60 mathematicians
  • The test comprises algebraic geometry to Zermelo–Fraenkel set theory
  • The company said older benchmarks do not truly test AI capabilities
Epoch AI Launches FrontierMath AI Benchmark to Test Capabilities of AI Models

The problems in FrontierMath are new and unpublished to avoid data contamination

Photo Credit: Epoch AI

Epoch AI, a California-based research institute launched a new artificial intelligence (AI) benchmark last week. Dubbed FrontierMath, the new AI benchmark tests large language models (LLMs) on their capability of reseasoning and mathematical problem-solving. The AI firm claims that existing math benchmarks are not very useful due to factors like data contamination and AI models scoring very high scores on them. Epoch AI claims that even the leading LLMs have scored less than two percent on the new benchmark.

Epoch AI Launches FrontierMath Benchmark

In a post on X (formerly known as Twitter), the AI firm explained that it collaborated with more than 60 mathematicians to create hundreds of origins and unpublished math problems. Epoch AI claims that these questions would take even mathematicians hours to solve. The reason behind developing the new benchmark was cited as the limitations with existing benchmarks such as GSM8K and MATH, where AI models generally score a high point.

The company claimed that the high scores achieved by LLMs are largely due to data contamination. This means the questions somehow were already fed into the AI models, resulting in them easily solving the questions.

FrontierMath solves the problem by including new problems that are unique and have not been published anywhere, mitigating the risks associated with data contamination. Further, the benchmark includes a wide range of questions including computationally intensive problems in number theory, real analysis, and algebraic geometry, as well as topics such as Zermelo–Fraenkel set theory. The AI firm says all the questions are “guess proof”, meaning they cannot be solved accidentally without strong reasoning.

Epoch AI highlighted that to measure AI's aptitude, benchmarks should be created on creative problem-solving where the AI has to maintain reasoning over multiple steps. Notably, many industry veterans believe that the existing benchmarks are not sufficient to correctly measure how advanced an AI model is.

Responding to the new benchmark in a post, Noam Brown, an OpenAI researcher who was behind the company's o1 model welcomed the new benchmark and said, “I love seeing a new eval with such low pass rates for frontier models.”

For the latest tech news and reviews, follow Gadgets 360 on X, Facebook, WhatsApp, Threads and Google News. For the latest videos on gadgets and tech, subscribe to our YouTube channel. If you want to know everything about top influencers, follow our in-house Who'sThat360 on Instagram and YouTube.

 
Show Full Article
Please wait...
Advertisement
Popular Mobile Brands
  1. Oppo Reno 14FS 5G Design and Specifications Leaked: Check Expected Price
  2. Samsung Galaxy S26 Series Could Offer These Alternatives to Google Gemini
  3. Google Pixel 10 Pro Series, Pixel Buds 2a, Pixel Watch 4 Design Leaked
  4. Smartphones Launched in India (July 2025): Check List
  1. Oppo Reno 14FS 5G Price, Design and Specifications Leaked Ahead of Anticipated Debut
  2. Samsung Galaxy A07 Listed on Google Play Console With Design, Key Specifications: Report
  3. Microsoft SharePoint Hack: Probe on Whether Chinese Hackers Found Flaw via Alert
  4. Samsung Reportedly in Talks With OpenAI, Perplexity to Offer Gemini AI Alternatives on Galaxy S26 Series
  5. Redmi 15 Design Renders, Specifications Leaked; Tipped to Arrive in Three Colourways
  6. Google Pixel 10 Pro, Pixel 10 Pro XL Spotted in Moonstone Colourway Alongside Pixel Buds 2a and Pixel Watch 4
  7. Meta Names ChatGPT Co-Creator Shengjia Zhao as Chief Scientist of Superintelligence Lab
  8. Who-Fi: An AI-Powered Wi-Fi Technology That Can Identify and Track Individuals Without Cameras
  9. NASA’s X-59 Moves Closer to First Flight with Advanced Taxi Tests and Augmented Vision
  10. Unusual Plasma Waves Above Jupiter’s North Pole Can Possibly Be Explained
Gadgets 360 is available in
Download Our Apps
App Store App Store
Available in Hindi
App Store
© Copyright Red Pixels Ventures Limited 2025. All rights reserved.
Trending Products »
Latest Tech News »