BigCodeArena: Judging code generations end to end with code executions

BigCodeArena: Judging code generations end to end with code executions BigCodeArena: Judging code generations end to end with code executions Team Article Published October 7, 2025 Upvote 22 Terry Yue Zhuo terryyz bigcode Evaluating the quality of AI-generated code is notoriously difficult. While humans can easily spot whether a piece of code "looks right," determining if it actually works correctly, handles edge cases properly, and produces the intended result requires running and testing it.

This ModelRelease is relevant to the technology intelligence record because it involves DeepSeek, GitHub, Hugging Face, Claude. The source article should remain the factual reference for follow-up coverage.

• BigCodeArena: Judging code generations end to end with code executions Team Article Published October 7, 2025 Upvote 22 Terry Yue Zhuo terryyz bigcode Evaluating the quality of AI-generated code is notoriously difficult.
• While humans can easily spot whether a piece of code "looks right," determining if it actually works correctly, handles edge cases properly, and produces the intended result requires running and testing it.
• This is why today, we're thrilled to announce BigCodeArena -- the first human-in-the-loop platform for evaluating code generation models through execution.
• Inspired by LMArena for LLMs, we've built a platform that allows anyone to compare code generation models side-by-side, but with a crucial difference: you can actually run the code and see what it produces .
• Just submit a coding task, watch two different models generate solutions, execute both programs, and vote on which model produced better results.
• The outcomes are organized into a leaderboard that displays the community's highest-rated models.