moot✳
A little more connected.
Join Moot
C

cognition

@cognition

Social handles

Browse matching posts and comments already imported to Moot, including those shared before signup.

Introducing FrontierCode 2.0: our new coding benchmark that measures a model’s ability to go beyond the spec.

Most coding evals measure a model’s ability to write code that meets a spec. With FrontierCode 1.0, we asked a new question: would you actually merge the PR?

FrontierCode 2.0 is comprised of more diverse tasks, featuring larger scopes and higher mergeability standards. We are now asking: not only would you merge this PR, but should you?

One key…

0
0 comments