BigCode

Introducing: StarCoder2 and The Stack v2 ⭐️

StarCoder2 is trained with a 16k token context and repo-level information for 4T+ tokens. All built on The Stack v2 - the largest code dataset with 900B+ tokens.

All code, data and models are fully open!

hf.co/bigcode/starco…

Introducing BigCodeArena, a human-in-the-loop platform for evaluating code through execution.

Unlike current open evaluation platforms that collect human preferences on text, it enables interaction with runnable code to assess functionality and quality across any language.

Releasing BigCodeBench-Hard: a subset of more challenging and user-facing tasks.

BigCodeBench-Hard provides more accurate model performance evaluations and we also investigate some recent model updates.

Read more: huggingface.co/blog/terryyz/b…
Leaderboard: huggingface.co/spaces/bigcode…

Introducing 🌸BigCodeBench: Benchmarking Large Language Models on Solving Practical and Challenging Programming Tasks!

BigCodeBench goes beyond simple evals like HumanEval and MBPP and tests LLMs on more realistic and challenging coding tasks.

Releasing StarCoder2 Instruct! 🚀

Achieves 72% HumanEval score using only self-generated content without any GPT-3.5/4 data. This work demonstrates that self-instruct works already well at the 15B scale without data from proprietary models!

Read more: huggingface.co/blog/sc2-instr…