Meta releases free ‘Code Llama 70B’ to challenge GPT-4 in AI coding race

Meta on Monday announced the release of its free code generation AI model and programming tool named Code Llama 70B. The new release is designed to generate and debug even larger programming strings compared to Meta’s previous offerings.

Code Llama 70B is “the largest and best-performing model” and one of the largest open-source AI models for code generation currently available, the company said. It was trained on 500 billion tokens of code and code-related data and has a context window of 100,000 tokens. Since the rollout of GPT 4 a few months ago, Meta has been working behind the scenes to close the gap in the AI coding race and take a bite out of GPT-4’s lead in code generation.

Code Llama 70B represents the latest version of Meta’s AI coding tool, building upon the foundation laid by their Llama 2 language model. With an impressive 70 billion parameters, it surpasses its predecessors in terms of size and capability.

According to Meta, Code Llama 70B is their most advanced code-generation AI model to date. Introduced in August, the Code Llama tools are freely available for both research and commercial use. As highlighted in a post on Meta’s AI blog, Code Llama 70B can handle a greater number of queries compared to earlier versions. This means developers can provide more prompts during programming, leading to increased accuracy in code generation.

In terms of performance, Code Llama 70B achieved a 53% accuracy score on the HumanEval benchmark, surpassing GPT-3.5’s 48.1% and approaching the 67% reported for GPT-4 in an OpenAI paper (PDF).

Mark Zuckerberg, Meta’s CEO, shared the news on Facebook, emphasizing the importance of AI models in coding and expressing pride in the advancements achieved with Code Llama.

“We’re open sourcing a new and improved Code Llama, including a larger 70B parameter model. Writing and editing code has emerged as one of the most important uses of AI models today. The ability to code has also proven to be important for AI models to process information in other domains more rigorously and logically. I’m proud of the progress here, and looking forward to including these advances in Llama 3 and future models as well,” Zuckerberg said.

Code Llama 70B is an extension of Llama 2, aiding developers in creating and debugging code strings from prompts. Last fall, Meta simultaneously launched two other Code Llama tools, Code Llama – Python and Code Llama – Instruct, each focused on specific coding languages.

Importantly, Code Llama 70B is accessible through three versions of the code generator and remains free for both research and commercial purposes. It was trained on a substantial 1TB of code and code-related data and is hosted on the code repository Hugging Face, which provides access to GPUs for running AI models.

Meta contends that its larger models, 34B, and 70B, deliver superior results, enhancing coding assistance. Notably, Code Llama 70B’s 53% accuracy score is a noteworthy improvement over GPT-3.5, positioning it as a formidable contender in the AI coding landscape.

Crucially, Code Llama 70B has sparked discussions about the democratization of AI and its potential impact on the software development industry. Unlike GPT-4, which is not publicly available, Code Llama 70B is completely free and open-source, making it a valuable tool for developers at all levels and fostering collaboration and development in the AI coding space.

The release of Code Llama 70B signals a significant milestone in AI programming, offering exciting possibilities for the future of software development. However, it also prompts important considerations regarding the role of AI in the workforce, with discussions about potential job displacement and ethical implications in AI coding.

Meta is not alone in this endeavor, as other AI developers, such as Amazon and Microsoft, have released code generators in the past year. Overall, the emergence of Code Llama 70B marks a noteworthy development, raising questions and expectations for the trajectory of AI in programming.