Alibaba's Qwen team has released Qwen2, a new family of open-source large language models, with its largest variant making a direct challenge to top-tier proprietary systems like OpenAI's GPT-4. The team claims its 72-billion-parameter model, Qwen2-72B, not only surpasses leading open-source competitors like Meta's Llama 3-70B but also edges out some closed-source giants on specific benchmarks, particularly in coding and mathematics.
The release, detailed in a company blog post, represents a significant step in the high-stakes competition to build capable and accessible AI. For developers and businesses, the arrival of a powerful, permissively licensed model with strengths in technical domains offers a new alternative in a landscape long dominated by a few key players.
The Qwen2 Model Family
Qwen2 is not a single model but a series of five, scaled to meet different computational needs. The family includes models with 0.5, 1.5, 7, 57 (A14B), and 72 billion parameters. Each is offered as a base model and an instruction-tuned variant, designed for conversational tasks. This range allows developers to choose a model that fits their hardware and performance requirements, from mobile applications to large-scale server deployments.
The models were trained on a diverse dataset and demonstrate significantly expanded multilingual capabilities compared to their predecessors. The Qwen team reports extending pre-training data to include 27 additional languages beyond English and Chinese, improving performance on non-English tasks.
Key technical specifications include Group Query Attention (GQA) across all model sizes for improved inference efficiency and context windows extending up to 128,000 tokens for the larger instruction-tuned models. The smaller models are optimized for a 32,000-token context.
Most models in the series are released under the Apache 2.0 license, making them available for commercial use. However, the two largest models, Qwen2-72B and Qwen2-57B-A14B, carry a specific license. While still permissive for most commercial applications, it requires companies with over 100 million monthly active users to seek a license from Alibaba.
Challenging the Leaders on Benchmarks
The central claim of the Qwen2 release is its performance, especially that of the Qwen2-72B-Instruct model. Alibaba published a comprehensive set of benchmark results comparing its flagship against other leading open-source and proprietary models.
According to the published data, Qwen2-72B outperforms Meta's Llama 3-70B across a wide array of evaluations, including natural language understanding (MMLU), math (GSM8K, MATH), and coding (HumanEval, NaturalCodeBench).
The most notable claim is in the domain of code generation. The Qwen team reports that Qwen2-72B-Instruct scored 90.0 on the HumanEval benchmark in a zero-shot, pass@1 setting. This score not only surpasses other open-source models but also reportedly exceeds the published score for OpenAI's GPT-4. This positions Qwen2 as a potentially powerful tool for software development, from code completion and debugging to generating complex algorithms from natural language prompts.
In mathematics, another area that has historically challenged language models, Qwen2-72B also shows strong results. On the GSM8K benchmark, a test of grade-school math problems, it achieved a score of 93.6. On the more complex MATH benchmark, it scored 61.1, again placing it at the top of the open-source field and in close competition with leading proprietary models.
In general language tasks, Qwen2-72B holds its own. It scored 82.3 on MMLU, a comprehensive test of general knowledge, slightly trailing Llama 3-70B but remaining highly competitive. The model's multilingual performance was evaluated on benchmarks like TydiQA and FLORES-200, where it demonstrated strong capabilities across languages such as French, German, Spanish, Russian, and Arabic.
How to Access Qwen2
For developers eager to test these claims, the Qwen2 models are readily available. The team has made them accessible through popular platforms in the machine learning ecosystem.
All models can be found on Hugging Face, the central repository for open-source AI models. They can be integrated into projects using standard libraries like transformers.
For example, loading the Qwen2-7B-Instruct model with the transformers library can be done with a few lines of Python:
from transformers import AutoModelForCausalLM, AutoTokenizer
device = "cuda" # or "cpu"
model = AutoModelForCausalLM.from_pretrained(
"qwen/Qwen2-7B-Instruct",
torch_dtype="auto",
device_map="auto"
)
tokenizer = AutoTokenizer.from_pretrained("qwen/Qwen2-7B-Instruct")
The models are also available on Alibaba's own ModelScope platform. The team has provided support for various inference frameworks, including Ollama and vLLM, to streamline local deployment and experimentation.
Community and Context
The release has generated significant interest within the AI development community. The announcement on the Qwen blog was accompanied by active discussions on platforms like Hacker News, which saw the story attract hundreds of points and comments. The Product Hunt launch page for the model series also became a hub for early feedback.
Discussions have centered on the verification of benchmark claims, the practical implications of its strong coding performance, and its position relative to other major open-source families like Llama and Mistral. As with any major model release, community members are now in the process of conducting their own independent evaluations to see how the published benchmarks translate to real-world performance on their specific tasks.
The rise of Qwen2 underscores a broader trend: the globalization of cutting-edge AI development. While much of the initial LLM race was centered in North America, powerful models are now emerging from teams across the world, providing more options and fostering a more competitive and diverse ecosystem.
What to Watch Next
The immediate next step is for the AI community to put Qwen2 through its paces. Independent, third-party evaluations will be critical to validating the impressive benchmark scores published by the Qwen team. Real-world performance on novel, complex tasks will determine if the model's coding and reasoning capabilities are as robust as the initial data suggests.
Attention will also be on how competitors respond. The open-source AI space is moving at a rapid pace, with Meta, Mistral AI, and others continuously pushing the performance frontier. Qwen2's strong showing will likely accelerate this cycle of innovation.
Finally, the Qwen team has hinted that this release is just one part of their roadmap. In their announcement, they mention that even larger and more capable Qwen2 models are in development. The performance of this current 72B model sets a high bar for what might come next from Alibaba's AI division.