EXAONE-Deep Model Overview
About EXAONE-Deep
EXAONE-Deep is a series of open-source large language models developed by LG AI Research, optimized for enhanced reasoning capabilities. These models excel in mathematical reasoning, scientific understanding, and code generation, making them particularly suitable for complex problem-solving tasks.
Available in three sizes (32B, 7.8B, and 2.4B parameters), EXAONE-Deep models offer a balance of performance and efficiency for different deployment scenarios, from high-performance servers to more resource-constrained environments.
What sets EXAONE-Deep apart is its focus on reasoning tasks, where it outperforms many similarly-sized open-source models across various benchmarks related to mathematics, science, and code generation.
Model Specifications
EXAONE-Deep is available in three model sizes to accommodate different computational resources and use cases:
| Model | Parameters | Layers | Attention Heads | Vocabulary Size | Context Length |
|---|---|---|---|---|---|
| EXAONE-Deep-32B | 30.95B (excluding embeddings) | 64 | GQA with 40 Q-heads and 8 KV-heads | 102,400 | 32,768 tokens |
| EXAONE-Deep-7.8B | 7.77B (excluding embeddings) | 32 | GQA with 32 Q-heads and 8 KV-heads | 102,400 | 32,768 tokens |
| EXAONE-Deep-2.4B | 2.37B (excluding embeddings) | 24 | Standard attention with 16 heads | 102,400 | 32,768 tokens |
All models are built on a decoder-only transformer architecture and utilize a tokenizer with a 102,400 vocabulary size. The models support sequence lengths of up to 32,768 tokens, making them suitable for processing lengthy documents and complex reasoning chains.
Key Features
Mathematical Reasoning
EXAONE-Deep excels at various mathematical tasks, demonstrating strong performance on benchmarks such as:
- MATH-500
A benchmark of challenging mathematical problems across various domains
- AIME 2024/2025
American Invitational Mathematics Examination problems
- CSAT Mathematics
College Scholastic Ability Test mathematics problems
Scientific Reasoning
The model demonstrates strong capabilities in scientific problem-solving:
- GPQA Diamond
Graduate-level physics, chemistry, and biology problems
- Multi-step reasoning
Ability to follow complex chains of scientific reasoning
- Domain expertise
Specialized knowledge across physics, chemistry, biology, and other scientific fields
Coding Abilities
EXAONE-Deep performs exceptionally well in code-related tasks:
- LiveCodeBench
Strong performance on coding benchmarks requiring algorithmic problem-solving
- Multiple languages
Support for Python, JavaScript, Java, C++, and other programming languages
- Code explanation
Ability to explain code functionality and assist with debugging
Deployment Support
The models are designed for practical deployment:
- Multiple model sizes
32B, 7.8B, and 2.4B variants for different resource constraints
- Quantization support
AWQ and GGUF formats for optimized inference
- Framework compatibility
Support for various inference frameworks including Transformers, vLLM, llama.cpp, and TensorRT-LLM
Applications
EXAONE-Deep is designed for applications requiring strong reasoning capabilities:
Math Problem Solving
Solving complex mathematical problems with step-by-step reasoning, particularly useful for educational applications, tutoring systems, and research assistance.
Scientific Research Assistance
Analyzing scientific literature, generating hypotheses, and assisting researchers in exploring scientific concepts across various domains including physics, chemistry, and biology.
AI Inference Applications
Powering reasoning-focused AI applications that require logical thinking, problem-solving, and complex decision-making capabilities across various domains.
Research Background
EXAONE-Deep was developed by LG AI Research with the goal of creating large language models that excel specifically at reasoning tasks. The development focused on optimizing model architecture and training methodology to enhance performance on complex reasoning problems while maintaining efficiency.
The models leverage architectural innovations such as Grouped-Query Attention (GQA) in the larger models, which helps balance computational efficiency and model performance. The training process incorporated specialized datasets and techniques to enhance reasoning capabilities across mathematical, scientific, and programming domains.
EXAONE-Deep represents an important step in the development of specialized AI systems that can handle complex reasoning tasks that were previously challenging for language models, particularly in domains requiring structured logical thinking.
Related Resources
Technical Documentation
Detailed technical specifications and implementation guides
View Documentation