OpenAI has introduced GPT-4o mini, a cost-efficient small model aimed at making AI more accessible. Priced at 15 cents per million input tokens and 60 cents per million output tokens, it is significantly cheaper than previous models, including GPT-3.5 Turbo. GPT-4o mini scores 82% on the MMLU benchmark and outperforms GPT-41 in chat preferences on the LMSYS leaderboard.
Key Features
- Cost and Latency: Enables a broad range of tasks, including chaining or parallelizing multiple model calls, handling large volumes of context, and providing real-time text responses.
- Support: Currently supports text and vision in the API, with future plans to include text, image, video, and audio inputs and outputs.
- Context Window: Offers a context window of 128K tokens and supports up to 16K output tokens per request.
- Multilingual: Improved tokenizer for cost-effective handling of non-English text.
Performance
- Reasoning Tasks: Scores 82.0% on MMLU, outperforming Gemini Flash (77.9%) and Claude Haiku (73.8%).
- Math and Coding: Excels in mathematical reasoning and coding tasks, scoring 87.0% on MGSM and 87.2% on HumanEval.
- Multimodal Reasoning: Strong performance on MMMU, scoring 59.4%.
Safety Measures
- Pre-Training: Filters out undesirable content like hate speech and spam.
- Post-Training: Uses reinforcement learning with human feedback (RLHF) to align the model’s behavior with policies.
- Expert Evaluation: More than 70 external experts tested the model for potential risks, which were addressed.
- Instruction Hierarchy: First model to apply this method, improving resistance to jailbreaks and prompt injections.
Availability and Pricing
- APIs: Available in Assistants API, Chat Completions API, and Batch API.
- ChatGPT Access: Free, Plus, and Team users can access GPT-4o mini starting today, replacing GPT-3.5. Enterprise users will have access starting next week.
- Fine-Tuning: Plans to roll out fine-tuning for GPT-4o mini in the coming days.
GPT-4o mini aims to democratize AI by making it more affordable and accessible while maintaining high performance and safety standards.
