Meta unveils Llama 3.1, the most advanced open-source AI model to date

July 23, 2024 at 5:22:13 PM - Trending 🔥

TL;DR Meta introduced Llama 3.1 405B, its largest open-source AI model with 405 billion parameters, competing with GPT-4o and Claude 3.5 Sonnet. Trained on 16,000 Nvidia H100 GPUs, it's available on cloud platforms and used in WhatsApp and Meta.ai. It handles tasks like coding and summarizing in eight languages but is text-only. With a context window of 128,000 tokens, Meta also released smaller models, Llama 3.1 8B and 70B.

Meta unveils Llama 3.1, the most advanced open-source AI model to date

Meta has introduced its largest open-source AI model to date, Llama 3.1 405B, which contains 405 billion parameters. This model is not the largest ever but is the biggest in recent years and is competitive with leading proprietary models like OpenAI’s GPT-4o and Anthropic’s Claude 3.5 Sonnet. Trained using 16,000 Nvidia H100 GPUs, it benefits from advanced training techniques and is available for download or use on cloud platforms like AWS, Azure, and Google Cloud. It is also being used in WhatsApp and Meta.ai to power chatbots for U.S.-based users.

Key Features and Capabilities

Llama 3.1 405B can perform various tasks such as coding, answering math questions, and summarizing documents in eight languages. However, it is text-only and cannot handle image-based queries. Meta is also working on multimodal Llama models that can recognize images, videos, and generate speech, but these are not yet publicly available.

The model was trained using a dataset of 15 trillion tokens, equivalent to 750 billion words. Meta refined its data curation and quality assurance processes for this model. Synthetic data, generated by other AI models, was also used to fine-tune Llama 3.1 405B. However, Meta has not disclosed the exact sources of its training data, citing competitive and legal reasons. Key Features and Capabilities llama 3.1

Context Window and Tools

Llama 3.1 405B has a larger context window of 128,000 tokens, allowing it to summarize longer texts and maintain context in conversations better than previous models. Meta also released two smaller models, Llama 3.1 8B and Llama 3.1 70B, which share the same context window. These models can use third-party tools and APIs for tasks like answering questions about recent events, solving math problems, and validating code.

Performance and Licensing

Llama 3.1 405B performs comparably to OpenAI’s GPT-4 and shows mixed results against GPT-4o and Claude 3.5 Sonnet. It excels in executing code and generating plots but is weaker in multilingual capabilities and general reasoning. Due to its size, it requires substantial hardware to run. Meta is promoting its smaller models for general-purpose applications and sees Llama 3.1 405B as suitable for model distillation and generating synthetic data.

Llama 3.1, performance

Meta has updated Llama’s license to allow developers to use outputs from the Llama 3.1 model family to develop third-party AI models. However, developers with apps exceeding 700 million monthly users must request a special license from Meta.

Llama 3.1 performance

Getting Started

The models available to the community for download on llama.meta.com and Hugging Face and available for immediate development on our broad ecosystem of partner platforms.

Q&A

Have more questions on this topic? Ask our AI assistant for in-depth insights.

Read more from sources 👇

The Only Digital Marketing Feed You'll Ever Need.

Stay informed your way. Tailored updates when and how you want them. 100% Free.

10,000+ Users

500+ Sources

1000+ Tools

Or

Related Posts

Top-Notch Google Analytics Audit Tool

Top-Notch Google Analytics Audit Tool

Sponsored
GA4 Auditor
GA4 Auditor

Verified Sponsor

Verified Sponsor

GA4 Auditor is a Verified Sponsor. Want to get featured here? Contact us.

Verified Sponsor
Meta Unveils Llama 3.2 with Vision and Edge AI for Open-Source Models

Meta Unveils Llama 3.2 with Vision and Edge AI for Open-Source Models

Meta
Meta

Official Source

Official Source

Meta is a Official Source. The source has been verified by Swipe Insight team.

Official Source
Meta AI Now Lets You Talk and Share Photos for Faster Answers Trending ️‍🔥

Meta AI Now Lets You Talk and Share Photos for Faster Answers

Meta
Meta

Official Source

Official Source

Meta is a Official Source. The source has been verified by Swipe Insight team.

Official Source
Meta Launches New Developer Resources with Revamped Hub and Centers

Meta Launches New Developer Resources with Revamped Hub and Centers

Meta for Developers
Meta for Developers

Official Source

Official Source

Meta for Developers is a Official Source. The source has been verified by Swipe Insight team.

Official Source
Meta, Snap, TikTok Unite to Halt Spread of Harmful Content with Thrive

Meta, Snap, TikTok Unite to Halt Spread of Harmful Content with Thrive

Meta
Meta

Official Source

Official Source

Meta is a Official Source. The source has been verified by Swipe Insight team.

Official Source
Meta Rolls Out Secure Third-Party Chat Interoperability in Europe

Meta Rolls Out Secure Third-Party Chat Interoperability in Europe

Meta
Meta

Official Source

Official Source

Meta is a Official Source. The source has been verified by Swipe Insight team.

Official Source
Meta Announces New Data Access Renewal Process for Developers

Meta Announces New Data Access Renewal Process for Developers

Meta for Developers
Meta for Developers

Official Source

Official Source

Meta for Developers is a Official Source. The source has been verified by Swipe Insight team.

Official Source
Meta Launches Llama 3.1 Impact Grants Offering $2M for AI Projects

Meta Launches Llama 3.1 Impact Grants Offering $2M for AI Projects

Meta
Meta

Official Source

Official Source

Meta is a Official Source. The source has been verified by Swipe Insight team.

Official Source

Related Tools

GA4 Auditor logo

GA4 Auditor

Verified Tool

Verified Tool

GA4 Auditor is a Verified Tool. Want to get this badge? Contact us.

Verified Tool

Automated GA4 audits with actionable insights

Get Featured Here

Showcase your tool in this list.

Contact Us
Dash Hudson logo

Dash Hudson

Manage social media with insights and workflow tools

Organic Social