ai/llama3.2 repository overview

⁠Llama 3.2 Instruct

Llama 3.2 introduced lightweight 1B and 3B models at bfloat16 (BF16) precision, later adding quantized versions. The quantized models are significantly faster, with a much lower memory footprint and reduced power consumption, while maintaining nearly the same accuracy as their BF16 counterparts.

⁠Intended uses

Llama 3.2 instruct models are designed for:

• AI assistance on edge devices, Running chatbots and virtual assistants with minimal latency on low-power * hardware.
• Code assistance , Writing, debugging, and optimizing code on mobile or embedded systems.
• Content generation ,Drafting emails, summaries, and creative content on lightweight devices.
• Low-power AI for smart gadgets, Enhancing voice assistants on wearables and IoT devices.
• Edge-based data processing, Summarizing and analyzing data locally for security and efficiency.

⁠Characteristics

⁠Available model variants

¹: VRAM estimated based on model characteristics.

⁠Use this AI model with Docker Model Runner

First, pull the model:

docker model pull ai/llama3.2

Then run the model:

docker model run ai/llama3.2

For more information on Docker Model Runner, explore the documentation⁠.

⁠Benchmark performance

⁠Links

• Llama⁠