Meta Llama 3.3 70B Howto

Get Access: Fill out the access request form on HuggingFace to get access to the gated repository for Llama 3.3 70B. Generate a HuggingFace READ token which is free to create.

Install Dependencies: Install the required dependencies including transformers library and PyTorch

Load the Model: Import and load the model using the following code: import transformers import torch model_id = 'meta-llama/Llama-3.3-70B-Instruct' pipeline = transformers.pipeline('text-generation', model=model_id, model_kwargs={'torch_dtype': torch.bfloat16}, device_map='auto')

Format Input Messages: Structure your input messages as a list of dictionaries with 'role' and 'content' keys. For example: messages = [ {'role': 'system', 'content': 'You are a helpful assistant'}, {'role': 'user', 'content': 'Your question here'} ]

Generate Output: Generate text by passing messages to the pipeline: outputs = pipeline(messages, max_new_tokens=256) print(outputs[0]['generated_text'])

Hardware Requirements: Ensure you have adequate GPU memory. The model requires significantly less computational resources compared to Llama 3.1 405B while delivering similar performance.

Follow Usage Policy: Comply with Meta's Acceptable Use Policy available at https://www.llama.com/llama3_3/use-policy and ensure usage adheres to applicable laws and regulations