Hugging Face · Transformers
Generate a chat reply
Format messages with the model's chat template, generate, and decode only the new tokens.
messages = [{"role": "user", "content": "Give me three tips for learning Python."}]
inputs = tokenizer.apply_chat_template(messages, add_generation_prompt=True, return_tensors="pt", return_dict=True).to(model.device)
outputs = model.generate(**inputs, max_new_tokens=200)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True))
Official Hugging Face documentation