Instructions to use PygmalionAI/Pygmalion-3-12B with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use PygmalionAI/Pygmalion-3-12B with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="PygmalionAI/Pygmalion-3-12B")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("PygmalionAI/Pygmalion-3-12B")
model = AutoModelForCausalLM.from_pretrained("PygmalionAI/Pygmalion-3-12B")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Inference
Notebooks
Google Colab
Kaggle
Local Apps

vLLM

How to use PygmalionAI/Pygmalion-3-12B with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "PygmalionAI/Pygmalion-3-12B"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "PygmalionAI/Pygmalion-3-12B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/PygmalionAI/Pygmalion-3-12B

SGLang

How to use PygmalionAI/Pygmalion-3-12B with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "PygmalionAI/Pygmalion-3-12B" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "PygmalionAI/Pygmalion-3-12B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "PygmalionAI/Pygmalion-3-12B" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "PygmalionAI/Pygmalion-3-12B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use PygmalionAI/Pygmalion-3-12B with Docker Model Runner:
```
docker model run hf.co/PygmalionAI/Pygmalion-3-12B
```

Feedback

by Leaf45 - opened Mar 24, 2025

Discussion

Leaf45

Mar 24, 2025

•

edited Mar 24, 2025

The model its responses are a lot more human than most of the other models. It doesn't ramble and generally like the responses. Most other models seem to share a lot of training data which this one doesn't have seemingly. It feels like every single other model often say "and hey", "but yeah", and "and yeah".

When it comes to the short comings the responses can sometimes be a bit too short. I write a paragraph for example and the model might respond with one sentence. But the thing that this model tends to do what I haven't experienced with other models is that it sometimes responds with a message it has send before. I'm in the middle of a conversation with it and then it suddenly repeats a past message it had send me about 6 messages ago. It's as if it gets confused on what it's responding to. Just now i removed the example dialogue from the CFG scale and it might have confused the CFG scale with the examples of dialogue department and the present dialogue. It happened again even when having removed the CFG dialogue examples.

But thanks for the model Pygmalion. I hope the next iteration is even better!

I remember trying Eleusis but I like the text responses from Pygmalion 3 better.

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment