Instructions to use shibatch/tinydiffusiongemmamoe4m with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use shibatch/tinydiffusiongemmamoe4m with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="shibatch/tinydiffusiongemmamoe4m")

# Load model directly
from transformers import AutoModel
model = AutoModel.from_pretrained("shibatch/tinydiffusiongemmamoe4m", device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use shibatch/tinydiffusiongemmamoe4m with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "shibatch/tinydiffusiongemmamoe4m"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "shibatch/tinydiffusiongemmamoe4m",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/shibatch/tinydiffusiongemmamoe4m

SGLang

How to use shibatch/tinydiffusiongemmamoe4m with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "shibatch/tinydiffusiongemmamoe4m" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "shibatch/tinydiffusiongemmamoe4m",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "shibatch/tinydiffusiongemmamoe4m" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "shibatch/tinydiffusiongemmamoe4m",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use shibatch/tinydiffusiongemmamoe4m with Docker Model Runner:
```
docker model run hf.co/shibatch/tinydiffusiongemmamoe4m
```

tinydiffusiongemmamoe4m / eval_text_generation.json

shibatch

Upload folder using huggingface_hub

d657346 verified 4 days ago

Raw

History Blame Contribute Delete

1.3 kB

	{
	"normal": {
	"Once upon": "Once upon a time, there was a happy voice. He was very kindful and calm. He had made his friends who wrap them the time. One day, he saw a new geater in his bright flower. He thought a special game, so it could dry",
	"There was a little": "There was a little girl named Lily. She liked to play with her toys and spend all her things inls with her friends. One day, Lily's.",
	"One day": "One day, a big harage named Tim wanted to play near it. He saw a big mottle in the lard. He wanted to say sorry to his friend, \"Rrue, Tim. He ruffed you. He was very curious. He looked around his play"
	},
	"ema": {
	"Once upon": "Once upon a time, there was a little girl. She loved to play on the swings and play. One day, she decided to go.",
	"There was a little": "There was a little girl named Lily. She liked to play with her toys and spend all her toys in her room with her mommy. One day, she went to play with her toys and Lily saw a beautiful yellow berriy. She wanted to make it, but her mom said her it was right. Her",
	"One day": "One day, a little girl named Mia and Ben went walking in the park. They saw a big block and a big car. Lily wanted to toub it, but she just wanted it.\n\n\"Oh, Ben, what is that, doing?\" Ben asked.\n\n\"Well,"
	}
	}