How to use QuantLLM/functiongemma-270m-it-4bit-mlx with MLX:
# Download the model from the Hub pip install huggingface_hub[hf_xet] huggingface-cli download --local-dir functiongemma-270m-it-4bit-mlx QuantLLM/functiongemma-270m-it-4bit-mlx
How to use QuantLLM/functiongemma-270m-it-4bit-mlx with Transformers:
# Load model directly from transformers import AutoModel model = AutoModel.from_pretrained("QuantLLM/functiongemma-270m-it-4bit-mlx", dtype="auto")
