Development · Articlepublished 51d
Transformers adds packed GGUF inference on Apple Silicon with ggml kernels
Hugging Face has added packed GGUF inference to Transformers main for Apple Silicon, allowing selected quantised Qwen3.5 and compatible Qwen3.8 checkpoints to stay compressed on Metal while reusing ggml kernels through its kernels library. The same checkpoints can be served behind an OpenAI-compatible endpoint, but the packed path remains MPS-only, architecture-limited and pending a stable Transformers release.