How to use p1atdev/multi-tokenizers-processor-sample with Transformers:
# Load model directly from transformers import AutoModel model = AutoModel.from_pretrained("p1atdev/multi-tokenizers-processor-sample", dtype="auto")
