How to use papluca/xlm-roberta-base-language-detection with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-classification", model="papluca/xlm-roberta-base-language-detection")
# Load model directly from transformers import AutoTokenizer, AutoModelForSequenceClassification tokenizer = AutoTokenizer.from_pretrained("papluca/xlm-roberta-base-language-detection") model = AutoModelForSequenceClassification.from_pretrained("papluca/xlm-roberta-base-language-detection", device_map="auto")
https://huggingface.co/papluca/xlm-roberta-base-language-detection/resolve/714fcf8b61e6a080f733bb238687d31488dec634/tokenizer.json
hf download hf://papluca/xlm-roberta-base-language-detection@714fcf8b61e6a080f733bb238687d31488dec634/tokenizer.json
curl -L -o tokenizer.json https://huggingface.co/papluca/xlm-roberta-base-language-detection/resolve/714fcf8b61e6a080f733bb238687d31488dec634/tokenizer.json
