Multilingual UnigramLM

company

https://cimeister.github.io/blog/unigramlm/

AI & ML interests

Multilingual Tokenization

Recent Activity

suchirsalhan updated a model about 3 hours ago

MultilingualUnigramLM/ft-langmap-qwen2_5-1_5b-fineweb100M-tur-original

suchirsalhan published a model about 3 hours ago

MultilingualUnigramLM/ft-langmap-qwen2_5-1_5b-fineweb100M-tur-original

suchirsalhan updated a model about 3 hours ago

MultilingualUnigramLM/ft-langmap-qwen2_5-1_5b-fineweb100M-hun-original

View all activity

MultilingualUnigramLM 's datasets 35

MultilingualUnigramLM/FineWeb2-500M-tur_Latn

Viewer • Updated about 4 hours ago • 419k

MultilingualUnigramLM/FineWeb2-500M-hun_Latn

Viewer • Updated about 4 hours ago • 312k

MultilingualUnigramLM/FineWeb2-500M-fin_Latn

Viewer • Updated about 4 hours ago • 387k • 1

MultilingualUnigramLM/FineWeb2-khm_Khmr-100M

Viewer • Updated 13 days ago • 1.01M • 13

MultilingualUnigramLM/FineWeb2-tha_Thai-100M

Viewer • Updated 13 days ago • 710k • 13

MultilingualUnigramLM/FineWeb2-tam_Taml-100M

Viewer • Updated 13 days ago • 264k • 12

MultilingualUnigramLM/FineWeb2-arb_Arab-100M

Viewer • Updated 13 days ago • 188k • 11

MultilingualUnigramLM/FineWeb2-ben_Beng-100M

Viewer • Updated 13 days ago • 254k • 11

MultilingualUnigramLM/FineWeb2-amh_Ethi-100M

Viewer • Updated 13 days ago • 197k • 13

MultilingualUnigramLM/FineWeb2-yor_Latn-100M

Viewer • Updated 13 days ago • 80k • 11

MultilingualUnigramLM/FineWeb2-wol_Latn-100M

Viewer • Updated 13 days ago • 6.62k • 14

MultilingualUnigramLM/FineWeb2-hin_Deva-100M

Viewer • Updated 13 days ago • 193k • 15

MultilingualUnigramLM/FineWeb2-swh_Latn-100M

Viewer • Updated 13 days ago • 240k • 13

MultilingualUnigramLM/FineWeb2-deu_Latn-100M

Viewer • Updated 13 days ago • 234k • 15

MultilingualUnigramLM/FineWeb2-tur_Latn-100M

Viewer • Updated 15 days ago • 263k • 55

MultilingualUnigramLM/FineWeb2-fin_Latn-100M

Viewer • Updated 15 days ago • 261k • 49

MultilingualUnigramLM/FineWeb2-hun_Latn-100M

Viewer • Updated 15 days ago • 198k • 49

MultilingualUnigramLM/LangMap-TheStack-go-100M

Viewer • Updated Apr 13 • 35k • 21

MultilingualUnigramLM/LangMap-TheStack-typescript-100M

Viewer • Updated Apr 13 • 64.2k • 22

MultilingualUnigramLM/LangMap-TheStack-markdown-100M

Viewer • Updated Apr 13 • 56.5k • 42

MultilingualUnigramLM/LangMap-TheStack-css-100M

Viewer • Updated Apr 13 • 18.6k • 22

MultilingualUnigramLM/LangMap-TheStack-html-100M

Viewer • Updated Apr 13 • 20.3k • 32

MultilingualUnigramLM/LangMap-TheStack-php-100M

Viewer • Updated Apr 13 • 78.3k • 34

MultilingualUnigramLM/LangMap-TheStack-csharp-100M

Viewer • Updated Apr 13 • 79.5k • 111 • 1

MultilingualUnigramLM/LangMap-TheStack-cpp-100M

Viewer • Updated Apr 13 • 26.4k • 14

MultilingualUnigramLM/LangMap-TheStack-c-100M

Viewer • Updated Apr 13 • 28.6k • 14

MultilingualUnigramLM/LangMap-TheStack-java-100M

Viewer • Updated Apr 13 • 73k • 64

MultilingualUnigramLM/LangMap-TheStack-javascript-100M

Viewer • Updated Apr 13 • 47.4k • 15

MultilingualUnigramLM/LangMap-TheStack-python-100M

Viewer • Updated Apr 13 • 49.9k • 56 • 1

MultilingualUnigramLM/FineWeb2-100M

Updated Apr 11 • 6