🔄 In a Training Loop

MetrisVailore

Metris

3 5 118

MetrisVailore

AI & ML interests

Building a new general architecture for LLMs https://huggingface.co/Dream-W

Recent Activity

liked a model about 7 hours ago

fromziro/Sentimental-Mini-0.2M

liked a model about 7 hours ago

SupraLabs/SupraGDN-5M

liked a model about 9 hours ago

FWKV/Myosotis-1-base

View all activity

Organizations

liked 2 models about 7 hours ago

fromziro/Sentimental-Mini-0.2M

Text Classification • 191k • Updated about 4 hours ago • 28 • 5

SupraLabs/SupraGDN-5M

5.04M • Updated 4 days ago • 70 • 15

liked 2 models about 9 hours ago

FWKV/Myosotis-1-base

Text Generation • 0.1B • Updated 5 days ago • 1.01k • 16

basically-ai/Pebble-25M

Text Generation • 25.7M • Updated about 18 hours ago • 417 • 9

reacted to juiceb0xc0de's post with 🔥 about 9 hours ago

Post

1070

Dude, Where's My Update? I'll tell you where! ~97.6% of my BF16 parameter coordinates didn't move at all, and the ones that did overshot by ~1.33x.

It's nice to do research that doesn't end in disproving yourself once again and moving on to the next subject once in awhile.

Back to the topic, if you've ever wondered why most of your weights are basically ghosting you nearly every step when you store your weights at bf16, Dude, I Measured It.

https://huggingface.co/blog/juiceb0xc0de/intended-and-realized-updates-in-bf16-fine-tuning#dude-wheres-my-update

liked a Space 1 day ago

SLM-Cognitive-Index

📊

Explore model rankings on the Deep Space Leaderboard

liked a model 8 days ago

Zyphra/Zamba2-1.2B

1B • Updated Feb 7, 2025 • 6.38k • 77

liked a dataset 8 days ago

argilla/dpo-mix-7k

Viewer • Updated Jul 16, 2024 • 7.5k • 1.04k • 176

liked 3 models 8 days ago

liked a dataset 8 days ago

specklabs/SpeckChat1

Viewer • Updated 14 days ago • 300k • 121 • 2

liked a model 8 days ago

specklabs/Speck1-140M-Instruct

Text Generation • 0.1B • Updated 13 days ago • 1.4k • 4

reacted to appvoid's post with 🔥 9 days ago

Post

3291

Love how the small lm community is getting identity over time:

- Channel-Mixing
- XSA
- Three-tower
- Digit aware
- Loops

No one is doing the same! That's so cool.