Post
1141
SLM Roundups, a weekly post where I summarize everything thats happened in the world of SLMs (or a majority of it)
Glint-Research/blog
Glint-Research/blog
- 1 reply
Lane Fiedler
CompactAI
AI & ML interests
None yet
Recent Activity
new activity about 20 hours ago
Compactbot/slm-parameter-audit:ping for @Compactbot new activity about 23 hours ago
musafa901/Minimax_H3-Age_Slider:๐ฉ Report: Illegal or restricted content reacted to theirpost with โ about 24 hours ago
SLM Roundups, a weekly post where I summarize everything thats happened in the world of SLMs (or a majority of it)
https://huggingface.co/spaces/Glint-Research/blogOrganizations
reacted to their post with โ about 24 hours ago
posted an update about 24 hours ago
Post
1141
SLM Roundups, a weekly post where I summarize everything thats happened in the world of SLMs (or a majority of it)
Glint-Research/blog
Glint-Research/blog
- 1 reply
reacted to GGUFGuy's post with ๐คฏ about 24 hours ago
Post
5308
wait why can i post
- 23 replies
reacted to Enderchef's post with ๐ about 2 months ago
Post
2380
I've left Glint Research.
After a long time in Glint Research, an entire distributed training grid built free for them, and more, I've decided that I no longer want to be affiliated with Glint Research.
More updates will follow. Comments/questions are welcome.
While you're reading this, follow these orgs! Following takes just a few seconds, and can change someone's day.
AxiomicLabs
fromziro
SupraLabs
After a long time in Glint Research, an entire distributed training grid built free for them, and more, I've decided that I no longer want to be affiliated with Glint Research.
More updates will follow. Comments/questions are welcome.
While you're reading this, follow these orgs! Following takes just a few seconds, and can change someone's day.
- 2 replies
reacted to Crownelius's post with ๐ฅ 4 months ago
Post
5106
Howdy,
CompactAI-O is launching a tiny Model Golf, and the winner walks away with $50 in RunPod credits. Monthly. Every month. Show up, build, somebody wins.
What it is
Build the best language model you can under 100 million parameters, with at least a 1028-token context window. That's it. Any architecture, any tokenizer, any training scheme you can dream up at 3am. The only catch is it's gotta be open source (MIT, GPL, Apache, AGPL) take your pick.
It scratches the same itch as a Kaggle comp without the dataset\leaderboard nonsense. No fixed benchmark to game. No llama.cpp compatibility hoops. If you wanna train a 50M-param MoE with five experts and a tokenizer built on cookbooks, you can do that. Nothing stopping you.
The rules are listed in the discord and on the organization page if you're interested.
Why $50????
It's symbolic. It ain't gonna make anyone rich. But it's enough to cover a weekend of GPU time, enough to keep enthusiasts coming back, and not so much that it pulls in people who are just there for the money. Enthusiasts build interesting things. Interesting things move the field forward. A little incentive. I'd do it for $50 lol.
How to join
First round opens soon. Landing page is here:
โ https://huggingface.co/spaces/CompactAI-O/Tiny-model-golf
For questions or to swap ideas, the Discord's open:
โ https://discord.gg/y2jTct6Cxv
Excited to see what yall come up with. โฅ
โ Shane
CompactAI-O is launching a tiny Model Golf, and the winner walks away with $50 in RunPod credits. Monthly. Every month. Show up, build, somebody wins.
What it is
Build the best language model you can under 100 million parameters, with at least a 1028-token context window. That's it. Any architecture, any tokenizer, any training scheme you can dream up at 3am. The only catch is it's gotta be open source (MIT, GPL, Apache, AGPL) take your pick.
It scratches the same itch as a Kaggle comp without the dataset\leaderboard nonsense. No fixed benchmark to game. No llama.cpp compatibility hoops. If you wanna train a 50M-param MoE with five experts and a tokenizer built on cookbooks, you can do that. Nothing stopping you.
The rules are listed in the discord and on the organization page if you're interested.
Why $50????
It's symbolic. It ain't gonna make anyone rich. But it's enough to cover a weekend of GPU time, enough to keep enthusiasts coming back, and not so much that it pulls in people who are just there for the money. Enthusiasts build interesting things. Interesting things move the field forward. A little incentive. I'd do it for $50 lol.
How to join
First round opens soon. Landing page is here:
โ https://huggingface.co/spaces/CompactAI-O/Tiny-model-golf
For questions or to swap ideas, the Discord's open:
โ https://discord.gg/y2jTct6Cxv
Excited to see what yall come up with. โฅ
โ Shane
- 9 replies
reacted to BlueNipples's post with ๐ 4 months ago
Post
3001
Good news, llama.cpp seems to be close to supporting MTP on qwen models. Bad news, every single gguf will have to be redone when it is.
- 1 reply
Def going to use this somewhere๐ฅ
replied to BlueNipples's post 4 months ago
Seems like they could find a way to add backwards compatibility
replied to Crownelius's post 4 months ago
40k?
Easy, spend 15k in Anthropic highest subscription & the rest in API
The rest? Spend it on a H100 (27k), then dump the rest into hardware
reacted to Crownelius's post with ๐ 4 months ago
Post
4335
Day 4-6 [05/05/2026]
Howdy,
Is anybody else willing to put a second mortgage on their house, just to spend 40k USD in compute credits? Just me? k...
I got dreams, man. The datasets I could build with 40k would be insane.
Somebody called me a genius the other day, they'd be shocked to find out, that I would put my house on the line for 30 days of runpod usage.
What would you do with it?
I would turn arxiv into a dataset. Turn each arxiv paper into a QnA.
Or... maybe if I got 40k USD in credit's Id end up like those 16 lost scientists.
Food for thought.
Anyways, I think I'm going to make a post once a week.
In the meantime you can find me building small llm's in discord here:
https://discord.gg/4DdwS9D8x9
edit:
So to be clear, I will not actually do this. But think of it this way. If you could pivot an entire industry with 10-30k would you do it?
Howdy,
Is anybody else willing to put a second mortgage on their house, just to spend 40k USD in compute credits? Just me? k...
I got dreams, man. The datasets I could build with 40k would be insane.
Somebody called me a genius the other day, they'd be shocked to find out, that I would put my house on the line for 30 days of runpod usage.
What would you do with it?
I would turn arxiv into a dataset. Turn each arxiv paper into a QnA.
Or... maybe if I got 40k USD in credit's Id end up like those 16 lost scientists.
Food for thought.
Anyways, I think I'm going to make a post once a week.
In the meantime you can find me building small llm's in discord here:
https://discord.gg/4DdwS9D8x9
edit:
So to be clear, I will not actually do this. But think of it this way. If you could pivot an entire industry with 10-30k would you do it?
- 7 replies
replied to Crownelius's post 5 months ago
2.3B tokens!?
reacted to Crownelius's post with ๐ฅ 5 months ago
Post
3846
[DAY ONE] PROJECT CROWFEATHER 4/30/2026
...The day I forgot to attach wandb.ai
Just dropped Crowfeather-50m, the first checkpoint in a series, and yeah, no graphs.
Crowfeather/Crowfeather-50m
54.5M params. Pretrain only. 17,500 steps banked on FineWeb-edu before Thunder credits ran dry. About 2.3B tokens, no SFT yet.
Architecture: Gemma-4 alternating sliding/global attention (1024 window, last layer always global) plus DeepSeek-V4 Muon optimizer plus WSD scheduler plus Gemma-2 logit soft-cap plus PaLM z-loss. Recipe in the model card.
What it can do: writes grammatical English. Knows that France has Rhine-adjacent monasteries (it picked Rouen instead of Paris but the vocabulary is in there). Tells stories about Mr. Fabien.
What it can't do yet: facts, code, math. Base LM, no SFT, no instruction tuning.
The series:
Every additional training run becomes another model card here
Every model card gets a matching post on this profile
Continuation goes to Colab next, picking up from step 17500 out of 100k
Limited to one post a day on Hugging Face, so updates will trickle out at that pace. Follow [@Crownelius](@Crownelius ) and [@Crowfeather](
Crowfeather ) if you want to watch this thing learn in public. Next drop will either come with the finished pre-train or whatever step I land on before the bank takes my credit card away.
Graphs will be available on my NEXT model lol
-Shane
...The day I forgot to attach wandb.ai
Just dropped Crowfeather-50m, the first checkpoint in a series, and yeah, no graphs.
Crowfeather/Crowfeather-50m
54.5M params. Pretrain only. 17,500 steps banked on FineWeb-edu before Thunder credits ran dry. About 2.3B tokens, no SFT yet.
Architecture: Gemma-4 alternating sliding/global attention (1024 window, last layer always global) plus DeepSeek-V4 Muon optimizer plus WSD scheduler plus Gemma-2 logit soft-cap plus PaLM z-loss. Recipe in the model card.
What it can do: writes grammatical English. Knows that France has Rhine-adjacent monasteries (it picked Rouen instead of Paris but the vocabulary is in there). Tells stories about Mr. Fabien.
What it can't do yet: facts, code, math. Base LM, no SFT, no instruction tuning.
The series:
Every additional training run becomes another model card here
Every model card gets a matching post on this profile
Continuation goes to Colab next, picking up from step 17500 out of 100k
Limited to one post a day on Hugging Face, so updates will trickle out at that pace. Follow [@Crownelius](@Crownelius ) and [@Crowfeather](
Graphs will be available on my NEXT model lol
-Shane
- 3 replies
reacted to Crownelius's post with ๐ฅ 5 months ago
Post
5978
My Huggingface journey has been a trip!
I wanted to take the time to thank each and every one of you for using my dataset and getting it to go as far as it did. Believe it or not, some neanderthal was and maybe still is trending on huggingface.
Not only did my dataset reach number one, my fine-tuned qwen3.5 model did as well. Top 10. Honestly, ain't much left to do here.
Y'all have given me the desire, no... the craving for more. I am absolutely obsessed with AI now. I want to tweak it... I want to take it apart, just to see what makes everything tick. I want to put it together like Frankenstein and his monster.
The only thing that's stopping this guy is compute. I don't mind spending every penny I have on this. I desperately want to drive AI forward, even just a little bit.
I never knew the clanker hater from a year ago would be saying this.
Thank you all from the bottom of my heart.
Looking forward to showing you what I'm cooking up next. @CompactAI is your only hint!
I wanted to take the time to thank each and every one of you for using my dataset and getting it to go as far as it did. Believe it or not, some neanderthal was and maybe still is trending on huggingface.
Not only did my dataset reach number one, my fine-tuned qwen3.5 model did as well. Top 10. Honestly, ain't much left to do here.
Y'all have given me the desire, no... the craving for more. I am absolutely obsessed with AI now. I want to tweak it... I want to take it apart, just to see what makes everything tick. I want to put it together like Frankenstein and his monster.
The only thing that's stopping this guy is compute. I don't mind spending every penny I have on this. I desperately want to drive AI forward, even just a little bit.
I never knew the clanker hater from a year ago would be saying this.
Thank you all from the bottom of my heart.
Looking forward to showing you what I'm cooking up next. @CompactAI is your only hint!
- 3 replies
replied to Crownelius's post 5 months ago
Nice ending :)
Not sure I know how to do it in CPU, I just know CUDA. :/
They may be able to help in some way.
reacted to danielhanchen's post with ๐ 8 months ago
Post
3604
You can now run Kimi K2.5 locally! ๐ฅ
We shrank the 1T model to 240GB (-60%) via Dynamic 1-bit.
Get >40 tok/s on 242GB or 622GB VRAM/RAM for near full precision.
GGUF: unsloth/Kimi-K2.5-GGUF
Guide: https://unsloth.ai/docs/models/kimi-k2.5
We shrank the 1T model to 240GB (-60%) via Dynamic 1-bit.
Get >40 tok/s on 242GB or 622GB VRAM/RAM for near full precision.
GGUF: unsloth/Kimi-K2.5-GGUF
Guide: https://unsloth.ai/docs/models/kimi-k2.5
- 7 replies