Bc-AI/nova-dump
0 Bytes
Boning Cui
Bc-AI
AI & ML interests
I like LLM's and VLM's.
Recent Activity
updated a model about 5 hours ago
Smilyai-labs/Nova-1-standard-pretrained-preview published a model about 5 hours ago
Smilyai-labs/Nova-1-standard-pretrained-preview repliedto their post about 7 hours ago
# š„ Nova-1 Beta: Test Our New LLMs!
**Smilyai Labs** is building **Nova-1** ā open-source LLMs with novel architectures. Join our beta program!
## šÆ Available Now:
**Nova-1-Standard (1.2B)** ā Phase 2 of pretraining in progress
- PPL 13.5 (beats GPT-2 Large!)
- 48K tok/s on consumer GPUs
- Great for code, reasoning, edge deployment
**Nova-1-Large (3.5B)** ā Training live RIGHT NOW
- Current: 30.9 PPL, improving fast, loss at 3.5 right now
- Will finish with ~1.7B tokens today
- Better reasoning & longer context
**Nova-1-XL (10B MoE)** ā Coming soon (We dont know yet! haha)
- Final Specs not decided yet
## What Makes Nova Special?
āØ **Mixture of Depths (MoD)** ā Routes tokens dynamically, 30% faster
āØ **Grouped Query Attention** ā Efficient like LLaMA 2/3
āØ **Phased Training** ā Fresh 1B tokens each phase (no overfitting!)
āØ **RoPE** ā Context extendable to 8K+
## š¤ Join Beta Testing:
š **[Smilyai-labs-beta-testers](https://huggingface.co/Smilyai-labs-beta-testers)**
Get early access, shape the roadmap, and help build transparent open-source AI!
Organizations
Posts 1
Post
101
# š„ Nova-1 Beta: Test Our New LLMs!
**Smilyai Labs** is building **Nova-1** ā open-source LLMs with novel architectures. Join our beta program!
## šÆ Available Now:
**Nova-1-Standard (1.2B)** ā Phase 2 of pretraining in progress
- PPL 13.5 (beats GPT-2 Large!)
- 48K tok/s on consumer GPUs
- Great for code, reasoning, edge deployment
**Nova-1-Large (3.5B)** ā Training live RIGHT NOW
- Current: 30.9 PPL, improving fast, loss at 3.5 right now
- Will finish with ~1.7B tokens today
- Better reasoning & longer context
**Nova-1-XL (10B MoE)** ā Coming soon (We dont know yet! haha)
- Final Specs not decided yet
## What Makes Nova Special?
āØ **Mixture of Depths (MoD)** ā Routes tokens dynamically, 30% faster
āØ **Grouped Query Attention** ā Efficient like LLaMA 2/3
āØ **Phased Training** ā Fresh 1B tokens each phase (no overfitting!)
āØ **RoPE** ā Context extendable to 8K+
## š¤ Join Beta Testing:
š **[Smilyai-labs-beta-testers]( Smilyai-labs-beta-testers
Get early access, shape the roadmap, and help build transparent open-source AI!
**Smilyai Labs** is building **Nova-1** ā open-source LLMs with novel architectures. Join our beta program!
## šÆ Available Now:
**Nova-1-Standard (1.2B)** ā Phase 2 of pretraining in progress
- PPL 13.5 (beats GPT-2 Large!)
- 48K tok/s on consumer GPUs
- Great for code, reasoning, edge deployment
**Nova-1-Large (3.5B)** ā Training live RIGHT NOW
- Current: 30.9 PPL, improving fast, loss at 3.5 right now
- Will finish with ~1.7B tokens today
- Better reasoning & longer context
**Nova-1-XL (10B MoE)** ā Coming soon (We dont know yet! haha)
- Final Specs not decided yet
## What Makes Nova Special?
āØ **Mixture of Depths (MoD)** ā Routes tokens dynamically, 30% faster
āØ **Grouped Query Attention** ā Efficient like LLaMA 2/3
āØ **Phased Training** ā Fresh 1B tokens each phase (no overfitting!)
āØ **RoPE** ā Context extendable to 8K+
## š¤ Join Beta Testing:
š **[Smilyai-labs-beta-testers](
Get early access, shape the roadmap, and help build transparent open-source AI!