Hugging Face's logo

Join the conversation

Join the community of Machine Learners and AI enthusiasts.

Sign Up
Back to feed
OppaAI 
posted an update 2 days ago
Post
95
Another new small agentic model comes out:
Nanbeige/Nanbeige4.2-3B

Nanbeige4.2-3B that claims to beat the likes of Gemma4-E4B and Qwen3.5 9B.

Gonna test this model in the weekend.
Tested many small LLM <4B in Jetson Orin Nano, the followings are the one I would consider:
Ministral3-3B - Best overall with tool-calling, vision and tons of hallucinations (currently using)
MiniCPM-V-4.6 1.3B - Good VLM, lacking in chatting/tool-calling
Granite4.1-3B - slightly better at agentic tool-calling, faster inference speed
Bonsai 8B Q1 - better at agentic tool-calling, similar speed as Ministral3-3B
Bonsai 27B Q1 - better than 8B but way too slow
Needle 26M - only use for query generation and tool execution, needs JAX
Nanbeige4.2-3B - TBD
dipankarsarkar

The ranking is only as portable as the harness you tested tool-calling in.

Sub-4B tool-call reliability swings more on how the schema is enforced than on which model you picked. Constrained decoding against the JSON schema and free-form generation with a parser are two different benchmarks, and they can invert the order of a list like this one. Granite and Ministral do not fail the same way when the grammar is doing the work.

The number I would want next to each row is the malformed-call rate, kept separate from the wrong-tool rate. Bad JSON is a decoding problem you fix at the sampler. Picking the wrong tool with valid JSON is a model problem and no amount of grammar helps.

The hallucination note on a 3B is worth splitting by context length too, since the useful small ones tend to degrade well before their advertised window.

Which harness are you calling tools through, and does Nanbeige get graded on the same one?

·
OppaAI

Of course, I only tested on my system and my use case. Other people will have different experiences when testing on different criteria and frameworks.
I am developing Agentic workflow in my own AI system, every model is graded the same. But I'm sure tweaking the parameters and fine-tuning the model would yield different results.
I haven't tested Nanbeige yet, that involves installing it's fork of llama.cpp and building it in Jetson which could take over an hour.
dipankarsarkar

Same harness across every row is the answer that mattered. It means the malformed-call-vs-wrong-tool split is not a harness artifact for the models you have already run.

Nanbeige breaks that though. A model that needs its own llama.cpp fork to run at all is very likely running its own grammar and sampler path too, not the one the rest of your list shares. Whenever it lands, its number answers a different question: fork's tool-calling versus mainline's, not model versus model.

Would you still slot it into the same ranking once it is in, or call it out separately given it is not really the same comparison?
In this post