QtMeshEditor — Vegetation Part Segmentation
A point-cloud part-segmentation network (PointNet++-style) that labels each
point of a tree / plant mesh as trunk, branch, foliage, root, or
flower (fruit), exported to ONNX for local inference via ONNX Runtime.
One of the category-specialised segmentation models built for
QtMeshEditor (epic #818,
Track B2) — a free, open-source 3D mesh & animation editor. The app
auto-detects the mesh category with a companion
point-cloud classifier
and dispatches to this model for vegetation; the
body model
covers characters, with
vehicle
and
building
siblings. The aggregate download source used by the app is
QtMeshEditor-models
(segment/meshseg_vegetation.onnx).
Model
- Input: a sampled point cloud
float32 [1, N, 3](normalised to a centred unit box; +Y up). - Output: per-point class logits over 6 channels
(
unknown, trunk, branch, foliage, root, flower); argmax → label, scattered back to mesh vertices/faces by nearest sampled point. - Architecture: shared per-point MLP + two kNN local-aggregation blocks
(in-graph
cdist+topk, ONNX-exportable) + a global max-pooled feature; ~0.78 MB. Trained at the app's inference sample size (4096 points).
Training data & license
Trained from scratch, 100% on procedurally generated synthetic trees we own (no third-party data at all): parametric broadleaf / pine / palm / dead-tree / bush regimes with surface-sampled capsule trunks and branches, canopy-vs-per-tip foliage blobs, surface roots, and flower/fruit clusters — labels are exact by construction. Weights released under CC-BY-4.0; please credit QtMeshEditor.
Evaluation
- Held-out synthetic validation accuracy: 93.3% (v1.1 — a harder big-canopy / drooping-oak distribution; v1.0 scored 93.8% on the narrower set), per-point, unknown masked. Real-world CC0 vegetation packs are the planned next data slice (mined via material/submesh-name labels).
- Real-world check (stylized oak FBX, huge low canopy + stubby trunk): v1.0 mislabelled ~63k canopy verts as trunk; v1.1 cut that ~6× (63k→10k trunk verts, foliage 63%→93% of the mesh); v1.2 halves the residual root over-prediction (root 1980→845 verts, canopy root-scatter ~gone). Root/trunk on real meshes is the weakest remaining class — the durable fix is the planned CC0 real-vegetation data slice.
Reproducing
scripts/export-meshseg-onnx.py --category vegetation in the QtMeshEditor
repo (one-time, offline; the app never runs Python). Strategy + roadmap:
docs/MESH_SEGMENTATION_STRATEGY.md.
Versions
- v1.2.0 (current) — root/trunk robustness: roots now appear in only ~20% of trees (most real tree meshes model no roots — they're underground), and when present are thick, buttress-like, and strictly ground-hugging (not twig-thin); a new trunk base FLARE is labelled trunk so the model stops reading a widening base as root. Real oak: root over-prediction 1980→845 verts, scattered canopy root-flecks essentially gone.
- v1.1.0 — big-canopy robustness: added an
oakregime (stubby trunk, canopy 1.1–2.2× larger, foliage drooping down around/below the trunk top with a low skirt) and solid-VOLUME canopy fill (real leaf-card canopies are dense volumes, not the hollow shells v1.0 trained on). Fixes the verified real-world failure where a stylized oak's low canopy was labelled trunk. - v1.0.0 — initial synthetic-only release (#818 Track B2).