FleXray: VictorButoi/flexray

• Website and in-browser demo: flexray.csail.mit.edu
• Code: github.com/VictorButoi/FleXray
• Data: VictorButoi/flexray-data
• Tutorial: Colab notebook
• Paper: FleXray: Universal Clinical X-ray Segmentation

FleXray is a single 2D UNet that segments anatomy from standard radiographs across body regions, projections, and acquisition settings. It predicts 60 anatomical structures (plus background) as independent sigmoid channels at 256 x 256 resolution.

This repository holds the flagship model and the four sibling models of the FleXray ensemble, one bundle per members/ subfolder (see Repository layout). flexify and FleXraySegmenter.from_pretrained load the flagship by default.

Quick start

python -m pip install flexray
flexify --input ./image.png --output-dir ./predictions

from fxr.inference import FleXraySegmenter

segmenter = FleXraySegmenter.from_pretrained("VictorButoi/flexray")
prediction = segmenter.predict("./image.png", threshold=0.5)
prediction.masks          # uint8, BxCxHxW thresholded masks
prediction.probabilities  # float32, BxCxHxW sigmoid probabilities
prediction.logits         # float32, BxCxHxW raw scores

flexify writes <name>_masks.npy, <name>_probabilities.npy, and <name>_logits.npy per image, each shaped CxHxW. The Python API keeps the batch dimension (B=1 for a single image). Channel order follows label_schema.json. Pass --binary LABEL (for example --binary femurs) to write one label. See docs/inference.md for the full CLI and Python API.

The FleXray ensemble

The flagship was trained with a 0.375 FluXray proportion in the training mix. Four sibling models share its architecture, label schema, preprocessing, and training recipe and differ only in that proportion:

Subfolder	FluXray proportion	Role
members/flux000	0.0	ensemble member
members/flux025	0.25	ensemble member
members/flux0375	0.375	flagship (loaded by default)
members/flux050	0.5	ensemble member
members/flux075	0.75	ensemble member

ensemble.json at the repository root lists the flagship and the members. Because the members share one output space, they are averaged in probability space:

flexify --ensemble --tta-samples 16 --input ./image.png --output-dir ./predictions
flexify --subfolder members/flux000 --input ./image.png --output-dir ./predictions

segmenter = FleXraySegmenter.from_pretrained("VictorButoi/flexray", ensemble=True)
prediction = segmenter.predict("./image.png", tta_samples=16)

member = FleXraySegmenter.from_pretrained(
    "VictorButoi/flexray", subfolder="members/flux000"
)

The website demo exposes the same choices as quality modes: Low runs the flagship once, Normal runs the flagship with 8-pass TTA, High runs the five-model ensemble once, and X-High runs the ensemble with 8-pass TTA. The members are also listed in MODEL_ZOO.md.

Test-time augmentation

The reported results use 16 passes per model (--tta-samples 16 or predict(..., tta_samples=16)). The browser demo uses 8 passes per model in Normal and X-High modes; its current settings are published in the demo manifest.

tta_samples=N runs one un-augmented pass plus N - 1 randomly augmented passes and averages their sigmoid probabilities, then converts that mean back to logits. The package and browser implement the released tta_v3 chain in this order:

Transform	Probability	Parameters
Horizontal flip	0.5	Exactly inverted on the prediction before averaging
Invert intensities	0.5	1 - image
CLAHE	0.1	Clip limit 1.0-2.0; 8 x 8 grid
Gamma	0.25	Gamma 0.9-1.1; gain 0.9-1.1; mutually exclusive with CLAHE
Contrast	0.25	Multiply intensities by 0.7-1.3 and clamp to [0, 1]
Sharpness	0.5	Factor 0.7-1.3
Gaussian noise	0.25	Standard deviation 0.01

The flip is the only geometric transform; intensity transforms are not inverted. The CLAHE/gamma branch leaves the image unchanged with probability 0.65. See the Python implementation and browser implementation.

Use predict(..., tta_samples=16, seed=42) to reproduce the Python augmentation draws without changing the global torch RNG. With no seed, draws use the global torch RNG. The browser uses its own random-number source, so matching augmentation settings do not imply identical random views.

With an ensemble, each view is drawn once and run through every member. M models and N passes therefore require M x N forward passes: 80 for the five-model ensemble at N=16, or 40 for the browser's X-High mode at N=8. tta_samples<=1 disables augmentation.

Input contract

preprocessing.json is applied automatically by the public loaders:

• grayscale input (RGB is converted), any 8-bit or 16-bit PNG / JPEG / TIFF / BMP
• per-image percentile min-max normalization to [0, 1] (0.5th / 99.5th percentiles, eps = 1e-8)
• zero-pad to a square, then resize to 256 x 256
• outputs are multilabel sigmoid probabilities; masks use threshold 0.5

Outputs are at the 256 x 256 model resolution; the CLI and Python API do not resample back to the original image size.

Output labels

FleXray outputs 60 foreground masks (61 channels including background). The broader dataset/evaluation protocol also recognizes aggregate lumbar_spine and thoracolumbar_spine annotations; these are evaluated by combining the relevant per-vertebra outputs and are not checkpoint channels. Channel order is stored in each bundle's label_schema.json.

• Skull / shoulder girdle: skull, scapulae, clavicles
• Upper limb: humeri, radii, ulnae, carpals, metacarpals, phalanges
• Lower limb: femurs, patellae, tibiae, fibulae, tarsals, metatarsals, toes
• Thorax: rib_1 - rib_12, sternum
• Spine: vertebra_c1 - c7, t1 - t12, l1 - l5, sacrum
• Pelvis: hips
• Soft tissue: lungs, heart, liver, spleen, kidneys

Paired structures are merged (for example femurs covers both sides); laterality is not predicted.

Architecture

fxr.models.UNet, 2D, 1 input channel, 61 output channels; filters [64, 128, 256, 512, 512, 720, 1024], 3 convolutions per block, residual blocks with instance norm, align_corners=True upsampling. The full architecture is in each bundle's config.yml.

Training data

The models were trained on three source types unified under the FleXray label protocol. For training mixture proportions, please refer to the paper:

• Real X-ray masks: HandBones, FootBones, MURA forearm, and MURA humerus, with our own annotations.
• Generated FluXray images: digitally reconstructed radiographs from the MOOSE CTs, generatively edited toward real X-ray appearance, with exact overlapping masks for every protocol structure.
• Online CT->DRR rendering: MOOSE / ENHANCE-PET 1.6k, Shoulder-CT, HaN-Seg, PedsCT, RSNA cervical-spine fracture CTs, and ElbowCT, rendered to DRRs at random poses during training with per-label attenuation jitter.

Training used AdamW (lr 3e-4, cosine schedule), a Dice + binary cross-entropy loss routed per dataset (partially labeled sources ignore unlabeled channels), and separate augmentation presets for CT-derived and X-ray inputs. The exact recipe is fxr/configs/training/base.yml in the code release.

Every dataset's license, redistribution status, and download pointer is documented in the VictorButoi/flexray-data card. That repository ships the real X-ray sources whose licenses permit redistribution as image/mask pairs with packaging manifests, the MURA masks, and the FluXray database.

Evaluation

FleXray was evaluated on eight real-radiograph datasets held out from training (DarwinCVD19, DeepFluoro, ElbowLat, HipRay, LowerLimbs, RAM-W600, PedsTorso, and VinDr-Rib), spanning lungs, ribs, peripheral bones, spine, and pelvis. Against supported generalist baselines (FluoroSAM, TotalSegmentator2D, PAXray), FleXray performs best or ties on all eight datasets, with significant improvements on seven and no statistically detectable difference from PAXray on VinDr-Rib. Per-dataset numbers and confidence intervals are in the paper; the benchmark figure is on the project website. Evaluation ignores ground-truth labels covering less than 0.1% of the image.

Intended use and limitations

Research use only. FleXray is not a medical device and is not cleared for clinical diagnosis, treatment planning, or patient-care decisions.

• Targets conventional radiographs; dental and mammographic images are out of scope.
• Predicts anatomy, not pathology.
• No laterality (left/right) and no uncertainty estimates.
• Performance on acquisition settings, views, or populations far from the training sources has not been validated.

Repository layout

• README.md: this card.
• ensemble.json: the flagship subfolder and the members list with their FluXray proportions.
• members/<name>/model.safetensors: exported model weights.
• members/<name>/config.yml: architecture and protocol config consumed by from_pretrained.
• members/<name>/label_schema.json: ordered output labels.
• members/<name>/preprocessing.json: public preprocessing contract.
• members/<name>/checksums.json: SHA256 checksums of the bundle files.
• members/<name>/onnx/flexray-<name>-256-fp16.onnx: fp16 ONNX export (opset 18, sigmoid baked in) used by the in-browser demo; parity-checked against the PyTorch weights by scripts.release.export_web_demo in the release tooling.

Licenses

• Code: MIT
• Weights: CC-BY-NC-4.0

Citation

@misc{butoi2026flexray,
      title={FleXray: Universal Clinical X-ray Segmentation},
      author={Victor Ion Butoi and Vivek Gopalakrishnan and John V. Guttag and Adrian V. Dalca and Neel Dey},
      year={2026},
      eprint={2609.26756},
      archivePrefix={arXiv},
      primaryClass={cs.CV},
      url={https://arxiv.org/abs/2609.26756},
}

Please also cite the source datasets listed in the flexray-data card for any dataset you use.