OpenVoice V2 — ToneColorConverter (Core ML)

Core ML (.mlpackage) exports of the ToneColorConverter from myshell-ai/OpenVoiceV2, for on-device inference on macOS and iOS with Apple’s Core ML runtime.

This repository is private and intended for on-device iOS / internal workflows. It does not include the PyTorch checkpoints or MeloTTS—only the converted Core ML packages and small JSON sidecars describing tensor layouts.

What’s included

Artifact	Description
`ToneColorConverter_ReferenceEncoder.mlpackage`	Reference mel → tone embedding `g`
`ToneColorConverter_PosteriorEncoder.mlpackage`	Base linear STFT → latent `z`
`ToneColorConverter_Flow_Forward.mlpackage`	Source tone removal (`z` → `z_p`)
`ToneColorConverter_Flow_Reverse.mlpackage`	Target tone injection (`z_p` → `z_hat`)
`ToneColorConverter_Decoder.mlpackage`	HiFi-GAN decoder (`z_hat` → waveform)
`ToneColorConverter_E2E.mlpackage`	Single-shot pipeline: STFT + `sid_src` / `sid_tgt` → waveform
`pipeline_info.json`	Shapes and execution order for the 5-split path
`pipeline_info_e2e.json`	Inputs/outputs for the E2E model

Sampling rate and STFT settings match the OpenVoice V2 converter (e.g. 22050 Hz, linear STFT layout consistent with the PyTorch reference implementation). See the JSON files for exact ranks and notes.

Audio samples

Example WAV outputs are in the samples/ folder (listening tests only):

File	Description
`samples/clone_1_text_eng.wav`	MeloTTS English base + Core ML E2E tone conversion; timbre from a Japanese reference clip (~5.5 s, 22050 Hz mono).
`samples/tts_coreml_out.wav`	Python (`run_openvoice_tts_coreml.py`) + Core ML run.
`samples/swift_tts_out.wav`	Swift CLI (`openvoice-coreml-tts`) run.

See samples/SAMPLE_INFO.md for a short legend.

Intended use

Voice / tone conversion on Apple Silicon or Apple Neural Engine–capable devices, using embeddings extracted from a short reference clip.
E2E path: supply linear magnitude STFT y, frame count, and precomputed source/target embeddings (sid_src, sid_tgt).
5-split path: orchestrate the five models in order (see pipeline_info.json) when you need intermediate tensors or tighter control.

Base speech synthesis (e.g. MeloTTS) is not part of these packages; generate or load a base waveform (or spectrogram) in your app or server, then run ToneColorConverter.

Limitations

Posterior path uses a deterministic (tau≈0 style) formulation suitable for Core ML export; behaviour may differ slightly from full stochastic sampling in PyTorch.
You may see MILCompilerForANE / E5RT warnings on some macOS versions; inference can still fall back to CPU/GPU.
Large .mlpackage weights are binary blobs—clone with Git LFS if you mirror this repo outside the Hub.

Download

export HF_TOKEN="***"   # read token with access to this private repo
huggingface-cli download aoiandroid/OpenVoiceV2-CoreML-mirror --local-dir ./OpenVoiceV2-CoreML-on-device iOS client

Or with Python:

from huggingface_hub import snapshot_download
snapshot_download(
    "aoiandroid/OpenVoiceV2-CoreML-mirror",
    local_dir="OpenVoiceV2-CoreML-on-device iOS client",
    token=os.environ["HF_TOKEN"],
)

Source & license

Upstream model: myshell-ai/OpenVoiceV2 (research / demo use per upstream terms).
OpenVoice code & weights are subject to the original MIT license (see OpenVoice LICENSE).
This Core ML conversion is a derivative packaging for Apple platforms; retain upstream notices when redistributing.

Citation

If you use OpenVoice, cite the upstream project as in the official repository.

Model card maintained for the on-device iOS client OpenVoice V2 Core ML export (2026).

Downloads last month: 22