Commit ·
a5764d2
1
Parent(s): 402e59e
Add the model card standard
Browse filesEvery model repository's README follows one shape: four sections generated
from the tree and the config.json files, plus hand-written free sections for
what the artifacts cannot express. Records which facts a card must not
restate, so a card and the files it describes cannot disagree.
- MODEL_CARD.md +167 -0
- README.md +4 -3
MODEL_CARD.md
ADDED
|
@@ -0,0 +1,167 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# Model card standard
|
| 2 |
+
|
| 3 |
+
Every `software-mansion/react-native-executorch-*` repository carries a
|
| 4 |
+
`README.md` in this shape. Four sections are **generated** from the repository
|
| 5 |
+
tree and its `config.json` files, so they cannot drift from what is published.
|
| 6 |
+
Anything a model needs beyond them is a **free section**, written by hand and
|
| 7 |
+
kept as-is.
|
| 8 |
+
|
| 9 |
+
Before this standard the estate had 12 distinct heading sequences, four
|
| 10 |
+
heading-level conventions, and five different ExecuTorch versions claimed in
|
| 11 |
+
prose while 28 of 54 cards claimed none.
|
| 12 |
+
|
| 13 |
+
## Front matter
|
| 14 |
+
|
| 15 |
+
Hugging Face card metadata. Required on every repository:
|
| 16 |
+
|
| 17 |
+
| key | value |
|
| 18 |
+
| --- | --- |
|
| 19 |
+
| `license` | An SPDX-style Hub identifier for the **upstream** license, not ours. Use `other` plus `license_name` and `license_link` when the upstream license has no Hub identifier. |
|
| 20 |
+
| `pipeline_tag` | The Hub task this model serves, so it appears in task search. |
|
| 21 |
+
| `library_name` | Always `executorch`. |
|
| 22 |
+
|
| 23 |
+
Optional: `base_model`, `language`, `tags`. Keep the same identifier here and in
|
| 24 |
+
the `license` field of every `config.json` in the repository; they are checked
|
| 25 |
+
against each other.
|
| 26 |
+
|
| 27 |
+
## Generated sections
|
| 28 |
+
|
| 29 |
+
### 1. Title and introduction
|
| 30 |
+
|
| 31 |
+
The `# ` title is the repository slug with the `react-native-executorch-`
|
| 32 |
+
prefix dropped. One paragraph naming the library, then the upstream model on
|
| 33 |
+
its own line:
|
| 34 |
+
|
| 35 |
+
```markdown
|
| 36 |
+
# <slug>
|
| 37 |
+
|
| 38 |
+
This repository hosts the **<slug>** models exported for the
|
| 39 |
+
[React Native ExecuTorch](https://www.npmjs.com/package/react-native-executorch)
|
| 40 |
+
library as ExecuTorch `.pte` programs, ready to run on device.
|
| 41 |
+
|
| 42 |
+
Upstream model: [<name>](<url>)
|
| 43 |
+
```
|
| 44 |
+
|
| 45 |
+
Omit the upstream line when there is no single upstream to point at, rather
|
| 46 |
+
than guessing a URL.
|
| 47 |
+
|
| 48 |
+
### 2. Variants
|
| 49 |
+
|
| 50 |
+
One row per published `.pte`, read from the `variants` array of every
|
| 51 |
+
`config.json`.
|
| 52 |
+
|
| 53 |
+
```markdown
|
| 54 |
+
## Variants
|
| 55 |
+
|
| 56 |
+
| Path | Backend | Precision |
|
| 57 |
+
| --- | --- | --- |
|
| 58 |
+
| `xnnpack/model_xnnpack_fp32.pte` | xnnpack | fp32 |
|
| 59 |
+
| `xnnpack/model_xnnpack_int8.pte` | xnnpack | int8 |
|
| 60 |
+
| `coreml/model_coreml_fp16.pte` | coreml | fp16 |
|
| 61 |
+
```
|
| 62 |
+
|
| 63 |
+
A variant that ships several `.pte` files declares them as a `components` map
|
| 64 |
+
of role to filename, and contributes one row per component. Those repositories
|
| 65 |
+
gain a `Component` column naming the role; repositories without components do
|
| 66 |
+
not carry the column, so the common case stays three columns wide.
|
| 67 |
+
|
| 68 |
+
```markdown
|
| 69 |
+
| Path | Component | Backend | Precision |
|
| 70 |
+
| --- | --- | --- | --- |
|
| 71 |
+
| `coreml/clip_image_coreml_fp16.pte` | image | coreml | fp16 |
|
| 72 |
+
| `coreml/clip_text_coreml_fp16.pte` | text | coreml | fp16 |
|
| 73 |
+
```
|
| 74 |
+
|
| 75 |
+
The precision token is the whole story: `precisions.json` is authoritative for
|
| 76 |
+
which tokens are quantized, so the card does not repeat that partition. Nor
|
| 77 |
+
does it name a default, because the library resolves one per platform and a
|
| 78 |
+
second answer here would only compete with it.
|
| 79 |
+
|
| 80 |
+
### 3. Repository structure
|
| 81 |
+
|
| 82 |
+
Every tracked file except `README.md` and dotfiles, with its size, so a reader
|
| 83 |
+
can see the download cost before fetching.
|
| 84 |
+
|
| 85 |
+
```markdown
|
| 86 |
+
## Repository structure
|
| 87 |
+
|
| 88 |
+
```
|
| 89 |
+
coreml/config.json 1 kB
|
| 90 |
+
coreml/model_coreml_fp16.pte 8.3 MB
|
| 91 |
+
xnnpack/config.json 3 kB
|
| 92 |
+
xnnpack/model_xnnpack_fp32.pte 31.1 MB
|
| 93 |
+
```
|
| 94 |
+
```
|
| 95 |
+
|
| 96 |
+
Sizes render as `B`, `kB`, `MB` or `GB`.
|
| 97 |
+
|
| 98 |
+
### 4. Compatibility
|
| 99 |
+
|
| 100 |
+
Fixed text, parameterised only by the ExecuTorch runtime version. It states
|
| 101 |
+
the runtime the files are **published for**, and deliberately does not claim
|
| 102 |
+
the version each file was exported with: that is unrecorded for older exports,
|
| 103 |
+
and the five conflicting numbers the cards used to carry were all stale.
|
| 104 |
+
|
| 105 |
+
No react-native-executorch version appears here. Naming one would force an
|
| 106 |
+
edit to every card on each library release even when neither the runtime nor
|
| 107 |
+
the files changed, which is exactly the drift this standard exists to stop.
|
| 108 |
+
|
| 109 |
+
```markdown
|
| 110 |
+
## Compatibility
|
| 111 |
+
|
| 112 |
+
These files are published for the **ExecuTorch v1.4.1** runtime. ExecuTorch
|
| 113 |
+
gives no forward compatibility guarantee, so an older runtime may fail to load
|
| 114 |
+
them.
|
| 115 |
+
|
| 116 |
+
To use them in React Native ExecuTorch, pass the model constant shipped in the
|
| 117 |
+
library's model registry to the corresponding task pipeline. See the
|
| 118 |
+
[documentation](https://docs.swmansion.com/react-native-executorch/docs/fundamentals/loading-models).
|
| 119 |
+
|
| 120 |
+
To load these files in your own ExecuTorch runtime, read the
|
| 121 |
+
[compatibility note](https://github.com/pytorch/executorch/blob/main/runtime/COMPATIBILITY.md)
|
| 122 |
+
first.
|
| 123 |
+
```
|
| 124 |
+
|
| 125 |
+
Bump the version here only when the runtime the files are published for
|
| 126 |
+
changes.
|
| 127 |
+
|
| 128 |
+
## Free sections
|
| 129 |
+
|
| 130 |
+
Everything a model needs that neither the skeleton nor the repository's JSON
|
| 131 |
+
can express, appended after Compatibility at `##` level and left exactly as
|
| 132 |
+
written.
|
| 133 |
+
|
| 134 |
+
- `Export notes`, `CoreML notes (iOS)`: export decisions and per-backend
|
| 135 |
+
caveats. These live nowhere else, so the card is their home.
|
| 136 |
+
- `Model details`, `Languages`, `Classes (N)`: **only** when the model is
|
| 137 |
+
niche or the upstream link does not already cover it. For a model with a
|
| 138 |
+
well-documented upstream this duplicates a page we do not control and
|
| 139 |
+
cannot keep in sync.
|
| 140 |
+
|
| 141 |
+
Do not write a free section for:
|
| 142 |
+
|
| 143 |
+
- **The I/O contract.** The `config.json` files declare it, so restating it
|
| 144 |
+
here breaks rule 5. (Two shapes are in play today, the schema and the older
|
| 145 |
+
one reporting only upper bounds for dims, which is a reason to converge
|
| 146 |
+
them, not to re-document I/O by hand in 54 places.)
|
| 147 |
+
- **Latency or benchmark numbers.** They belong on the docs page, where they
|
| 148 |
+
are versioned and compared across models. Nobody looks in a model card for
|
| 149 |
+
them, and a stale number here is worse than none.
|
| 150 |
+
- **Required preprocessing.** `models.ts` declares it, and these files are
|
| 151 |
+
published to be consumed through the library, so its readers already have
|
| 152 |
+
the authoritative copy. A second copy here is one more thing to keep in
|
| 153 |
+
sync for an audience that does not need it.
|
| 154 |
+
|
| 155 |
+
Write a free section only when the answer to "how do I call this correctly?"
|
| 156 |
+
follows from neither the variant table nor the JSON.
|
| 157 |
+
|
| 158 |
+
## Rules
|
| 159 |
+
|
| 160 |
+
1. Heading levels are fixed: one `#` title, everything else `##`.
|
| 161 |
+
2. Never hand-edit a generated section. Fix the tree or the `config.json` and
|
| 162 |
+
re-render, so the card and the artifacts cannot disagree.
|
| 163 |
+
3. One ExecuTorch version appears in a card, in Compatibility.
|
| 164 |
+
4. Free sections carry the model-specific truth. Boilerplate belongs in the
|
| 165 |
+
generated skeleton, where it is written once.
|
| 166 |
+
5. Nothing in a card restates a fact the artifacts already hold. Derive it
|
| 167 |
+
instead, so the two cannot disagree.
|
README.md
CHANGED
|
@@ -18,13 +18,14 @@ model repositories under
|
|
| 18 |
| -------------------- | ----------------------------------------------------------------------------------------- |
|
| 19 |
| `config.schema.json` | JSON Schema for the per-backend `config.json` files shipped alongside each `.pte`. |
|
| 20 |
| `precisions.json` | Authoritative quantized / non-quantized partition for precision tokens used in file names. |
|
|
|
|
| 21 |
| `README.md` | This file. |
|
| 22 |
|
| 23 |
## Source of truth
|
| 24 |
|
| 25 |
-
The artifacts in this repository are the spec. Edit `config.schema.json`
|
| 26 |
-
`precisions.json` here, and treat them as authoritative
|
| 27 |
-
`config.json`.
|
| 28 |
|
| 29 |
## Versioning
|
| 30 |
|
|
|
|
| 18 |
| -------------------- | ----------------------------------------------------------------------------------------- |
|
| 19 |
| `config.schema.json` | JSON Schema for the per-backend `config.json` files shipped alongside each `.pte`. |
|
| 20 |
| `precisions.json` | Authoritative quantized / non-quantized partition for precision tokens used in file names. |
|
| 21 |
+
| `MODEL_CARD.md` | The README standard every model repository's card follows. |
|
| 22 |
| `README.md` | This file. |
|
| 23 |
|
| 24 |
## Source of truth
|
| 25 |
|
| 26 |
+
The artifacts in this repository are the spec. Edit `config.schema.json`,
|
| 27 |
+
`precisions.json` and `MODEL_CARD.md` here, and treat them as authoritative
|
| 28 |
+
for every published `config.json` and `README.md`.
|
| 29 |
|
| 30 |
## Versioning
|
| 31 |
|