fixed the readmes
Browse files
README.md
CHANGED
|
@@ -9,27 +9,30 @@ an RT-DETR-based **document layout detector** (~33M params), for the
|
|
| 9 |
exported to `.pte` for the **ExecuTorch** runtime (XNNPACK, CoreML, Vulkan). It finds and
|
| 10 |
classifies document regions — titles, paragraphs, tables, figures, formulas, headers/footers,
|
| 11 |
etc. — and is a companion to
|
| 12 |
-
[`react-native-executorch-
|
| 13 |
|
| 14 |
If you'd like to run these models in your own ExecuTorch runtime, refer to the
|
| 15 |
[official documentation](https://pytorch.org/executorch/stable/index.html) for setup instructions.
|
| 16 |
|
| 17 |
-
The `.pte` is a pure tensor→tensor function;
|
| 18 |
-
threshold
|
| 19 |
|
| 20 |
## Output contract
|
| 21 |
|
| 22 |
-
A single
|
|
|
|
|
|
|
| 23 |
|
| 24 |
```
|
| 25 |
-
in [1, 3, 800, 800]
|
| 26 |
-
out
|
| 27 |
-
|
|
|
|
| 28 |
```
|
| 29 |
|
| 30 |
-
PP-DocLayoutV3 is a **DETR set-prediction** model → **no NMS**.
|
| 31 |
-
|
| 32 |
-
|
| 33 |
|
| 34 |
### Classes (25)
|
| 35 |
|
|
@@ -42,16 +45,14 @@ seal, table, text, vision_footnote` (some indices map to the same display label;
|
|
| 42 |
|
| 43 |
| backend | target | precision | size | latency |
|
| 44 |
|---|---|---|---|---|
|
| 45 |
-
| `xnnpack` | CPU | fp32 | 132 MB | ~2.0 s (S24) |
|
| 46 |
-
| `coreml` | Apple ANE | fp16 | 91 MB |
|
| 47 |
-
| `vulkan` | Android GPU | fp16 (mixed-delegate) | **66 MB** | **~0.86 s (S24)** |
|
| 48 |
|
| 49 |
> **Vulkan is the recommended Android backend** — ~2.4× faster than XNNPACK and half the size.
|
| 50 |
> It's mixed-delegate: most of RT-DETR runs fp16 on the GPU, while the box-head matmuls run on
|
| 51 |
-
> XNNPACK (they delegate as `addmm`→`linear`). XNNPACK stays fp32 because
|
| 52 |
-
>
|
| 53 |
-
|
| 54 |
-
|
| 55 |
|
| 56 |
## Compatibility
|
| 57 |
|
|
@@ -61,4 +62,3 @@ the compatibility note in the
|
|
| 61 |
[ExecuTorch GitHub repository](https://github.com/pytorch/executorch/blob/main/runtime/COMPATIBILITY.md).
|
| 62 |
If you work with React Native ExecuTorch, the library constants guarantee compatibility with the
|
| 63 |
runtime used behind the scenes.
|
| 64 |
-
|
|
|
|
| 9 |
exported to `.pte` for the **ExecuTorch** runtime (XNNPACK, CoreML, Vulkan). It finds and
|
| 10 |
classifies document regions — titles, paragraphs, tables, figures, formulas, headers/footers,
|
| 11 |
etc. — and is a companion to
|
| 12 |
+
[`react-native-executorch-pp-ocrv6`](https://huggingface.co/software-mansion/react-native-executorch-pp-ocrv6).
|
| 13 |
|
| 14 |
If you'd like to run these models in your own ExecuTorch runtime, refer to the
|
| 15 |
[official documentation](https://pytorch.org/executorch/stable/index.html) for setup instructions.
|
| 16 |
|
| 17 |
+
The `.pte` is a pure tensor→tensor function; pre-processing (resize, normalize) and the final
|
| 18 |
+
score threshold are the client's job.
|
| 19 |
|
| 20 |
## Output contract
|
| 21 |
|
| 22 |
+
A single **fixed-shape** method `forward` (shape also declared in `config.json`; no
|
| 23 |
+
shape-discovery companion methods on this model). The RT-DETR box decode is **baked into the
|
| 24 |
+
graph** — outputs are ready-to-threshold:
|
| 25 |
|
| 26 |
```
|
| 27 |
+
in [1, 3, 800, 800] # RGB, ImageNet-normalized by the client: (x/255 - mean)/std
|
| 28 |
+
out boxes [300, 4] # (x1, y1, x2, y2) in 800×800 model-input pixel space
|
| 29 |
+
scores [300] # max-class sigmoid score per query
|
| 30 |
+
classes [300] # float class index per query (argmax)
|
| 31 |
```
|
| 32 |
|
| 33 |
+
PP-DocLayoutV3 is a **DETR set-prediction** model → **no NMS**. All 300 queries are returned;
|
| 34 |
+
post-processing is just: keep rows with `score ≥ threshold`, scale boxes from the 800×800
|
| 35 |
+
input space to your image, and map `classes[i]` through `labels.json` (index → label).
|
| 36 |
|
| 37 |
### Classes (25)
|
| 38 |
|
|
|
|
| 45 |
|
| 46 |
| backend | target | precision | size | latency |
|
| 47 |
|---|---|---|---|---|
|
| 48 |
+
| `xnnpack` | CPU | fp32 | 132 MB | ~2.0 s (Galaxy S24) |
|
| 49 |
+
| `coreml` | Apple ANE | fp16 | 91 MB | ~50 ms (Apple M-series ANE) |
|
| 50 |
+
| `vulkan` | Android GPU | fp16 (mixed-delegate) | **66 MB** | **~0.86 s (Galaxy S24)** |
|
| 51 |
|
| 52 |
> **Vulkan is the recommended Android backend** — ~2.4× faster than XNNPACK and half the size.
|
| 53 |
> It's mixed-delegate: most of RT-DETR runs fp16 on the GPU, while the box-head matmuls run on
|
| 54 |
+
> XNNPACK (they delegate as `addmm`→`linear`). XNNPACK stays fp32 because int8/int4
|
| 55 |
+
> quantization loses whole boxes on this model.
|
|
|
|
|
|
|
| 56 |
|
| 57 |
## Compatibility
|
| 58 |
|
|
|
|
| 62 |
[ExecuTorch GitHub repository](https://github.com/pytorch/executorch/blob/main/runtime/COMPATIBILITY.md).
|
| 63 |
If you work with React Native ExecuTorch, the library constants guarantee compatibility with the
|
| 64 |
runtime used behind the scenes.
|
|
|