benito47 commited on
Commit
ea25c16
·
1 Parent(s): ef42e3d

fixed the readmes

Browse files
Files changed (1) hide show
  1. README.md +18 -18
README.md CHANGED
@@ -9,27 +9,30 @@ an RT-DETR-based **document layout detector** (~33M params), for the
9
  exported to `.pte` for the **ExecuTorch** runtime (XNNPACK, CoreML, Vulkan). It finds and
10
  classifies document regions — titles, paragraphs, tables, figures, formulas, headers/footers,
11
  etc. — and is a companion to
12
- [`react-native-executorch-paddleocr`](https://huggingface.co/software-mansion/react-native-executorch-PP-OCRv6).
13
 
14
  If you'd like to run these models in your own ExecuTorch runtime, refer to the
15
  [official documentation](https://pytorch.org/executorch/stable/index.html) for setup instructions.
16
 
17
- The `.pte` is a pure tensor→tensor function; all pre/post-processing (resize, normalize, score
18
- threshold, box convert) is the client's job.
19
 
20
  ## Output contract
21
 
22
- A single static method `forward`, fixed input (no buckets):
 
 
23
 
24
  ```
25
- in [1, 3, 800, 800] # RGB, ImageNet-normalized (x/255 - mean)/std
26
- out logits [1, 300, 25] # 25 layout classes, per query (apply sigmoid)
27
- pred_boxes [1, 300, 4] # (cx, cy, w, h), normalized [0,1]
 
28
  ```
29
 
30
- PP-DocLayoutV3 is a **DETR set-prediction** model → **no NMS**. Post-processing is just:
31
- `score = sigmoid(logits)`, keep queries above a threshold, convert `(cx,cy,w,h) (x1,y1,x2,y2)`,
32
- scale to image size. Class names are in `labels.json` (index → label).
33
 
34
  ### Classes (25)
35
 
@@ -42,16 +45,14 @@ seal, table, text, vision_footnote` (some indices map to the same display label;
42
 
43
  | backend | target | precision | size | latency |
44
  |---|---|---|---|---|
45
- | `xnnpack` | CPU | fp32 | 132 MB | ~2.0 s (S24) |
46
- | `coreml` | Apple ANE | fp16 | 91 MB | ANE fp16 |
47
- | `vulkan` | Android GPU | fp16 (mixed-delegate) | **66 MB** | **~0.86 s (S24)** |
48
 
49
  > **Vulkan is the recommended Android backend** — ~2.4× faster than XNNPACK and half the size.
50
  > It's mixed-delegate: most of RT-DETR runs fp16 on the GPU, while the box-head matmuls run on
51
- > XNNPACK (they delegate as `addmm`→`linear`). XNNPACK stays fp32 because RT-DETR's deformable
52
- > attention feeds non-contiguous tensors that int8/portable paths mis-handle.
53
-
54
-
55
 
56
  ## Compatibility
57
 
@@ -61,4 +62,3 @@ the compatibility note in the
61
  [ExecuTorch GitHub repository](https://github.com/pytorch/executorch/blob/main/runtime/COMPATIBILITY.md).
62
  If you work with React Native ExecuTorch, the library constants guarantee compatibility with the
63
  runtime used behind the scenes.
64
-
 
9
  exported to `.pte` for the **ExecuTorch** runtime (XNNPACK, CoreML, Vulkan). It finds and
10
  classifies document regions — titles, paragraphs, tables, figures, formulas, headers/footers,
11
  etc. — and is a companion to
12
+ [`react-native-executorch-pp-ocrv6`](https://huggingface.co/software-mansion/react-native-executorch-pp-ocrv6).
13
 
14
  If you'd like to run these models in your own ExecuTorch runtime, refer to the
15
  [official documentation](https://pytorch.org/executorch/stable/index.html) for setup instructions.
16
 
17
+ The `.pte` is a pure tensor→tensor function; pre-processing (resize, normalize) and the final
18
+ score threshold are the client's job.
19
 
20
  ## Output contract
21
 
22
+ A single **fixed-shape** method `forward` (shape also declared in `config.json`; no
23
+ shape-discovery companion methods on this model). The RT-DETR box decode is **baked into the
24
+ graph** — outputs are ready-to-threshold:
25
 
26
  ```
27
+ in [1, 3, 800, 800] # RGB, ImageNet-normalized by the client: (x/255 - mean)/std
28
+ out boxes [300, 4] # (x1, y1, x2, y2) in 800×800 model-input pixel space
29
+ scores [300] # max-class sigmoid score per query
30
+ classes [300] # float class index per query (argmax)
31
  ```
32
 
33
+ PP-DocLayoutV3 is a **DETR set-prediction** model → **no NMS**. All 300 queries are returned;
34
+ post-processing is just: keep rows with `score threshold`, scale boxes from the 800×800
35
+ input space to your image, and map `classes[i]` through `labels.json` (index → label).
36
 
37
  ### Classes (25)
38
 
 
45
 
46
  | backend | target | precision | size | latency |
47
  |---|---|---|---|---|
48
+ | `xnnpack` | CPU | fp32 | 132 MB | ~2.0 s (Galaxy S24) |
49
+ | `coreml` | Apple ANE | fp16 | 91 MB | ~50 ms (Apple M-series ANE) |
50
+ | `vulkan` | Android GPU | fp16 (mixed-delegate) | **66 MB** | **~0.86 s (Galaxy S24)** |
51
 
52
  > **Vulkan is the recommended Android backend** — ~2.4× faster than XNNPACK and half the size.
53
  > It's mixed-delegate: most of RT-DETR runs fp16 on the GPU, while the box-head matmuls run on
54
+ > XNNPACK (they delegate as `addmm`→`linear`). XNNPACK stays fp32 because int8/int4
55
+ > quantization loses whole boxes on this model.
 
 
56
 
57
  ## Compatibility
58
 
 
62
  [ExecuTorch GitHub repository](https://github.com/pytorch/executorch/blob/main/runtime/COMPATIBILITY.md).
63
  If you work with React Native ExecuTorch, the library constants guarantee compatibility with the
64
  runtime used behind the scenes.