tianma Cursor commited on
Commit
4c4a6cb
·
0 Parent(s):

Initial commit: FlexICM codebase, configs, and documentation.

Browse files
This view is limited to 50 files because it contains too many changes.   See raw diff
Files changed (50) hide show
  1. .gitignore +26 -0
  2. README.md +316 -0
  3. checkpoints/README.md +64 -0
  4. checkpoints/base_codec/NOTE.txt +2 -0
  5. checkpoints/base_codec/PLACEHOLDER_base_codec_1.pth.tar.txt +2 -0
  6. checkpoints/base_codec/PLACEHOLDER_base_codec_2.pth.tar.txt +2 -0
  7. checkpoints/base_codec/PLACEHOLDER_base_codec_3.pth.tar.txt +2 -0
  8. checkpoints/base_codec/PLACEHOLDER_base_codec_4.pth.tar.txt +2 -0
  9. checkpoints/ctaic/s1_det_instance/stage1/1/PLACEHOLDER +2 -0
  10. checkpoints/ctaic/s1_det_instance/stage1/2/PLACEHOLDER +2 -0
  11. checkpoints/ctaic/s1_det_instance/stage1/3/PLACEHOLDER +2 -0
  12. checkpoints/ctaic/s1_det_instance/stage1/4/PLACEHOLDER +2 -0
  13. checkpoints/ctaic/s1_det_instance/stage2/1/PLACEHOLDER +2 -0
  14. checkpoints/ctaic/s1_det_instance/stage2/2/PLACEHOLDER +2 -0
  15. checkpoints/ctaic/s1_det_instance/stage2/3/PLACEHOLDER +2 -0
  16. checkpoints/ctaic/s1_det_instance/stage2/4/PLACEHOLDER +2 -0
  17. checkpoints/ctaic/s2_sem_panoptic/stage1/1/PLACEHOLDER +2 -0
  18. checkpoints/ctaic/s2_sem_panoptic/stage1/2/PLACEHOLDER +2 -0
  19. checkpoints/ctaic/s2_sem_panoptic/stage1/3/PLACEHOLDER +2 -0
  20. checkpoints/ctaic/s2_sem_panoptic/stage1/4/PLACEHOLDER +2 -0
  21. checkpoints/ctaic/s2_sem_panoptic/stage2/1/PLACEHOLDER +2 -0
  22. checkpoints/ctaic/s2_sem_panoptic/stage2/2/PLACEHOLDER +2 -0
  23. checkpoints/ctaic/s2_sem_panoptic/stage2/3/PLACEHOLDER +2 -0
  24. checkpoints/ctaic/s2_sem_panoptic/stage2/4/PLACEHOLDER +2 -0
  25. checkpoints/ctaic/s3_det_pose/stage1/1/PLACEHOLDER +2 -0
  26. checkpoints/ctaic/s3_det_pose/stage1/2/PLACEHOLDER +2 -0
  27. checkpoints/ctaic/s3_det_pose/stage1/3/PLACEHOLDER +2 -0
  28. checkpoints/ctaic/s3_det_pose/stage1/4/PLACEHOLDER +2 -0
  29. checkpoints/ctaic/s3_det_pose/stage2/1/PLACEHOLDER +2 -0
  30. checkpoints/ctaic/s3_det_pose/stage2/2/PLACEHOLDER +2 -0
  31. checkpoints/ctaic/s3_det_pose/stage2/3/PLACEHOLDER +2 -0
  32. checkpoints/ctaic/s3_det_pose/stage2/4/PLACEHOLDER +2 -0
  33. checkpoints/taic/detection/1/PLACEHOLDER +2 -0
  34. checkpoints/taic/detection/2/PLACEHOLDER +2 -0
  35. checkpoints/taic/detection/3/PLACEHOLDER +2 -0
  36. checkpoints/taic/detection/4/PLACEHOLDER +2 -0
  37. checkpoints/taic/instance/1/PLACEHOLDER +2 -0
  38. checkpoints/taic/instance/2/PLACEHOLDER +2 -0
  39. checkpoints/taic/instance/3/PLACEHOLDER +2 -0
  40. checkpoints/taic/instance/4/PLACEHOLDER +2 -0
  41. checkpoints/taic/panoptic/1/PLACEHOLDER +2 -0
  42. checkpoints/taic/panoptic/2/PLACEHOLDER +2 -0
  43. checkpoints/taic/panoptic/3/PLACEHOLDER +2 -0
  44. checkpoints/taic/panoptic/4/PLACEHOLDER +2 -0
  45. checkpoints/taic/pose/1/PLACEHOLDER +2 -0
  46. checkpoints/taic/pose/2/PLACEHOLDER +2 -0
  47. checkpoints/taic/pose/3/PLACEHOLDER +2 -0
  48. checkpoints/taic/pose/4/PLACEHOLDER +2 -0
  49. checkpoints/taic/semantic/1/PLACEHOLDER +2 -0
  50. checkpoints/taic/semantic/2/PLACEHOLDER +2 -0
.gitignore ADDED
@@ -0,0 +1,26 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ __pycache__/
2
+ *.py[cod]
3
+ *.egg-info/
4
+ .eggs/
5
+ dist/
6
+ build/
7
+ .DS_Store
8
+ logs/
9
+ *.log
10
+ coco_log/
11
+ .ipynb_checkpoints/
12
+ _figs/
13
+ _paper_extract.txt
14
+
15
+ # paper PDF — do not upload
16
+ FlexICM.pdf
17
+ *.pdf
18
+
19
+ # real weights (keep PLACEHOLDER text files and README)
20
+ checkpoints/**/*.pth.tar
21
+ checkpoints/**/*.tar
22
+ checkpoints/**/*.pkl
23
+ checkpoints/**/*.pth
24
+ !checkpoints/**/PLACEHOLDER*
25
+ !checkpoints/**/*.txt
26
+ !checkpoints/README.md
README.md ADDED
@@ -0,0 +1,316 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # FlexICM: A Flexible Image Coding for Machines Framework
2
+
3
+ Official codebase for the paper **FlexICM: A Flexible Image Coding for Machines Framework** (Tianma Shen, Ying Liu).
4
+
5
+ Built on the **TIC (Transformer-based Image Compression)** base codec, this repository implements:
6
+
7
+ - **TAIC (Base Layer)**: five single-task codecs that decode task intermediate features `h` **without** full image reconstruction
8
+ - **C-TAIC (Extension Layer)**: three multi-task scenarios that condition on the base-layer latent \(\hat{y}_b\) via cross-attention
9
+
10
+ ## Five Tasks and Three Scenarios
11
+
12
+ ### TAIC (five task codecs)
13
+
14
+ | Task | Teacher / Task Network | Feature Alignment | Metric |
15
+ |------|------------------------|-------------------|--------|
16
+ | Object Detection | Faster R-CNN + **Swin-B** | FPN `P2..P6` (Eq. 2) | mAP-bbox |
17
+ | Semantic Segmentation | UPerNet + **Swin-B** | FPN `P2..P6` | mIoU |
18
+ | Instance Segmentation | Mask R-CNN + **Swin-B** | FPN `P2..P6` | mAP-mask |
19
+ | Panoptic Segmentation | MaskFormer + **Swin-B** | Stages `F1..F4` (Eq. 3) | PQ |
20
+ | Pose Estimation | **HigherHRNet** | Stages `F1..F4` | mAP-OKS |
21
+
22
+ ### C-TAIC (three scenarios)
23
+
24
+ | Scenario | Base Layer | Extension Layer |
25
+ |----------|------------|-----------------|
26
+ | **s1** | Object Detection | Instance Segmentation |
27
+ | **s2** | Semantic Segmentation | Panoptic Segmentation |
28
+ | **s3** | Object Detection | Pose Estimation |
29
+
30
+ ---
31
+
32
+ ## Environment Setup
33
+
34
+ > **Important:** Codec training **requires** task networks (teachers) to be available.
35
+ > The loss \(D\) is computed from frozen teacher features, so you cannot train TAIC / C-TAIC
36
+ > with only the codec packages. Install the teacher stack in **Task networks (teachers)** before the first training run.
37
+
38
+ ### Recommended environment
39
+
40
+ - Ubuntu / RHEL, **CUDA 11.7+**, single **NVIDIA A100** (paper setting)
41
+ - Python **3.8–3.10**
42
+ - PyTorch **≥ 1.12** (2.0+ recommended)
43
+
44
+ ```bash
45
+ conda create -n flexicm python=3.9 -y
46
+ conda activate flexicm
47
+ pip install torch torchvision --index-url https://download.pytorch.org/whl/cu118
48
+ pip install -r requirements.txt
49
+ ```
50
+
51
+ ### Core codec dependencies
52
+
53
+ | Package | Role |
54
+ |---------|------|
55
+ | `compressai` | EntropyBottleneck / GaussianConditional / conv-deconv |
56
+ | `timm` | **Required** Swin-B teacher backbone for feature alignment |
57
+ | `PyYAML` | Training configs |
58
+
59
+ ### Task networks (teachers) — **required before training**
60
+
61
+ Teachers are already implemented in `flexicm/tasks/` and are constructed automatically by
62
+ `scripts/train_taic.py` / `scripts/train_ctaic.py` via `build_teacher(...)`.
63
+ You still must install their runtime dependencies and allow pretrained weights to download.
64
+
65
+ | Task | Teacher used in training | What you need installed |
66
+ |------|--------------------------|-------------------------|
67
+ | Detection / Instance / Semantic / Panoptic | Swin-B backbone (+ FPN or stages) via `timm` | `timm` (from `requirements.txt`); first run downloads ImageNet-pretrained Swin-B |
68
+ | Pose | HigherHRNet-style HRNet stem (original HRNet, not Swin) | Implemented in-repo; no extra package beyond PyTorch |
69
+
70
+ Checklist before training:
71
+
72
+ 1. `pip install -r requirements.txt` (includes `timm`)
73
+ 2. Machine can reach the internet **or** you have cached `timm` Swin-B weights (for the four Swin tasks)
74
+ 3. Verify teachers import cleanly:
75
+
76
+ ```bash
77
+ python -c "from flexicm.tasks import build_teacher; build_teacher('detection'); print('teachers ok')"
78
+ ```
79
+
80
+ Without a working teacher, training will fail when computing the feature-alignment term \(D\).
81
+
82
+ ### Task heads for metric evaluation
83
+
84
+ To evaluate paper metrics (mAP / mIoU / PQ / OKS) with full task heads, also install:
85
+
86
+ ```bash
87
+ pip install -U openmim
88
+ mim install mmengine mmcv
89
+ mim install mmdet mmsegmentation mmpose
90
+ # or Detectron2 (alternative for detection / instance evaluation)
91
+ ```
92
+
93
+ Recommended official weights (same model families as the paper):
94
+
95
+ - **Faster / Mask R-CNN + Swin-B**: MMDetection Model Zoo
96
+ - **UPerNet + Swin-B**: MMSegmentation Model Zoo
97
+ - **MaskFormer + Swin-B**: MMDetection / Mask2Former
98
+ - **HigherHRNet**: MMPose Model Zoo (**HRNet backbone**)
99
+
100
+ These full heads are **not** required to start codec training; they are for final rate–accuracy evaluation.
101
+
102
+ ---
103
+
104
+ ## Repository Layout
105
+
106
+ ```
107
+ FlexICM/
108
+ ├── FlexICM.pdf # paper
109
+ ├── requirements.txt
110
+ ├── README.md
111
+ ├── configs/
112
+ │ ├── taic/ # five single-task configs
113
+ │ │ ├── detection.yaml
114
+ │ │ ├── semantic.yaml
115
+ │ │ ├── instance.yaml
116
+ │ │ ├── panoptic.yaml
117
+ │ │ └── pose.yaml
118
+ │ └── ctaic/ # three multi-task scenarios
119
+ │ ├── s1_det_instance.yaml
120
+ │ ├── s2_sem_panoptic.yaml
121
+ │ └── s3_det_pose.yaml
122
+ ├── scripts/
123
+ │ ├── download_base_codecs.sh
124
+ │ ├── train_taic.py
125
+ │ └── train_ctaic.py
126
+ ├── flexicm/
127
+ │ ├── models/ # TAIC / C-TAIC / SFMA / TaskConnector / Conditional
128
+ │ ├── layers/ # RSTB / WindowAttention (same lineage as AdaptiveICMH)
129
+ │ ├── tasks/ # teachers + feature-alignment losses
130
+ │ ├── data/ # COCO / COCO-WholeBody image loading
131
+ │ └── utils/
132
+ └── checkpoints/ # placeholder tree for base / TAIC / C-TAIC weights
133
+ # see checkpoints/README.md
134
+ ```
135
+
136
+ Eval configs (stub until full metrics are implemented): `configs/eval/`.
137
+ Eval entry points: `scripts/eval_taic.py`, `scripts/eval_ctaic.py` (currently only check that real checkpoints replaced `PLACEHOLDER` files).
138
+
139
+ ---
140
+
141
+ ## Dataset Preparation
142
+
143
+ ### COCO-2017 (detection / instance / semantic / panoptic)
144
+
145
+ ```text
146
+ /data/coco2017/
147
+ ├── train2017/
148
+ ├── val2017/
149
+ └── annotations/
150
+ ├── instances_train2017.json
151
+ ├── instances_val2017.json
152
+ ├── panoptic_train2017.json
153
+ ├── panoptic_val2017.json
154
+ ├── panoptic_train2017/ # PNG
155
+ ├── panoptic_val2017/
156
+ ├── stuff_train2017.json # semantic / stuff (if used)
157
+ └── stuff_val2017.json
158
+ ```
159
+
160
+ Download:
161
+
162
+ ```bash
163
+ # images
164
+ wget http://images.cocodataset.org/zips/train2017.zip
165
+ wget http://images.cocodataset.org/zips/val2017.zip
166
+ # annotations
167
+ wget http://images.cocodataset.org/annotations/annotations_trainval2017.zip
168
+ wget http://images.cocodataset.org/annotations/panoptic_annotations_trainval2017.zip
169
+ ```
170
+
171
+ Set in the corresponding YAML:
172
+
173
+ ```yaml
174
+ dataset_path: "/data/coco2017"
175
+ ```
176
+
177
+ ### COCO-WholeBody (pose estimation)
178
+
179
+ Pose uses the same COCO `train2017/val2017` images plus WholeBody keypoint annotations:
180
+
181
+ - Project page: [COCO-WholeBody](https://github.com/jin-s13/COCO-WholeBody)
182
+ - Place JSON files under `annotations/`; evaluate with MMPose HigherHRNet + WholeBody configs
183
+
184
+ Codec **training** only needs images for feature alignment, so `train2017` images are sufficient for that stage.
185
+
186
+ ### Data processing (aligned with task networks)
187
+
188
+ The paper requires **codec training preprocessing to match task-network preprocessing**. Defaults in this repo:
189
+
190
+ 1. **Codec input**: RGB, `ToTensor()` → `[0,1]`; training uses `Resize → RandomCrop(256) → RandomHorizontalFlip`
191
+ 2. **Inside the teacher**: ImageNet mean/std normalization (consistent with Swin / HRNet pretraining)
192
+ 3. **Spatial alignment**: TIC requires spatial size divisible by **256** (256 crop for training; pad at inference)
193
+
194
+ If you use official MMDet/MMSeg pipelines (short-side resize, normalization, etc.), ensure:
195
+
196
+ - Teacher feature extraction uses the **same normalize / resize logic** as that task network
197
+ - Codec and teacher see geometrically consistent tensors (same crop / same pad)
198
+
199
+ Edit points: `flexicm/data/datasets.py`, `flexicm/tasks/swin_teacher.py`, `flexicm/tasks/__init__.py` (HigherHRNet).
200
+
201
+ ---
202
+
203
+ ## Base Codec (TIC) Checkpoints
204
+
205
+ The paper uses the same TIC pretrained weights as AdaptiveICMH / TransTIC:
206
+
207
+ | Quality | λ (paper) | Checkpoint |
208
+ |:-------:|:---------:|------------|
209
+ | 1 | 0.0035 | [base_codec_1](https://github.com/NYCU-MAPL/TransTIC/releases/download/v1.0/base_codec_1.pth.tar) |
210
+ | 2 | 0.0067 | [base_codec_2](https://github.com/NYCU-MAPL/TransTIC/releases/download/v1.0/base_codec_2.pth.tar) |
211
+ | 3 | 0.0130 | [base_codec_3](https://github.com/NYCU-MAPL/TransTIC/releases/download/v1.0/base_codec_3.pth.tar) |
212
+ | 4 | 0.0250 | [base_codec_4](https://github.com/NYCU-MAPL/TransTIC/releases/download/v1.0/base_codec_4.pth.tar) |
213
+
214
+ ```bash
215
+ bash scripts/download_base_codecs.sh
216
+ # downloads into checkpoints/base_codec/base_codec_{1,2,3,4}.pth.tar
217
+ ```
218
+
219
+ Config example:
220
+
221
+ ```yaml
222
+ base_codec: "./checkpoints/base_codec/base_codec_1.pth.tar"
223
+ quality_level: 1
224
+ lmbda: 0.0035
225
+ ```
226
+
227
+ For each bitrate point, switch the matching `base_codec_k` and `lmbda`.
228
+
229
+ Trained TAIC / C-TAIC weights for eval should be placed under `checkpoints/taic/` and
230
+ `checkpoints/ctaic/` (see `checkpoints/README.md`). Until then, each quality folder
231
+ contains a `PLACEHOLDER` file.
232
+
233
+
234
+ ---
235
+
236
+ ## Training
237
+
238
+ Paper settings:
239
+
240
+ - Optimizer: **AdamW**, `lr=1e-4`
241
+ - TAIC: `batch_size=80`, `epochs=35`
242
+ - C-TAIC: `batch_size=40`, `epochs=40`
243
+ - `λ ∈ {0.0035, 0.0067, 0.0130, 0.0250}`
244
+
245
+ > If GPU memory is insufficient, reduce `batch_size` (optionally use gradient accumulation to approximate the paper effective batch).
246
+
247
+ ### Train five TAIC models
248
+
249
+ ```bash
250
+ # edit dataset_path / base_codec / lmbda / gpu_id in configs/taic/*.yaml as needed
251
+ python scripts/train_taic.py -c configs/taic/detection.yaml
252
+ python scripts/train_taic.py -c configs/taic/semantic.yaml
253
+ python scripts/train_taic.py -c configs/taic/instance.yaml
254
+ python scripts/train_taic.py -c configs/taic/panoptic.yaml
255
+ python scripts/train_taic.py -c configs/taic/pose.yaml
256
+ ```
257
+
258
+ Trainable modules: **encoder SFMA + Task Connector**; TIC trunk is frozen.
259
+
260
+ ### Train three C-TAIC scenarios
261
+
262
+ Requires a trained **base TAIC** checkpoint (to provide \(\hat{y}_b\)) and Stage-1 weights for the extension task.
263
+
264
+ ```bash
265
+ # ---- s1: det → instance ----
266
+ python scripts/train_ctaic.py -c configs/ctaic/s1_det_instance.yaml --stage 1
267
+ python scripts/train_ctaic.py -c configs/ctaic/s1_det_instance.yaml --stage 2
268
+
269
+ # ---- s2: semantic → panoptic ----
270
+ python scripts/train_ctaic.py -c configs/ctaic/s2_sem_panoptic.yaml --stage 1
271
+ python scripts/train_ctaic.py -c configs/ctaic/s2_sem_panoptic.yaml --stage 2
272
+
273
+ # ---- s3: det → pose ----
274
+ python scripts/train_ctaic.py -c configs/ctaic/s3_det_pose.yaml --stage 1
275
+ python scripts/train_ctaic.py -c configs/ctaic/s3_det_pose.yaml --stage 2
276
+ ```
277
+
278
+ Stage meanings:
279
+
280
+ | Stage | Mode | Trainable modules | `ŷ_b` |
281
+ |:-----:|------|-------------------|-------|
282
+ | 1 | TAIC mode | SFMA + Task Connector | not used |
283
+ | 2 | C-TAIC mode | Prompt Generator + Condition Generator | from frozen base TAIC AD output |
284
+
285
+ Check these config fields:
286
+
287
+ ```yaml
288
+ base_taic_checkpoint: # trained TAIC for the base task
289
+ taic_init: # extension-task TAIC (optional Stage-1 init)
290
+ stage1_checkpoint: # Stage-1 result loaded in Stage 2
291
+ ```
292
+
293
+ ---
294
+
295
+ ## Code Map to the Paper
296
+
297
+ | Paper component | Code location |
298
+ |-----------------|---------------|
299
+ | SFMA | `flexicm/models/sfma.py` |
300
+ | Task Connector | `flexicm/models/task_connector.py` |
301
+ | TAIC | `flexicm/models/taic.py` |
302
+ | C-TAIC + two-stage freeze | `flexicm/models/ctaic.py` |
303
+ | Prompt / Mask / Cd | `flexicm/models/conditional.py` |
304
+ | Cross-attention (Q from features; K/V include prompts) | `flexicm/models/cross_attention.py` |
305
+ | \(R+\lambda D\) | `flexicm/tasks/losses.py` |
306
+ | Five teachers | `flexicm/tasks/__init__.py` |
307
+
308
+
309
+ ---
310
+
311
+ ## Citation
312
+
313
+ If you use this code or the paper, please cite FlexICM and acknowledge the base works:
314
+
315
+ - TIC: Lu et al., Transformer-based Image Compression
316
+ - TransTIC / AdaptiveICMH: task-adaptive SFMA tuning
checkpoints/README.md ADDED
@@ -0,0 +1,64 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # Checkpoints Layout (Placeholders)
2
+
3
+ Put pretrained / trained weights here before running **test / eval**.
4
+ Training still writes to `logs/` by default; after training, copy (or symlink) best
5
+ checkpoints into this tree so eval configs have a stable path.
6
+
7
+ > Files named `PLACEHOLDER` are not real weights. Replace each with the matching
8
+ > `.pth.tar` checkpoint, then update or keep the path expected by eval scripts.
9
+
10
+ ## Directory map
11
+
12
+ ```text
13
+ checkpoints/
14
+ ├── base_codec/ # frozen TIC (TransTIC / AdaptiveICMH)
15
+ │ ├── base_codec_1.pth.tar # λ = 0.0035
16
+ │ ├── base_codec_2.pth.tar # λ = 0.0067
17
+ │ ├── base_codec_3.pth.tar # λ = 0.0130
18
+ │ └── base_codec_4.pth.tar # λ = 0.0250
19
+
20
+ ├── taic/ # five single-task TAIC codecs
21
+ │ ├── detection/{1,2,3,4}/checkpoint_best_loss.pth.tar
22
+ │ ├── semantic/{1,2,3,4}/checkpoint_best_loss.pth.tar
23
+ │ ├── instance/{1,2,3,4}/checkpoint_best_loss.pth.tar
24
+ │ ├── panoptic/{1,2,3,4}/checkpoint_best_loss.pth.tar
25
+ │ └── pose/{1,2,3,4}/checkpoint_best_loss.pth.tar
26
+
27
+ └── ctaic/ # three multi-task scenarios
28
+ ├── s1_det_instance/
29
+ │ ├── stage1/{1,2,3,4}/checkpoint_best_loss.pth.tar
30
+ │ └── stage2/{1,2,3,4}/checkpoint_best_loss.pth.tar
31
+ ├── s2_sem_panoptic/
32
+ │ ├── stage1/{1,2,3,4}/checkpoint_best_loss.pth.tar
33
+ │ └── stage2/{1,2,3,4}/checkpoint_best_loss.pth.tar
34
+ └── s3_det_pose/
35
+ ├── stage1/{1,2,3,4}/checkpoint_best_loss.pth.tar
36
+ └── stage2/{1,2,3,4}/checkpoint_best_loss.pth.tar
37
+ ```
38
+
39
+ Quality folders `{1,2,3,4}` match paper λ / TIC quality levels.
40
+
41
+ ## Download base TIC codecs
42
+
43
+ ```bash
44
+ bash scripts/download_base_codecs.sh
45
+ # downloads into checkpoints/base_codec/
46
+ ```
47
+
48
+ ## After training: copy into placeholders
49
+
50
+ ```bash
51
+ # example: TAIC detection, quality 1
52
+ cp logs/taic_detection/1/checkpoint_best_loss.pth.tar \
53
+ checkpoints/taic/detection/1/checkpoint_best_loss.pth.tar
54
+
55
+ # example: C-TAIC s1 stage2, quality 1
56
+ cp logs/ctaic_s1_stage2/1/checkpoint_best_loss.pth.tar \
57
+ checkpoints/ctaic/s1_det_instance/stage2/1/checkpoint_best_loss.pth.tar
58
+ ```
59
+
60
+ ## Eval configs
61
+
62
+ See `configs/eval/` — they point to these placeholder paths.
63
+ Eval scripts will refuse to run if a `PLACEHOLDER` file is still present
64
+ or if the `.pth.tar` is missing.
checkpoints/base_codec/NOTE.txt ADDED
@@ -0,0 +1,2 @@
 
 
 
1
+ Legacy config paths used ./checkpoints/base_codec_k.pth.tar.
2
+ Prefer checkpoints/base_codec/base_codec_k.pth.tar after download.
checkpoints/base_codec/PLACEHOLDER_base_codec_1.pth.tar.txt ADDED
@@ -0,0 +1,2 @@
 
 
 
1
+ PLACEHOLDER: replace this file with the real checkpoint (.pth.tar).
2
+ See checkpoints/README.md for the expected filename and training copy commands.
checkpoints/base_codec/PLACEHOLDER_base_codec_2.pth.tar.txt ADDED
@@ -0,0 +1,2 @@
 
 
 
1
+ PLACEHOLDER: replace this file with the real checkpoint (.pth.tar).
2
+ See checkpoints/README.md for the expected filename and training copy commands.
checkpoints/base_codec/PLACEHOLDER_base_codec_3.pth.tar.txt ADDED
@@ -0,0 +1,2 @@
 
 
 
1
+ PLACEHOLDER: replace this file with the real checkpoint (.pth.tar).
2
+ See checkpoints/README.md for the expected filename and training copy commands.
checkpoints/base_codec/PLACEHOLDER_base_codec_4.pth.tar.txt ADDED
@@ -0,0 +1,2 @@
 
 
 
1
+ PLACEHOLDER: replace this file with the real checkpoint (.pth.tar).
2
+ See checkpoints/README.md for the expected filename and training copy commands.
checkpoints/ctaic/s1_det_instance/stage1/1/PLACEHOLDER ADDED
@@ -0,0 +1,2 @@
 
 
 
1
+ PLACEHOLDER: replace this file with the real checkpoint (.pth.tar).
2
+ See checkpoints/README.md for the expected filename and training copy commands.
checkpoints/ctaic/s1_det_instance/stage1/2/PLACEHOLDER ADDED
@@ -0,0 +1,2 @@
 
 
 
1
+ PLACEHOLDER: replace this file with the real checkpoint (.pth.tar).
2
+ See checkpoints/README.md for the expected filename and training copy commands.
checkpoints/ctaic/s1_det_instance/stage1/3/PLACEHOLDER ADDED
@@ -0,0 +1,2 @@
 
 
 
1
+ PLACEHOLDER: replace this file with the real checkpoint (.pth.tar).
2
+ See checkpoints/README.md for the expected filename and training copy commands.
checkpoints/ctaic/s1_det_instance/stage1/4/PLACEHOLDER ADDED
@@ -0,0 +1,2 @@
 
 
 
1
+ PLACEHOLDER: replace this file with the real checkpoint (.pth.tar).
2
+ See checkpoints/README.md for the expected filename and training copy commands.
checkpoints/ctaic/s1_det_instance/stage2/1/PLACEHOLDER ADDED
@@ -0,0 +1,2 @@
 
 
 
1
+ PLACEHOLDER: replace this file with the real checkpoint (.pth.tar).
2
+ See checkpoints/README.md for the expected filename and training copy commands.
checkpoints/ctaic/s1_det_instance/stage2/2/PLACEHOLDER ADDED
@@ -0,0 +1,2 @@
 
 
 
1
+ PLACEHOLDER: replace this file with the real checkpoint (.pth.tar).
2
+ See checkpoints/README.md for the expected filename and training copy commands.
checkpoints/ctaic/s1_det_instance/stage2/3/PLACEHOLDER ADDED
@@ -0,0 +1,2 @@
 
 
 
1
+ PLACEHOLDER: replace this file with the real checkpoint (.pth.tar).
2
+ See checkpoints/README.md for the expected filename and training copy commands.
checkpoints/ctaic/s1_det_instance/stage2/4/PLACEHOLDER ADDED
@@ -0,0 +1,2 @@
 
 
 
1
+ PLACEHOLDER: replace this file with the real checkpoint (.pth.tar).
2
+ See checkpoints/README.md for the expected filename and training copy commands.
checkpoints/ctaic/s2_sem_panoptic/stage1/1/PLACEHOLDER ADDED
@@ -0,0 +1,2 @@
 
 
 
1
+ PLACEHOLDER: replace this file with the real checkpoint (.pth.tar).
2
+ See checkpoints/README.md for the expected filename and training copy commands.
checkpoints/ctaic/s2_sem_panoptic/stage1/2/PLACEHOLDER ADDED
@@ -0,0 +1,2 @@
 
 
 
1
+ PLACEHOLDER: replace this file with the real checkpoint (.pth.tar).
2
+ See checkpoints/README.md for the expected filename and training copy commands.
checkpoints/ctaic/s2_sem_panoptic/stage1/3/PLACEHOLDER ADDED
@@ -0,0 +1,2 @@
 
 
 
1
+ PLACEHOLDER: replace this file with the real checkpoint (.pth.tar).
2
+ See checkpoints/README.md for the expected filename and training copy commands.
checkpoints/ctaic/s2_sem_panoptic/stage1/4/PLACEHOLDER ADDED
@@ -0,0 +1,2 @@
 
 
 
1
+ PLACEHOLDER: replace this file with the real checkpoint (.pth.tar).
2
+ See checkpoints/README.md for the expected filename and training copy commands.
checkpoints/ctaic/s2_sem_panoptic/stage2/1/PLACEHOLDER ADDED
@@ -0,0 +1,2 @@
 
 
 
1
+ PLACEHOLDER: replace this file with the real checkpoint (.pth.tar).
2
+ See checkpoints/README.md for the expected filename and training copy commands.
checkpoints/ctaic/s2_sem_panoptic/stage2/2/PLACEHOLDER ADDED
@@ -0,0 +1,2 @@
 
 
 
1
+ PLACEHOLDER: replace this file with the real checkpoint (.pth.tar).
2
+ See checkpoints/README.md for the expected filename and training copy commands.
checkpoints/ctaic/s2_sem_panoptic/stage2/3/PLACEHOLDER ADDED
@@ -0,0 +1,2 @@
 
 
 
1
+ PLACEHOLDER: replace this file with the real checkpoint (.pth.tar).
2
+ See checkpoints/README.md for the expected filename and training copy commands.
checkpoints/ctaic/s2_sem_panoptic/stage2/4/PLACEHOLDER ADDED
@@ -0,0 +1,2 @@
 
 
 
1
+ PLACEHOLDER: replace this file with the real checkpoint (.pth.tar).
2
+ See checkpoints/README.md for the expected filename and training copy commands.
checkpoints/ctaic/s3_det_pose/stage1/1/PLACEHOLDER ADDED
@@ -0,0 +1,2 @@
 
 
 
1
+ PLACEHOLDER: replace this file with the real checkpoint (.pth.tar).
2
+ See checkpoints/README.md for the expected filename and training copy commands.
checkpoints/ctaic/s3_det_pose/stage1/2/PLACEHOLDER ADDED
@@ -0,0 +1,2 @@
 
 
 
1
+ PLACEHOLDER: replace this file with the real checkpoint (.pth.tar).
2
+ See checkpoints/README.md for the expected filename and training copy commands.
checkpoints/ctaic/s3_det_pose/stage1/3/PLACEHOLDER ADDED
@@ -0,0 +1,2 @@
 
 
 
1
+ PLACEHOLDER: replace this file with the real checkpoint (.pth.tar).
2
+ See checkpoints/README.md for the expected filename and training copy commands.
checkpoints/ctaic/s3_det_pose/stage1/4/PLACEHOLDER ADDED
@@ -0,0 +1,2 @@
 
 
 
1
+ PLACEHOLDER: replace this file with the real checkpoint (.pth.tar).
2
+ See checkpoints/README.md for the expected filename and training copy commands.
checkpoints/ctaic/s3_det_pose/stage2/1/PLACEHOLDER ADDED
@@ -0,0 +1,2 @@
 
 
 
1
+ PLACEHOLDER: replace this file with the real checkpoint (.pth.tar).
2
+ See checkpoints/README.md for the expected filename and training copy commands.
checkpoints/ctaic/s3_det_pose/stage2/2/PLACEHOLDER ADDED
@@ -0,0 +1,2 @@
 
 
 
1
+ PLACEHOLDER: replace this file with the real checkpoint (.pth.tar).
2
+ See checkpoints/README.md for the expected filename and training copy commands.
checkpoints/ctaic/s3_det_pose/stage2/3/PLACEHOLDER ADDED
@@ -0,0 +1,2 @@
 
 
 
1
+ PLACEHOLDER: replace this file with the real checkpoint (.pth.tar).
2
+ See checkpoints/README.md for the expected filename and training copy commands.
checkpoints/ctaic/s3_det_pose/stage2/4/PLACEHOLDER ADDED
@@ -0,0 +1,2 @@
 
 
 
1
+ PLACEHOLDER: replace this file with the real checkpoint (.pth.tar).
2
+ See checkpoints/README.md for the expected filename and training copy commands.
checkpoints/taic/detection/1/PLACEHOLDER ADDED
@@ -0,0 +1,2 @@
 
 
 
1
+ PLACEHOLDER: replace this file with the real checkpoint (.pth.tar).
2
+ See checkpoints/README.md for the expected filename and training copy commands.
checkpoints/taic/detection/2/PLACEHOLDER ADDED
@@ -0,0 +1,2 @@
 
 
 
1
+ PLACEHOLDER: replace this file with the real checkpoint (.pth.tar).
2
+ See checkpoints/README.md for the expected filename and training copy commands.
checkpoints/taic/detection/3/PLACEHOLDER ADDED
@@ -0,0 +1,2 @@
 
 
 
1
+ PLACEHOLDER: replace this file with the real checkpoint (.pth.tar).
2
+ See checkpoints/README.md for the expected filename and training copy commands.
checkpoints/taic/detection/4/PLACEHOLDER ADDED
@@ -0,0 +1,2 @@
 
 
 
1
+ PLACEHOLDER: replace this file with the real checkpoint (.pth.tar).
2
+ See checkpoints/README.md for the expected filename and training copy commands.
checkpoints/taic/instance/1/PLACEHOLDER ADDED
@@ -0,0 +1,2 @@
 
 
 
1
+ PLACEHOLDER: replace this file with the real checkpoint (.pth.tar).
2
+ See checkpoints/README.md for the expected filename and training copy commands.
checkpoints/taic/instance/2/PLACEHOLDER ADDED
@@ -0,0 +1,2 @@
 
 
 
1
+ PLACEHOLDER: replace this file with the real checkpoint (.pth.tar).
2
+ See checkpoints/README.md for the expected filename and training copy commands.
checkpoints/taic/instance/3/PLACEHOLDER ADDED
@@ -0,0 +1,2 @@
 
 
 
1
+ PLACEHOLDER: replace this file with the real checkpoint (.pth.tar).
2
+ See checkpoints/README.md for the expected filename and training copy commands.
checkpoints/taic/instance/4/PLACEHOLDER ADDED
@@ -0,0 +1,2 @@
 
 
 
1
+ PLACEHOLDER: replace this file with the real checkpoint (.pth.tar).
2
+ See checkpoints/README.md for the expected filename and training copy commands.
checkpoints/taic/panoptic/1/PLACEHOLDER ADDED
@@ -0,0 +1,2 @@
 
 
 
1
+ PLACEHOLDER: replace this file with the real checkpoint (.pth.tar).
2
+ See checkpoints/README.md for the expected filename and training copy commands.
checkpoints/taic/panoptic/2/PLACEHOLDER ADDED
@@ -0,0 +1,2 @@
 
 
 
1
+ PLACEHOLDER: replace this file with the real checkpoint (.pth.tar).
2
+ See checkpoints/README.md for the expected filename and training copy commands.
checkpoints/taic/panoptic/3/PLACEHOLDER ADDED
@@ -0,0 +1,2 @@
 
 
 
1
+ PLACEHOLDER: replace this file with the real checkpoint (.pth.tar).
2
+ See checkpoints/README.md for the expected filename and training copy commands.
checkpoints/taic/panoptic/4/PLACEHOLDER ADDED
@@ -0,0 +1,2 @@
 
 
 
1
+ PLACEHOLDER: replace this file with the real checkpoint (.pth.tar).
2
+ See checkpoints/README.md for the expected filename and training copy commands.
checkpoints/taic/pose/1/PLACEHOLDER ADDED
@@ -0,0 +1,2 @@
 
 
 
1
+ PLACEHOLDER: replace this file with the real checkpoint (.pth.tar).
2
+ See checkpoints/README.md for the expected filename and training copy commands.
checkpoints/taic/pose/2/PLACEHOLDER ADDED
@@ -0,0 +1,2 @@
 
 
 
1
+ PLACEHOLDER: replace this file with the real checkpoint (.pth.tar).
2
+ See checkpoints/README.md for the expected filename and training copy commands.
checkpoints/taic/pose/3/PLACEHOLDER ADDED
@@ -0,0 +1,2 @@
 
 
 
1
+ PLACEHOLDER: replace this file with the real checkpoint (.pth.tar).
2
+ See checkpoints/README.md for the expected filename and training copy commands.
checkpoints/taic/pose/4/PLACEHOLDER ADDED
@@ -0,0 +1,2 @@
 
 
 
1
+ PLACEHOLDER: replace this file with the real checkpoint (.pth.tar).
2
+ See checkpoints/README.md for the expected filename and training copy commands.
checkpoints/taic/semantic/1/PLACEHOLDER ADDED
@@ -0,0 +1,2 @@
 
 
 
1
+ PLACEHOLDER: replace this file with the real checkpoint (.pth.tar).
2
+ See checkpoints/README.md for the expected filename and training copy commands.
checkpoints/taic/semantic/2/PLACEHOLDER ADDED
@@ -0,0 +1,2 @@
 
 
 
1
+ PLACEHOLDER: replace this file with the real checkpoint (.pth.tar).
2
+ See checkpoints/README.md for the expected filename and training copy commands.