huangfeice commited on
Commit
cedb79a
·
verified ·
1 Parent(s): b538ab1

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +101 -0
README.md CHANGED
@@ -43,6 +43,107 @@ hf download jdopensource/JoyAI-Image-Edit-ComfyUI \
43
 
44
  Workflow: [workflow_joyimage_edit.json](workflow_joyimage_edit.json)
45
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
46
 
47
  ## Links
48
 
 
43
 
44
  Workflow: [workflow_joyimage_edit.json](workflow_joyimage_edit.json)
45
 
46
+ ## Spatial Editing Reference
47
+
48
+ JoyAI-Image supports three spatial editing prompt patterns: **Object Move**, **Object Rotation**, and **Camera Control**. For the most stable behavior, we recommend following the prompt templates below as closely as possible.
49
+ For more information (including data curation and evaluation strategies), please refer to [SpatialEdit](https://github.com/EasonXiao-888/SpatialEdit).
50
+
51
+ ### Object Move
52
+
53
+ Use this pattern when you want to move a target object into a specified region.
54
+
55
+ **Prompt template:**
56
+
57
+ ```text
58
+ Move the <object> into the red box and finally remove the red box.
59
+ ```
60
+
61
+ **Rules:**
62
+
63
+ - Replace `<object>` with a clear description of the target object to be moved.
64
+ - The **red box** indicates the target destination in the image.
65
+ - The phrase **"finally remove the red box"** means the guidance box should not appear in the final edited result.
66
+
67
+ **Example:**
68
+
69
+ ```text
70
+ Move the apple into the red box and finally remove the red box.
71
+ ```
72
+
73
+ ### Object Rotation
74
+
75
+ Use this pattern when you want to rotate an object to a specific canonical view.
76
+
77
+ **Prompt template:**
78
+
79
+ ```text
80
+ Rotate the <object> to show the <view> side view.
81
+ ```
82
+
83
+ **Supported `<view>` values:**
84
+
85
+ ```
86
+ front, right, left, rear, front right, front left, rear right, rear left
87
+ ```
88
+
89
+ **Rules:**
90
+
91
+ - Replace `<object>` with a clear description of the object to rotate.
92
+ - Replace `<view>` with one of the supported directions above.
93
+ - This instruction is intended to change the **object orientation**, while keeping the object identity and surrounding scene as consistent as possible.
94
+
95
+ **Examples:**
96
+
97
+ ```text
98
+ Rotate the chair to show the front side view.
99
+ Rotate the car to show the rear left side view.
100
+ ```
101
+
102
+ ### Camera Control
103
+
104
+ Use this pattern when you want to change only the camera viewpoint while keeping the 3D scene itself unchanged.
105
+
106
+ **Prompt template:**
107
+
108
+ ```text
109
+ Move the camera.
110
+ - Camera rotation: Yaw {y_rotation}°, Pitch {p_rotation}°.
111
+ - Camera zoom: in/out/unchanged.
112
+ - Keep the 3D scene static; only change the viewpoint.
113
+ ```
114
+
115
+ **Rules:**
116
+
117
+ - `{y_rotation}` specifies the yaw rotation angle in degrees.
118
+ - `{p_rotation}` specifies the pitch rotation angle in degrees.
119
+ - `Camera zoom` must be one of: `in`, `out`,`unchanged`
120
+ - The last line is important: it explicitly tells the model to preserve the 3D scene content and geometry, and only adjust the camera viewpoint.
121
+
122
+ **Examples:**
123
+
124
+ ```text
125
+ Move the camera.
126
+ - Camera rotation: Yaw 45°, Pitch 0°.
127
+ - Camera zoom: in.
128
+ - Keep the 3D scene static; only change the viewpoint.
129
+ ```
130
+
131
+ ```text
132
+ Move the camera.
133
+ - Camera rotation: Yaw -90°, Pitch 20°.
134
+ - Camera zoom: unchanged.
135
+ - Keep the 3D scene static; only change the viewpoint.
136
+ ```
137
+
138
+ ### Application
139
+
140
+ **3D Reconstruction:**
141
+
142
+ The first and third examples show point clouds with only a single given viewpoint. The second and fourth examples are augmented by [SpatialEdit](https://github.com/EasonXiao-888/SpatialEdit), which synthesizes richer spatial observations from the sparse input view.
143
+
144
+ **Conditional-frames Based Video Generation:**
145
+
146
+ Given the first frame, [SpatialEdit](https://github.com/EasonXiao-888/SpatialEdit) first generates the final frame of the video, and a video generation model then creates a smooth rotational transition between them while maintaining background consistency.
147
 
148
  ## Links
149