Phitran21 commited on
Commit
4d6332d
·
verified ·
1 Parent(s): ff5af4a

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +488 -1
README.md CHANGED
@@ -1,3 +1,490 @@
1
  ---
2
- license: apache-2.0
 
 
 
 
 
 
 
 
 
 
 
 
3
  ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  ---
2
+ library_name: onnx
3
+ tags:
4
+ - computer-vision
5
+ - image-enhancement
6
+ - photo-retouching
7
+ - computational-photography
8
+ - color-grading
9
+ - onnx
10
+ - mobile
11
+ - lightweight
12
+ - multi-task-learning
13
+ datasets:
14
+ - Phitran21/adaptive-photo-retouching-6style
15
  ---
16
+
17
+ # AdaptivePhotoNet
18
+
19
+ **AdaptivePhotoNet** is a lightweight, scene-aware neural network for
20
+ automatic photo retouching.
21
+
22
+ Instead of generating a new image pixel-by-pixel, the model analyzes a
23
+ low-resolution preview of the photograph and predicts a compact
24
+ **21-dimensional retouching recipe** that can be applied to the original
25
+ full-resolution image by a deterministic image-processing pipeline.
26
+
27
+ The model also predicts **10 scene attributes** to provide auxiliary
28
+ scene understanding.
29
+
30
+ AdaptivePhotoNet contains **5,876,943 parameters** and supports six
31
+ retouching styles:
32
+
33
+ - Natural
34
+ - Vivid
35
+ - Cinema
36
+ - Portrait
37
+ - Film
38
+ - Moody
39
+
40
+ The model is exported to **ONNX** and designed with lightweight desktop,
41
+ mobile, and edge inference in mind.
42
+
43
+ ---
44
+
45
+ ## Model Concept
46
+
47
+ AdaptivePhotoNet separates **visual understanding** from
48
+ **full-resolution image processing**.
49
+
50
+ ```text
51
+ ┌─────────────────┐
52
+ │ Original Image │
53
+ └────────┬────────┘
54
+
55
+ resize / preview
56
+
57
+
58
+ ┌──────────────────┐
59
+ │ RGB 224 × 224 │
60
+ └────────┬─────────┘
61
+
62
+ ┌───────────────┴───────────────┐
63
+ │ │
64
+ Image Features Style ID
65
+ │ 0 ... 5
66
+ └───────────────┬───────────────┘
67
+
68
+ ┌──────────────────┐
69
+ │ AdaptivePhotoNet │
70
+ │ 5.88M params │
71
+ └────────┬─────────┘
72
+
73
+ ┌──────────┴──────────┐
74
+ ▼ ▼
75
+ 21D Retouch Recipe 10 Scene Scores
76
+
77
+
78
+ Deterministic Retouching
79
+ Pipeline
80
+
81
+
82
+ Full-Resolution Output Image
83
+
84
+ The neural network therefore does not need to reconstruct the full-resolution photograph.
85
+
86
+ It predicts how the photograph should be adjusted, while the final rendering is performed by conventional image-processing operations.
87
+
88
+ This design has several practical advantages:
89
+
90
+ low neural-network inference cost;
91
+
92
+ processing is independent of the original image resolution at the model stage;
93
+
94
+ deterministic full-resolution rendering;
95
+
96
+ compact ONNX deployment;
97
+
98
+ interpretable adjustment parameters;
99
+
100
+ selectable photographic styles;
101
+
102
+ suitable for mobile and edge applications.
103
+
104
+
105
+
106
+ ---
107
+
108
+ Model Specifications
109
+
110
+ Property Value
111
+
112
+ Model AdaptivePhotoNet
113
+ Parameters 5,876,943
114
+ Model input resolution 224 × 224
115
+ Image format RGB
116
+ Tensor layout NCHW
117
+ Image dtype float32
118
+ Image range [0.0, 1.0]
119
+ Style input int64
120
+ Number of styles 6
121
+ Recipe output 21 dimensions
122
+ Scene output 10 dimensions
123
+ Runtime format ONNX
124
+
125
+
126
+
127
+ ---
128
+
129
+ Inputs
130
+
131
+ Image
132
+
133
+ name: image
134
+ shape: [1, 3, 224, 224]
135
+ dtype: float32
136
+ layout: NCHW
137
+ color: RGB
138
+ range: 0.0 - 1.0
139
+
140
+ The original photograph should be converted to RGB, resized to 224 × 224, converted to float32, normalized to [0, 1], and arranged in NCHW format.
141
+
142
+ The 224 × 224 image is used for analysis only.
143
+
144
+ The final retouching operations can be applied separately to the original full-resolution photograph.
145
+
146
+ Style
147
+
148
+ name: style_id
149
+ shape: [1]
150
+ dtype: int64
151
+
152
+ ID Style
153
+
154
+ 0 Natural
155
+ 1 Vivid
156
+ 2 Cinema
157
+ 3 Portrait
158
+ 4 Film
159
+ 5 Moody
160
+
161
+
162
+ Changing style_id instructs the same model to predict a different retouching direction for the input photograph.
163
+
164
+
165
+ ---
166
+
167
+ Outputs
168
+
169
+ AdaptivePhotoNet produces two outputs.
170
+
171
+ 1. Retouching Recipe
172
+
173
+ shape: [1, 21]
174
+
175
+ The 21-dimensional vector describes the photographic adjustments that should be applied by the retouching engine.
176
+
177
+ Linear Parameters
178
+
179
+ Dimensions 0–16 represent:
180
+
181
+ Dim Parameter Range
182
+
183
+ 0 Exposure EV -2.0 → 2.0
184
+ 1 Temperature -1.0 → 1.0
185
+ 2 Tint -1.0 → 1.0
186
+ 3 Shadows -1.0 → 1.0
187
+ 4 Highlights -1.0 → 1.0
188
+ 5 Contrast -1.0 → 1.0
189
+ 6 Tone Curve 0 0.0 → 1.0
190
+ 7 Tone Curve 1 0.0 → 1.0
191
+ 8 Tone Curve 2 0.0 → 1.0
192
+ 9 Tone Curve 3 0.0 → 1.0
193
+ 10 Tone Curve 4 0.0 → 1.0
194
+ 11 Shadow Tone Strength 0.0 → 0.3
195
+ 12 Highlight Tone Strength 0.0 → 0.3
196
+ 13 Saturation -1.0 → 1.0
197
+ 14 Vibrance -1.0 → 1.0
198
+ 15 Fade 0.0 → 1.0
199
+ 16 Vignette 0.0 → 1.0
200
+
201
+
202
+ Hue Parameters
203
+
204
+ Hue is represented circularly using sine/cosine pairs rather than a single scalar value.
205
+
206
+ 17, 18 → shadow_tone_hue [sin, cos]
207
+ 19, 20 → highlight_tone_hue [sin, cos]
208
+
209
+ This avoids the discontinuity that occurs when representing circular hue values directly near the angle boundary.
210
+
211
+ The complete interpretation is defined in:
212
+
213
+ recipe_schema.json
214
+
215
+
216
+ ---
217
+
218
+ 2. Scene Probabilities
219
+
220
+ shape: [1, 10]
221
+
222
+ The auxiliary scene head predicts ten visual attributes:
223
+
224
+ Index Scene Attribute
225
+
226
+ 0 Human
227
+ 1 Face Visible
228
+ 2 Skin Visible
229
+ 3 Portrait
230
+ 4 Indoor
231
+ 5 Outdoor
232
+ 6 Night
233
+ 7 Low Light
234
+ 8 Backlit
235
+ 9 High Dynamic Range
236
+
237
+
238
+ These attributes provide additional scene understanding alongside the retouching prediction.
239
+
240
+ They can also be useful for debugging, analysis, UI features, or future scene-aware processing logic.
241
+
242
+
243
+ ---
244
+
245
+ Why Predict a Recipe Instead of Pixels?
246
+
247
+ Many neural photo-enhancement systems directly generate a complete output image.
248
+
249
+ AdaptivePhotoNet takes a different approach.
250
+
251
+ Pixel-to-pixel model:
252
+
253
+ Full Image → Neural Network → Full Image
254
+
255
+
256
+ AdaptivePhotoNet:
257
+
258
+ Small Preview → Neural Network → 21 Parameters
259
+
260
+ Original Full-Resolution Image → Retouching Engine → Output
261
+
262
+ For photographic retouching, much of the desired transformation can be expressed through global or structured photographic controls.
263
+
264
+ Predicting these controls instead of millions of output pixels allows the neural network to remain relatively small.
265
+
266
+ It also keeps the transformation interpretable.
267
+
268
+ For example, an application can inspect whether the network requested:
269
+
270
+ Exposure +0.32 EV
271
+ Temperature -0.08
272
+ Highlights -0.21
273
+ Contrast +0.14
274
+ Saturation +0.07
275
+ Vignette 0.11
276
+ ...
277
+
278
+ rather than receiving only an opaque generated image.
279
+
280
+
281
+ ---
282
+
283
+ Multi-Style Retouching
284
+
285
+ AdaptivePhotoNet uses a separate style_id input rather than requiring six independent models.
286
+
287
+ The same photograph can therefore be analyzed under different retouching directions:
288
+
289
+ ┌─ Natural
290
+ ├─ Vivid
291
+ Input Photograph ───├─ Cinema
292
+ ├─ Portrait
293
+ ├─ Film
294
+ └─ Moody
295
+
296
+ The style determines the intended aesthetic direction while the image content determines the actual adjustment recipe.
297
+
298
+ This means that Film, for example, is not intended to represent one fixed preset applied identically to every photograph.
299
+
300
+ Two photographs using the same style may receive different exposure, tone, color, curve, and other adjustments according to their visual characteristics.
301
+
302
+
303
+ ---
304
+
305
+ Training Dataset
306
+
307
+ AdaptivePhotoNet was developed together with:
308
+
309
+ Adaptive Photo Retouching 6-Style Dataset
310
+
311
+ https://huggingface.co/datasets/Phitran21/adaptive-photo-retouching-6style
312
+
313
+ The dataset contains original photographs paired with six adaptively retouched variants:
314
+
315
+ Original
316
+ ├── Natural
317
+ ├── Vivid
318
+ ├── Cinema
319
+ ├── Portrait
320
+ ├── Film
321
+ └── Moody
322
+
323
+ The target transformations were generated adaptively for individual images rather than by applying six globally fixed presets.
324
+
325
+ See the dataset card for details about dataset generation, source data, licensing, and limitations.
326
+
327
+
328
+ ---
329
+
330
+ Inference Pipeline
331
+
332
+ A typical application pipeline is:
333
+
334
+ 1. Load the original image
335
+
336
+ 2. Create 224 × 224 RGB preview
337
+
338
+ 3. Normalize to float32 [0, 1]
339
+
340
+ 4. Convert HWC → NCHW
341
+
342
+ 5. Select style_id
343
+
344
+ 6. Run AdaptivePhotoNet
345
+
346
+ 7. Decode the 21D recipe
347
+
348
+ 8. Apply recipe to original-resolution image
349
+
350
+ 9. Produce final retouched photograph
351
+
352
+ The original full-resolution image does not need to pass through the neural network.
353
+
354
+
355
+ ---
356
+
357
+ Minimal ONNX Runtime Example
358
+
359
+ import numpy as np
360
+ import onnxruntime as ort
361
+ from PIL import Image
362
+
363
+ STYLE = {
364
+ "natural": 0,
365
+ "vivid": 1,
366
+ "cinema": 2,
367
+ "portrait": 3,
368
+ "film": 4,
369
+ "moody": 5,
370
+ }
371
+
372
+ image = Image.open("photo.jpg").convert("RGB")
373
+ preview = image.resize((224, 224))
374
+
375
+ x = np.asarray(preview, dtype=np.float32) / 255.0
376
+ x = np.transpose(x, (2, 0, 1))
377
+ x = np.expand_dims(x, axis=0)
378
+
379
+ style_id = np.asarray([STYLE["film"]], dtype=np.int64)
380
+
381
+ session = ort.InferenceSession("AdaptivePhotoNet.onnx")
382
+
383
+ recipe_vector, scene_probs = session.run(
384
+ None,
385
+ {
386
+ "image": x,
387
+ "style_id": style_id,
388
+ },
389
+ )
390
+
391
+ print("Recipe:", recipe_vector)
392
+ print("Scene probabilities:", scene_probs)
393
+
394
+ The resulting recipe_vector must then be interpreted according to recipe_schema.json and applied by the corresponding image-retouching pipeline.
395
+
396
+
397
+ ---
398
+
399
+ Android / ONNX Runtime
400
+
401
+ Recommended execution-provider configuration:
402
+
403
+ FP32
404
+
405
+ XNNPACKExecutionProvider
406
+ ↓ fallback
407
+ CPUExecutionProvider
408
+
409
+ INT8
410
+
411
+ CPUExecutionProvider
412
+
413
+ Actual performance depends on device hardware, ONNX Runtime version, thread configuration, quantization method, and preprocessing pipeline.
414
+
415
+
416
+ ---
417
+
418
+ Intended Use
419
+
420
+ AdaptivePhotoNet is intended for experimentation and development in:
421
+
422
+ automatic photo retouching;
423
+
424
+ computational photography;
425
+
426
+ adaptive color grading;
427
+
428
+ scene-aware image enhancement;
429
+
430
+ mobile photo editing;
431
+
432
+ lightweight computer vision;
433
+
434
+ ONNX Runtime applications;
435
+
436
+ edge inference;
437
+
438
+ non-destructive image adjustment prediction.
439
+
440
+
441
+
442
+ ---
443
+
444
+ Limitations
445
+
446
+ AdaptivePhotoNet predicts photographic adjustments from a 224 × 224 representation of the image.
447
+
448
+ Fine details that disappear during resizing may therefore not influence the predicted recipe.
449
+
450
+ The model may also perform less reliably on images significantly outside its training distribution, including unusual lighting, extreme exposure, uncommon photographic styles, or heavily degraded images.
451
+
452
+ Retouching quality is inherently subjective. Different users may prefer different photographic interpretations of the same image.
453
+
454
+ The six supported styles represent only six predefined aesthetic directions and should not be interpreted as exhaustive photographic styles.
455
+
456
+ The model predicts retouching parameters rather than reconstructing or generating image content. It therefore cannot perform tasks such as object removal, image inpainting, semantic image editing, or generative relighting.
457
+
458
+
459
+ ---
460
+
461
+ Related Resources
462
+
463
+ Training Dataset
464
+
465
+ Adaptive Photo Retouching 6-Style Dataset
466
+
467
+ https://huggingface.co/datasets/Phitran21/adaptive-photo-retouching-6style
468
+
469
+ Source Code and Demo
470
+
471
+ https://github.com/phiiggfdg/adaptive-retouch-6m-onnx
472
+
473
+
474
+ ---
475
+
476
+ Author
477
+
478
+ Trần Phi
479
+
480
+ Hugging Face:
481
+ https://huggingface.co/Phitran21
482
+
483
+ GitHub:
484
+ https://github.com/phiiggfdg
485
+
486
+ Website:
487
+ https://toren.io.vn
488
+
489
+
490
+ ---