Instructions to use apple/DepthPro with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Depth Pro
How to use apple/DepthPro with Depth Pro:
# Download checkpoint pip install huggingface-hub huggingface-cli download --local-dir checkpoints apple/DepthPro
import depth_pro # Load model and preprocessing transform model, transform = depth_pro.create_model_and_transforms() model.eval() # Load and preprocess an image. image, _, f_px = depth_pro.load_rgb("example.png") image = transform(image) # Run inference. prediction = model.infer(image, f_px=f_px) # Results: 1. Depth in meters depth = prediction["depth"] # Results: 2. Focal length in pixels focallength_px = prediction["focallength_px"] - Notebooks
- Google Colab
- Kaggle
Cross-reference arXiv paper in README.md
#2
by amael-apple - opened
README.md
CHANGED
|
@@ -8,7 +8,7 @@ license: apple-ascl
|
|
| 8 |
|
| 9 |
We present a foundation model for zero-shot metric monocular depth estimation. Our model, Depth Pro, synthesizes high-resolution depth maps with unparalleled sharpness and high-frequency details. The predictions are metric, with absolute scale, without relying on the availability of metadata such as camera intrinsics. And the model is fast, producing a 2.25-megapixel depth map in 0.3 seconds on a standard GPU. These characteristics are enabled by a number of technical contributions, including an efficient multi-scale vision transformer for dense prediction, a training protocol that combines real and synthetic datasets to achieve high metric accuracy alongside fine boundary tracing, dedicated evaluation metrics for boundary accuracy in estimated depth maps, and state-of-the-art focal length estimation from a single image.
|
| 10 |
|
| 11 |
-
Depth Pro was introduced in **Depth Pro: Sharp Monocular Metric Depth in Less Than a Second**, by *Aleksei Bochkovskii, Amaël Delaunoy, Hugo Germain, Marcel Santos, Yichao Zhou, Stephan R. Richter, and Vladlen Koltun*.
|
| 12 |
|
| 13 |
The checkpoint in this repository is a reference implementation, which has been re-trained. Its performance is close to the model reported in the paper but does not match it exactly.
|
| 14 |
|
|
|
|
| 8 |
|
| 9 |
We present a foundation model for zero-shot metric monocular depth estimation. Our model, Depth Pro, synthesizes high-resolution depth maps with unparalleled sharpness and high-frequency details. The predictions are metric, with absolute scale, without relying on the availability of metadata such as camera intrinsics. And the model is fast, producing a 2.25-megapixel depth map in 0.3 seconds on a standard GPU. These characteristics are enabled by a number of technical contributions, including an efficient multi-scale vision transformer for dense prediction, a training protocol that combines real and synthetic datasets to achieve high metric accuracy alongside fine boundary tracing, dedicated evaluation metrics for boundary accuracy in estimated depth maps, and state-of-the-art focal length estimation from a single image.
|
| 10 |
|
| 11 |
+
Depth Pro was introduced in **[Depth Pro: Sharp Monocular Metric Depth in Less Than a Second](https://arxiv.org/abs/2410.02073)**, by *Aleksei Bochkovskii, Amaël Delaunoy, Hugo Germain, Marcel Santos, Yichao Zhou, Stephan R. Richter, and Vladlen Koltun*.
|
| 12 |
|
| 13 |
The checkpoint in this repository is a reference implementation, which has been re-trained. Its performance is close to the model reported in the paper but does not match it exactly.
|
| 14 |
|