|
|
--- |
|
|
license: other |
|
|
license_name: fair-noncommercial-research |
|
|
extra_gated_prompt: > |
|
|
FAIR Noncommercial Research License v1 Last Updated: August 18, 2025 |
|
|
|
|
|
“Acceptable Use Policy” means the FAIR Acceptable Use Policy, applicable to |
|
|
Research Materials, that is incorporated into this Agreement. |
|
|
|
|
|
“Agreement” means the terms and conditions for use, reproduction, distribution |
|
|
and modification of the Research Materials set forth herein. |
|
|
|
|
|
|
|
|
“Documentation” means the specifications, manuals and documentation |
|
|
accompanying Research Materials distributed by Meta. |
|
|
|
|
|
|
|
|
“Licensee” or “you” means you, or your employer or any other person or entity |
|
|
(if you are entering into this Agreement on such person or entity’s behalf), |
|
|
of the age required under applicable laws, rules or regulations to provide |
|
|
legal consent and that has legal authority to bind your employer or such other |
|
|
person or entity if you are entering in this Agreement on their behalf. |
|
|
|
|
|
|
|
|
“Meta” or “we” means Meta Platforms Ireland Limited (if you are located in or, |
|
|
if you are an entity, your principal place of business is in the EEA or |
|
|
Switzerland) and Meta Platforms, Inc. (if you are located outside of the EEA |
|
|
or Switzerland). |
|
|
|
|
|
“Noncommercial Research Uses” means noncommercial research use cases related |
|
|
to research, development, education, processing, or analysis and in each case, |
|
|
is not primarily intended for commercial advantage or monetary compensation to |
|
|
you or others. |
|
|
|
|
|
“Research Materials” means, collectively, Documentation and the models, |
|
|
software and algorithms, including machine-learning model code, trained model |
|
|
weights, inference-enabling code, training-enabling code, fine-tuning enabling |
|
|
code, demonstration materials and other elements of the foregoing distributed |
|
|
by Meta and made available under this Agreement. |
|
|
|
|
|
By clicking “I Accept” below or by using or distributing any portion or |
|
|
element of the Research Materials, you agree to be bound by this Agreement. |
|
|
|
|
|
|
|
|
1. License Rights and Redistribution. |
|
|
|
|
|
|
|
|
a. Grant of Rights. You are granted a non-exclusive, worldwide, |
|
|
non-transferable and royalty-free limited license under Meta’s intellectual |
|
|
property or other rights owned by Meta embodied in the Research Materials to |
|
|
use, reproduce, distribute, copy, create derivative works of, and make |
|
|
modifications to the Research Materials. |
|
|
|
|
|
b. Redistribution and Use. i. You will not use the Research Materials or any |
|
|
outputs or results of the Research Materials in connection with any commercial |
|
|
uses or for any uses other than Noncommercial Research Uses; |
|
|
|
|
|
|
|
|
ii. Distribution of Research Materials, and any derivative works thereof, are |
|
|
subject to the terms of this Agreement. If you distribute or make the Research |
|
|
Materials, or any derivative works thereof, available to a third party, you |
|
|
may only do so under the terms of this Agreement. You shall also provide a |
|
|
copy of this Agreement to such third party. |
|
|
|
|
|
|
|
|
iii. If you submit for publication the results of research you perform on, |
|
|
using, or otherwise in connection with Research Materials, you must |
|
|
acknowledge the use of Research Materials in your publication. |
|
|
|
|
|
|
|
|
iv. Your use of the Research Materials must comply with applicable laws and |
|
|
regulations (including Trade Control Laws) and adhere to the FAIR Acceptable |
|
|
Use Policy, which is hereby incorporated by reference into this Agreement. 2. |
|
|
User Support. Your Noncommercial Research Use of the Research Materials is |
|
|
done at your own discretion; Meta does not process any information nor provide |
|
|
any service in relation to such use. Meta is under no obligation to provide |
|
|
any support services for the Research Materials. Any support provided is “as |
|
|
is”, “with all faults”, and without warranty of any kind. |
|
|
|
|
|
|
|
|
3. Disclaimer of Warranty. UNLESS REQUIRED BY APPLICABLE LAW, THE RESEARCH |
|
|
MATERIALS AND ANY OUTPUT AND RESULTS THEREFROM ARE PROVIDED ON AN “AS IS” |
|
|
BASIS, WITHOUT WARRANTIES OF ANY KIND, AND META DISCLAIMS ALL WARRANTIES OF |
|
|
ANY KIND, BOTH EXPRESS AND IMPLIED, INCLUDING, WITHOUT LIMITATION, ANY |
|
|
WARRANTIES OF TITLE, NON-INFRINGEMENT, MERCHANTABILITY, OR FITNESS FOR A |
|
|
PARTICULAR PURPOSE. YOU ARE SOLELY RESPONSIBLE FOR DETERMINING THE |
|
|
APPROPRIATENESS OF USING OR REDISTRIBUTING THE RESEARCH MATERIALS AND ASSUME |
|
|
ANY RISKS ASSOCIATED WITH YOUR USE OF THE RESEARCH MATERIALS AND ANY OUTPUT |
|
|
AND RESULTS. |
|
|
|
|
|
4. Limitation of Liability. IN NO EVENT WILL META OR ITS AFFILIATES BE LIABLE |
|
|
UNDER ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, TORT, NEGLIGENCE, PRODUCTS |
|
|
LIABILITY, OR OTHERWISE, ARISING OUT OF THIS AGREEMENT, FOR ANY LOST PROFITS |
|
|
OR ANY DIRECT OR INDIRECT, SPECIAL, CONSEQUENTIAL, INCIDENTAL, EXEMPLARY OR |
|
|
PUNITIVE DAMAGES, EVEN IF META OR ITS AFFILIATES HAVE BEEN ADVISED OF THE |
|
|
POSSIBILITY OF ANY OF THE FOREGOING. |
|
|
|
|
|
5. Intellectual Property. |
|
|
|
|
|
|
|
|
a. Subject to Meta’s ownership of Research Materials and derivatives made by |
|
|
or for Meta, with respect to any derivative works and modifications of the |
|
|
Research Materials that are made by you, as between you and Meta, you are and |
|
|
will be the owner of such derivative works and modifications. |
|
|
|
|
|
b. If you institute litigation or other proceedings against Meta or any entity |
|
|
(including a cross-claim or counterclaim in a lawsuit) alleging that the |
|
|
Research Materials, outputs or results, or any portion of any of the |
|
|
foregoing, constitutes infringement of intellectual property or other rights |
|
|
owned or licensable by you, then any licenses granted to you under this |
|
|
Agreement shall terminate as of the date such litigation or claim is filed or |
|
|
instituted. You will indemnify and hold harmless Meta from and against any |
|
|
claim by any third party arising out of or related to your use or distribution |
|
|
of the Research Materials. |
|
|
|
|
|
6. Term and Termination. The term of this Agreement will commence upon your |
|
|
acceptance of this Agreement or access to the Research Materials and will |
|
|
continue in full force and effect until terminated in accordance with the |
|
|
terms and conditions herein. Meta may terminate this Agreement if you are in |
|
|
breach of any term or condition of this Agreement. Upon termination of this |
|
|
Agreement, you shall delete and cease use of the Research Materials. Sections |
|
|
3, 4 and 7 shall survive the termination of this Agreement. |
|
|
|
|
|
7. Governing Law and Jurisdiction. This Agreement will be governed and |
|
|
construed under the laws of the State of California without regard to choice |
|
|
of law principles, and the UN Convention on Contracts for the International |
|
|
Sale of Goods does not apply to this Agreement. The courts of California shall |
|
|
have exclusive jurisdiction of any dispute arising out of this Agreement. |
|
|
|
|
|
|
|
|
8. Modifications and Amendments. Meta may modify this Agreement from time to |
|
|
time; provided that they are similar in spirit to the current version of the |
|
|
Agreement, but may differ in detail to address new problems or concerns. All |
|
|
such changes will be effective immediately. Your continued use of the Research |
|
|
Materials after any modification to this Agreement constitutes your agreement |
|
|
to such modification. Except as provided in this Agreement, no modification or |
|
|
addition to any provision of this Agreement will be binding unless it is in |
|
|
writing and signed by an authorized representative of both you and Meta. |
|
|
|
|
|
|
|
|
FAIR Acceptable Use Policy |
|
|
|
|
|
The Fundamental AI Research (FAIR) team at Meta seeks to further understanding |
|
|
of new and existing research domains with the mission of advancing the |
|
|
state-of-the-art in artificial intelligence through open research for the |
|
|
benefit of all. |
|
|
|
|
|
As part of this mission, Meta makes certain research materials available for |
|
|
noncommercial research use. Meta is committed to promoting the safe and |
|
|
responsible use of such research materials. |
|
|
|
|
|
Prohibited Uses |
|
|
|
|
|
You agree you will not use, or allow others to use, Research Materials to: |
|
|
|
|
|
Violate the law or others’ rights, including to: Engage in, promote, generate, |
|
|
contribute to, encourage, plan, incite, or further illegal or unlawful |
|
|
activity or content, such as: Violence or terrorism Exploitation or harm to |
|
|
children, including the solicitation, creation, acquisition, or dissemination |
|
|
of child exploitative content or failure to report Child Sexual Abuse Material |
|
|
Human trafficking, exploitation, and sexual violence The illegal distribution |
|
|
of information or materials to minors, including obscene materials, or failure |
|
|
to employ legally required age-gating in connection with such information or |
|
|
materials. Sexual solicitation Any other criminal activity |
|
|
|
|
|
Engage in, promote, incite, or facilitate the harassment, abuse, threatening, |
|
|
or bullying of individuals or groups of individuals |
|
|
|
|
|
Engage in, promote, incite, or facilitate discrimination or other unlawful or |
|
|
harmful conduct in the provision of employment, employment benefits, credit, |
|
|
housing, other economic benefits, or other essential goods and services |
|
|
|
|
|
Engage in the unauthorized or unlicensed practice of any profession including, |
|
|
but not limited to, financial, legal, medical/health, or related professional |
|
|
practices |
|
|
|
|
|
Collect, process, disclose, generate, or infer health, demographic, or other |
|
|
sensitive personal or private information about individuals without rights and |
|
|
consents required by applicable laws |
|
|
|
|
|
Engage in or facilitate any action or generate any content that infringes, |
|
|
misappropriates, or otherwise violates any third-party rights, including the |
|
|
outputs or results of any technology using FAIR research materials |
|
|
|
|
|
Create, generate, or facilitate the creation of malicious code, malware, |
|
|
computer viruses or do anything else that could disable, overburden, interfere |
|
|
with or impair the proper working, integrity, operation or appearance of a |
|
|
website or computer system |
|
|
|
|
|
2. Engage in, promote, incite, facilitate, or assist in the planning or |
|
|
development of activities that present a risk of death or bodily harm to |
|
|
individuals, including use of research artifacts related to the following: |
|
|
|
|
|
Military, warfare, nuclear industries or applications, espionage, use for |
|
|
materials or activities that are subject to the International Traffic Arms |
|
|
Regulations (ITAR) maintained by the United States Department of State |
|
|
|
|
|
Guns and illegal weapons (including weapon development) |
|
|
|
|
|
Illegal drugs and regulated/controlled substances |
|
|
|
|
|
Operation of critical infrastructure, transportation technologies, or heavy |
|
|
machinery |
|
|
|
|
|
Self-harm or harm to others, including suicide, cutting, and eating disorders |
|
|
|
|
|
Any content intended to incite or promote violence, abuse, or any infliction |
|
|
of bodily harm to an individual |
|
|
|
|
|
3. Intentionally deceive or mislead others, including use of FAIR Research |
|
|
Materials related to the following: |
|
|
|
|
|
Generating, promoting, or furthering fraud or the creation or promotion of |
|
|
disinformation |
|
|
|
|
|
Generating, promoting, or furthering defamatory content, including the |
|
|
creation of defamatory statements, images, or other content |
|
|
|
|
|
Generating, promoting, or further distributing spam |
|
|
|
|
|
Impersonating another individual without consent, authorization, or legal |
|
|
right |
|
|
|
|
|
Representing that outputs of FAIR research materials or outputs from |
|
|
technology using FAIR research materials are human-generated |
|
|
|
|
|
Generating or facilitating false online engagement, including fake reviews and |
|
|
other means of fake online engagement |
|
|
|
|
|
4. Fail to appropriately disclose to end users any known dangers of your |
|
|
Research Materials. |
|
|
|
|
|
Please report any violation of this Policy or other problems that could lead |
|
|
to a violation of this Policy by submitting a report here |
|
|
[https://docs.google.com/forms/d/e/1FAIpQLSeb11cryAopJ7LNrC4nxEUXrHY26hfkXQMf_uH-oFgA3WlYZQ/viewform]. |
|
|
extra_gated_fields: |
|
|
First Name: text |
|
|
Last Name: text |
|
|
Date of birth: date_picker |
|
|
Country: country |
|
|
Affiliation: text |
|
|
Job title: |
|
|
type: select |
|
|
options: |
|
|
- Student |
|
|
- Research Graduate |
|
|
- AI researcher |
|
|
- AI developer/engineer |
|
|
- Reporter |
|
|
- Other |
|
|
geo: ip_location |
|
|
By clicking Submit below I accept the terms of the license and acknowledge that the information I provide will be collected stored processed and shared in accordance with the Meta Privacy Policy: checkbox |
|
|
extra_gated_description: >- |
|
|
The information you provide will be collected, stored, processed and shared in |
|
|
accordance with the [Meta Privacy |
|
|
Policy](https://www.facebook.com/privacy/policy/). |
|
|
extra_gated_button_content: Submit |
|
|
extra_gated_heading: >- |
|
|
Please be sure to provide your full legal name, date of birth, and full |
|
|
organization name with all corporate identifiers. Avoid the use of acronyms |
|
|
and special characters. Failure to follow these instructions may prevent you |
|
|
from accessing this model and others on Hugging Face. You will not have the |
|
|
ability to edit this form after submission, so please ensure all information |
|
|
is accurate. |
|
|
language: |
|
|
- en |
|
|
tags: |
|
|
- <relevant tags to be included in HF filters> |
|
|
- facebook |
|
|
- meta |
|
|
- pytorch |
|
|
- DepthLM |
|
|
base_model: |
|
|
- facebook/DepthLM |
|
|
library_name: transformers |
|
|
--- |
|
|
|
|
|
# Model Details |
|
|
|
|
|
Official model card of "[DepthLM: Metric Depth from Vision Language Models](https://arxiv.org/pdf/2509.25413)". See our [github](https://github.com/facebookresearch/DepthLM_Official) for the eval and training code. |
|
|
|
|
|
 |
|
|
|
|
|
 |
|
|
|
|
|
This model card includes the 12b model of DepthLM finetuned from [Pixtral](https://huggingface.co/mistralai/Pixtral-12B-2409). |
|
|
|
|
|
We show for the first time that VLMs can achieve comparable accuracy with pure vision models on metric depth estimation, with standard text-based SFT and no architecture chagne, i.e., no dense prediction head or regression/regularization loss is needed. Due to the simplicity, we can use DepthLM to train a unified VLM to handle various complex 3D understanding tasks such as speed or time estimation, and metric scale camera pose estimation, which require different architecture or hand-crafted pipelines in pure vision models. |
|
|
|
|
|
## Citation |
|
|
|
|
|
If you find our code useful for your research, please consider citing: |
|
|
|
|
|
@article{cai2025depthlm, |
|
|
title={DepthLM: Metric Depth from Vision Language Models}, |
|
|
author={Cai, Zhipeng and Yeh, Ching-Feng and Hu, Xu and Liu, Zhuang and Meyer, Gregory and Lei, Xinjie and Zhao, Changsheng and Li, Shang-Wen and Chandra, Vikas and Shi, Yangyang}, |
|
|
journal={arXiv preprint arXiv:2509.25413}, |
|
|
year={2025}, |
|
|
} |
|
|
|
|
|
## Contact |
|
|
Zhipeng Cai, Meta Inc, homepage: https://zhipengcai.github.io/, email: czptc2h at gmail dot com. |
|
|
|
|
|
## Results |
|
|
|
|
|
### Comparison with VLMs |
|
|
|
|
|
| Accuracy ($\delta_1$) | Argoverse2 | DDAD | NuScenes | ETH3D | ScanNet++ | sunRGBD | iBims1 | NYUv2 | avg. | |
|
|
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | |
|
|
| Qwen2.5-VL (3B) | 0.133 | 0.083 | 0.090 | 0.087 | 0.120 | 0.134 | 0.080 | 0.128 | 0.106 | |
|
|
| Qwen2.5-VL (7B) | 0.077 | 0.120 | 0.070 | 0.126 | 0.135 | 0.089 | 0.160 | 0.168 | 0.118 | |
|
|
| Qwen2.5-VL (72B) | 0.119 | 0.140 | 0.186 | 0.220 | 0.272 | 0.276 | 0.212 | 0.324 | 0.219 | |
|
|
| Seed1.5-VL | 0.009 | 0.012 | 0.013 | 0.219 | 0.495 | 0.321 | 0.459 | 0.412 | 0.243 | |
|
|
| Gemini-2.5-PRO | 0.280 | 0.252 | 0.365 | 0.328 | 0.380 | 0.270 | 0.466 | 0.394 | 0.342 | |
|
|
| GPT-5 | 0.218 | 0.302 | 0.382 | 0.313 | 0.428 | 0.471 | 0.307 | 0.540 | 0.370 | |
|
|
| **Ours (3B)** | 0.808 | 0.724 | **0.870** | **0.745** | 0.838 | 0.850 | 0.890 | 0.868 | 0.824 | |
|
|
| **Ours (7B)** | **0.833** | **0.747** | 0.865 | 0.718 | **0.850** | **0.859** | **0.920** | **0.915** | **0.838** | |
|
|
| **Ours - Pixtral (12B)** | 0.734 | 0.670 | 0.819 | 0.653 | 0.834 | 0.786 | 0.870 | 0.799 | 0.771 | |
|
|
|
|
|
|
|
|
### Comparison with pure vision models |
|
|
|
|
|
| Accuracy ($\delta_1$) | DDAD | NuScenes | ETH3D | sunRGBD | iBims1 | vs Ours | |
|
|
| --- | --- | --- | --- | --- | --- | --- | |
|
|
| ZoeDepth | 0.272 | 0.283 | 0.350 | 0.867 | 0.580 | -42.8% | |
|
|
| DepthAnything | - | 0.354 | 0.093 | 0.850 | 0.714 | -40.3% | |
|
|
| DepthAnythingV2 | - | 0.171 | 0.363 | 0.724 | - | -48.5% | |
|
|
| Metric3D | - | 0.723 | 0.456 | 0.154 | 0.797 | -36.6% | |
|
|
| Unidepth | 0.858 | 0.846 | 0.185 | 0.943 | 0.157 | -27.3% | |
|
|
| Depth Pro | 0.299 | 0.566 | 0.397 | 0.831 | 0.823 | -29.1% | |
|
|
| Metric3Dv2 | - | 0.841 | 0.900 | 0.812 | 0.684 | -3.8% | |
|
|
| UnidepthV2 | 0.882 | 0.870 | 0.852 | 0.964 | 0.945 | +9.2% | |
|
|
| **Ours (7B)** | 0.747 | 0.865 | 0.718 | 0.859 | 0.920 | - | |
|
|
|
|
|
# License |
|
|
|
|
|
DepthLM is [FAIR NC licensed](https://huggingface.co/facebook/DepthLM/blob/main/LICENSE) as of now |
