VBench-2.0 : Advancing Video Generation
Benchmark Suite for Intrinsic Faithfulness

Mar 2025
Dian Zheng1,3*, Ziqi Huang2*, Hongbo Liu1, Kai Zou1, Yinan He1, Fan Zhang1,
Yuanhan Zhang2, Jingwen He1,4, Wei-Shi Zheng3†, Yu Qiao1†, Ziwei Liu2†
(* equal contributions, † corresponding authors)
1 Shanghai Artificial Intelligence Laboratory    2 S-Lab, Nanyang Technological University   
3 Sun Yat-Sen University    4 The Chinese University of Hong Kong   
Paper (VBench-2.0) Video GitHub
Teaser.

Overview of VBench-2.0. (a) Scope of VBench-2.0. Video generative models have progressed from achieving superficial faithfulness in fundamental technical aspects such as pixel fidelity and basic prompt adherence, to addressing more complex challenges associated with intrinsic faithfulness, including commonsense reasoning, physics-based realism, human motion, and creative composition. While VBench primarily assessed early-stage technical quality, VBench-2.0 expands the benchmarking framework to evaluate these advanced capabilities, ensuring a more comprehensive assessment of next-generation models. (b) Evaluation Dimension of VBench-2.0. VBench-2.0 introduces a structured evaluation suite comprising five broad categories and 18 fine-grained capability dimensions.

Video

Abstract

Video generation has advanced significantly, evolving from producing unrealistic outputs to generating videos that appear visually convincing and temporally coherent. To evaluate these video generative models, benchmarks such as VBench have been developed to assess their faithfulness, measuring factors like per-frame aesthetics, temporal consistency, and basic prompt adherence. However, these aspects mainly represent superficial faithfulness, which focus on whether the video appears visually convincing rather than whether it adheres to real-world principles. While recent models perform increasingly well on these metrics, they still struggle to generate videos that are not just visually plausible but fundamentally realistic. To achieve real ``world models'' through video generation, the next frontier lies in intrinsic faithfulness to ensure that generated videos adhere to physical laws, commonsense reasoning, anatomical correctness, and compositional integrity. Achieving this level of realism is essential for applications such as AI-assisted filmmaking and simulated world modeling. To bridge this gap, we introduce VBench-2.0, a next-generation benchmark designed to automatically evaluate video generative models for their intrinsic faithfulness. VBench-2.0 assesses five key dimensions: Human Fidelity, Controllability, Creativity, Physics, and Commonsense, each further broken down into fine-grained capabilities. Tailored for individual dimensions, our evaluation framework integrates generalists such as state-of-the-art VLMs and LLMs, and specialists, including anomaly detection methods proposed for video generation. We conduct extensive human preference annotations to ensure evaluation alignment with human judgment. By pushing beyond superficial faithfulness toward intrinsic faithfulness, VBench-2.0 aims to set a new standard for the next generation of video generative models in pursuit of intrinsic faithfulness.

VBench-2.0 Evaluation Results of Recent Video Generative Models

We visualize the evaluation results of recent SOTA video generation models across 18 VBench-2.0 dimensions. We normalize the results per dimension for clearer comparisons.

The values have been normalized for better readability of the chart. The normalization process involves scaling each set of performance values to a common scale between 0.3 and 0.8. The formula used for normalization is: (value - min_value) / (max_value - min_value).

Prompt Suite Statistics

The two graphs provide an overview of our prompt suites. Left: the word cloud to visualize word distribution of our prompt suites. Right: the number of prompts across different evaluation dimensions and different content categories.

Example for Evaluating Plot Consistency.

BibTeX

If you find our work useful, please consider citing our works:

@article{zheng2025vbench2,
      title={{VBench-2.0}: Advancing Video Generation Benchmark Suite for Intrinsic Faithfulness},
      author={Zheng, Dian and Huang, Ziqi and Liu, Hongbo and Zou, Kai and He, Yinan and Zhang, Fan and Zhang, Yuanhan and He, Jingwen and Zheng, Wei-Shi and Qiao, Yu and Liu, Ziwei},
      journal={arXiv preprint arXiv:tbc.tbc},
      year={2025}
}
      
@article{huang2024vbench++,
  title={VBench++: Comprehensive and Versatile Benchmark Suite for Video Generative Models},
  author={Huang, Ziqi and Zhang, Fan and Xu, Xiaojie and He, Yinan and Yu, Jiashuo and Dong, Ziyue and Ma, Qianli and Chanpaisit, Nattapol and Si, Chenyang and Jiang, Yuming and Wang, Yaohui and Chen, Xinyuan and Chen, Ying-Cong and Wang, Limin and Lin, Dahua and Qiao, Yu and Liu, Ziwei},
  journal={arXiv preprint arXiv:2411.13503},
  year={2024}
}
      
@InProceedings{huang2023vbench,
  title={{VBench}: Comprehensive Benchmark Suite for Video Generative Models},
  author={Huang, Ziqi and He, Yinan and Yu, Jiashuo and Zhang, Fan and Si, Chenyang and Jiang, Yuming and Zhang, Yuanhan and Wu, Tianxing and Jin, Qingyang and Chanpaisit, Nattapol and Wang, Yaohui and Chen, Xinyuan and Wang, Limin and Lin, Dahua and Qiao, Yu and Liu, Ziwei},
  booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition},
  year={2024}
}