Ce Zhang

Excited to share our new work, “Searching Videos as Trees: Self-Correcting Agents for Grounded Long Video QA.”

Grounded LVQA requires a model to answer a question about a long video and localize the short interval supporting its answer. We introduce VideoTreeSearch (VTS), which organizes the video as an adaptive temporal tree and lets an agent search, backtrack, and revise its search path.

VTS brings strong gains on three grounded LVQA benchmarks and also transfers to general long-video QA.

(1/9)

Recent advances in test-time optimization have led to remarkable reasoning capabilities in LLMs. However, the reasoning capabilities of MLLMs still significantly lag, especially for complex video-language tasks.
We present SiLVR, a Simple Language-based Video Reasoning framework.

Excited to share that LLoVi is accepted to #EMNLP2024.

We will present our work in poster session 12, Nov. 14 (Thu.) 14:00-15:30 ET. Happy to have a chat!

Check out our paper at: arxiv.org/pdf/2312.17235
Code: github.com/CeeZh/LLoVi
Website: sites.google.com/cs.unc.edu/llo…

Prior long-range video understanding methods are often costly and require specialized long-range video modeling designs. We present LLoVi, a simple yet effective framework with LLM for long-range video question-answering (LVQA). 🧵

arxiv.org/abs/2312.17235
github.com/CeeZh/LLoVi