@skprat
We’re bringing agentic video understanding to our latest Gemini models.
They can now analyze videos with better accuracy while using up to 88% fewer tokens. 🧵View quoted post on X
Learn how you can build with the power of advanced video understanding with Gemini 2.5 🎥 ↓View quoted post on X
developers.googleblog.com/en/gemini-2-5-…
Welcome to the world, Gemini 2.0 ✨ our most capable AI model yet.
We're first releasing an experimental version of 2.0 Flash ⚡ It has better performance, new multimodal output, @Google tool use - and paves the way for new agentic experiences. 🧵 goo.gle/gemini-2View quoted post on X
Towards Truly Zero-shot Compositional Visual Reasoning with LLMs as Programmers
paper page: huggingface.co/papers/2401.01…
Visual reasoning is dominated by end-to-end neural networks scaled to billions of model parameters and training examples. However, even the largest models struggle with compositional reasoning, generalization, fine-grained spatial and temporal reasoning, and counting. Visual reasoning with large language models (LLMs) as controllers can, in principle, address these limitations by decomposing the task and solving subtasks by orchestrating a set of (visual) tools. Recently, these models achieved great performance on tasks such as compositional visual question answering, visual grounding, and video temporal reasoning. Nevertheless, in their current form, these models heavily rely on human engineering of in-context examples in the prompt, which are often dataset- and task-specific and require significant labor by highly skilled programmers. In this work, we present a framework that mitigates these issues by introducing spatially and temporally abstract routines and by leveraging a small number of labeled examples to automatically generate in-context examples, thereby avoiding human-created in-context examples. On a number of visual reasoning tasks, we show that our framework leads to consistent gains in performance, makes LLMs as controllers setup more robust, and removes the need for human engineering of in-context examples.View quoted post on X
Very happy to finally show our latest work: VCT, a transformer for video compression 🤖 We radically simplify the video compression setup by getting rid of motion prediction, warping, and residuals. Instead, a transformer learns to model temporal redundancies purely from data:View quoted post on X
