@mchang21
Today we are rolling out an experimental Gemini Embedding model for developers with:View quoted post on X
– SOTA performance on MTEB (Multilingual)
- Input context length of (3K --> 8K) tokens
– Output 3K dimensions
– Support for over (50 --> 100) languages
More details in 🧵
Can long-context language models (LCLMs) subsume retrieval, RAG, SQL, and more?
Introducing LOFT: a benchmark stress-testing LCLMs on million-token tasks like retrieval, RAG, and SQL. Surprisingly, LCLMs rival specialized models trained for these tasks!
arxiv.org/abs/2406.13121View quoted post on X
Google released Instruct-Imagen 🎉
Instruct-Imagen matches or surpasses prior task-specific models in-domain and demonstrates promising generalization to unseen and more complex tasks according to human evaluation.
arxiv.org/abs/2401.01952 (1/N)View quoted post on X
I’m very excited to share our work on Gemini today! Gemini is a family of multimodal models that demonstrate really strong capabilities across the image, audio, video, and text domains. Our most-capable model, Gemini Ultra, advances the state of the art in 30 of 32 benchmarks, including 10 of 12 popular text and reasoning benchmarks, 9 of 9 image understanding benchmarks, 6 of 6 video understanding benchmarks, and 5 of 5 speech recognition and speech translation benchmarks. Gemini Ultra is the first model to achieve human-expert performance on MMLU across 57 subjects with a score above 90%. It also achieves a new state-of-the-art score of 62.4% on the new MMMU multimodal reasoning benchmark, outperforming the previous best model by more than 5 percentage points.
Gemini was built by an awesome team of people from @GoogleDeepMind, @GoogleResearch, and elsewhere at @Google, and is one of the largest science and engineering efforts we’ve ever undertaken. As one of the two overall technical leads of the Gemini effort, along with my colleague @OriolVinyalsML, I am incredibly proud of the whole team, and we’re so excited to be sharing our work with you today!
There’s quite a lot of different material about Gemini available, starting with:
Main blog post: blog.google/technology/ai/…
60-page technical report authored by th Gemini Team: deepmind.google/gemini/gemini_…
In this thread, I’ll walk you through some of the highlights.View quoted post on X
