Twitxy Saved

Shelly Sheynin

@ShellySheynin

Research Scientist @AIatMeta; Working on Media Generation; Meta Movie Gen, Emu Edit, Make-a-Video 3D, KNN Diffusion, Make-A-Scene
Joined September 2020
208 Following 1.1K Followers 126 Posts

Latest posts 5

Pinned post

Shelly Sheynin @ShellySheynin
I’m thrilled and proud to share our model, Movie Gen, that we've been working on for the past year, and in particular, Movie Gen Edit, for precise video editing. 😍

Look how Movie Gen edited my video!
0:14
AI at Meta Verified account @AIatMeta Oct 4, 2024
🎥 Today we’re premiering Meta Movie Gen: the most advanced media foundation models to-date.

Developed by AI research teams at Meta, Movie Gen delivers state-of-the-art results across a range of capabilities. We’re excited for the potential of this line of research to usher in entirely new possibilities for casual creators and creative professionals alike.

More details and examples of what Movie Gen can do ➡️ go.fb.me/kx1nqm

🛠️ Movie Gen models and capabilities
Movie Gen Video: 30B parameter transformer model that can generate high-quality and high-definition images and videos from a single text prompt.

Movie Gen Audio: A 13B parameter transformer model that can take a video input along with optional text prompts for controllability to generate high-fidelity audio synced to the video. It can generate ambient sound, instrumental background music and foley sound — delivering state-of-the-art results in audio quality, video-to-audio alignment and text-to-audio alignment.

Precise video editing: Using a generated or existing video and accompanying text instructions as an input it can perform localized edits such as adding, removing or replacing elements — or global changes like background or style changes.

Personalized videos: Using an image of a person and a text prompt, the model can generate a video with state-of-the-art results on character preservation and natural movement in video.

We’re continuing to work closely with creative professionals from across the field to integrate their feedback as we work towards a potential release. We look forward to sharing more on this work and the creative possibilities it will enable in the future.
View quoted post on X
52 replies 87 reposts 804 likes 235K views View on X
Shelly Sheynin @ShellySheynin
Super excited to share our recent work, VideoJAM, for motion and physics improvement in T2V models. VideoJAM sets new SOTA in motion generation and understanding.🥳
Project Page: hila-chefer.github.io/videojam-paper…
hila-chefer.github.io VideoJAM VideoJAM: Joint Appearance-Motion Representations for Enhanced Motion Generation in Video Model
Hila Chefer Verified account @hila_chefer Feb 4, 2025
VideoJAM is our new framework for improved motion generation from @AIatMeta

We show that video generators struggle with motion because the training objective favors appearance over dynamics.
VideoJAM directly adresses this **without any extra data or scaling**

👇🧵
View quoted post on X
2 replies 1 reposts 39 likes 2.3K views View on X
Shelly Sheynin @ShellySheynin
Amazing job.
Release Announcement!📢💣
🥁🎷JASCO 🎶🪇 training & inference code + model weights are out!

Paper📜: arxiv.org/abs/2406.10970
Samples🔊: pages.cs.huji.ac.il/adiyoss-lab/JA…
Code🐍: github.com/facebookresear…
Models🤗: huggingface.co/facebook/jasco…

@lonziks @itai_gat @FelixKreuk @adiyossLC
View quoted post on X
Shelly Sheynin @ShellySheynin
Great highlights of Movie Gen by the incredible @Anrew__Brown__ d ✨
Andrew Brown Verified account @Andrew__Brown__ Oct 4, 2024
So how did we get to these amazing videos for Meta Movie Gen? One of the things I’m proudest of is that we released a very detailed technical report (ai.meta.com/research/movie……)

Lets dive into a technical summary of what we did & learnt

🧵 1/n

x.com/AIatMeta/statu…
View quoted post on X