Twitxy Saved

Alba Carballo Castro

@AlbaCbCs

PhD student @EPFL_AI_Center & @ELLISforEurope. Prev @ETH @unisevilla. Part-time musician 🎻✨
Lausanne, Switzerland fcarreras.org/es/donamedula Joined April 2012
505 Following 525 Followers 2.8K Posts

Latest posts 5

Pinned post

Alba Carballo Castro @AlbaCbCs
Muy feliz e ilusionada por todo lo que viene, nos vamos!! πŸš€πŸ‡¨πŸ‡­πŸ₯° @BecariosFLC #laCaixaFoundFellows
πŸ‘©β€πŸŽ“ @AlbaCbCs es alumna del Doble Grado de @matematicasUS y EstadΓ­stica. Ha participado en el programa @EUErasmusPlus y su futuro pasa por πŸ‡¨πŸ‡­ y la @ETH

πŸ—£οΈ Β«Los becados son personas con mucho talento y al final se crea una red de contactos muy buena de cara al futuroΒ».
View quoted post on X
Alba Carballo Castro @AlbaCbCs
See you on Monday! πŸ€—
Discrete Diffusion Reading Group Verified account @diffusion_llms Sep 4
πŸ€” Can masked models get deeper without getting bigger?

Three recent papers converge on the same insight from different directions: reuse computation instead of adding parameters. Depth is normally fixed at training time β€” these works make it something you can spend at inference.

πŸ”§ Three routes to the same frontier:

πŸ’‘ LoopMDM β€” selective looping: repeatedly apply the early-middle transformer layers within each step. Looping at training time gives a depth-scaling effect without adding parameters; varying the loop count at inference time enables flexible compute scaling.

πŸ“ˆ Matches same-size MDMs with up to 3.3Γ— fewer training FLOPs, and gains up to +8.5 points on GSM8K.

πŸ’‘ R-MDM β€” recursion as a third scaling axis: apply the same denoising transformer L times per diffusion step, enabling iterative refinement through pure parameter reuse.

πŸ“ˆ L recursive iterations often match non-recursive baselines carrying ~LΓ— more parameters on Sudoku and Countdown β€” and recursion can partly substitute for denoising steps, cutting inference forward passes.

πŸ’‘ FP-MGM / CoFRe β€” fixed-point solvers: replace part of the denoiser with a fixed-point solver over shared attention layers for adaptive depth, trained with a cross-step consistency loss and three-state reuse (3SR).

πŸ“ˆ Cuts parameters 38.8% and VRAM 16.9% on OpenWebText β€” and extends beyond text, cutting training time 48.6% on ImageNette image generation.

This Monday, the lead authors present their recent work on looped, recursive, and fixed-point masked models:

β–Έ Andrea Miele β€” @andreamiele_ β€” andreamiele.github.io
β–Έ Alba Carballo-Castro β€” @AlbaCbCs β€” acarballocastro.github.io
β–Έ Sanghyun Lee β€” lsh83210.github.io
View quoted post on X
0 replies 2 reposts 15 likes 1.2K views View on X
Alba Carballo Castro @AlbaCbCs
Happy to be an ELLIS PhD Student, thank you! πŸ’«
ELLIS @ELLISforEurope Jun 3
Say hi to Alba Carballo Castro, πŸŽ“ ELLIS PhD Student at @EPFL_en πŸ‡¨πŸ‡­ and @Cambridge_Uni πŸ‡¬πŸ‡§. She researches generative models for discrete data.

Her long-term goal is to contribute to making AI a safe, trustworthy, and inclusive field.

#WomenInELLIS
View quoted post on X
Alba Carballo Castro @AlbaCbCs
Check out our new work! ✨

@andreamiele_ explores the use of Fixed-Point solvers over shared attention layers to enable adaptive depth, making Masked Generative Models cheaper without sacrificing performance. πŸš€

Congrats Andrea, happy to be part of this project! πŸ™Œ
Andrea Miele @andreamiele_ Jun 1
πŸ”₯ New paper: Fixed-Point Masked Generative Modeling

Masked generative models are becoming a very exciting alternative to autoregressive generation, especially for language.

They decode in parallel, but every denoising step still runs a full bidirectional Transformer.

We make them cheaper and stronger with fixed-point denoisers 🧡

w/ @qinym710 @AlbaCbCs @jdeschena and @pafrossard
(1/12)
View quoted post on X