Twitxy Saved

Dimitriadis Nikos

@nikdimitriadis

Research scientist at Google DeepMind. PhD @ EPFL.
nik-dim.github.io Joined December 2020
238 Following 216 Followers 58 Posts

Latest posts 5

Dimitriadis Nikos @nikdimitriadis
Going to Singapore for ICLR πŸ‡ΈπŸ‡¬

Happy to meet people and discuss anything related to LLMs, model merging, multi-task and continual learning! DM me if you want to chat!

We will also be be presenting two papers:

1. LiNeS x.com/nikdimitriadis…

2. PaLoRA arxiv.org/abs/2407.08056
Dimitriadis Nikos @nikdimitriadis Oct 25, 2024
Fine-tuning pre-trained models leads to catastrophic forgetting, gains on one task cause losses on others. These issues worsen in multi-task merging scenarios.

Enter LiNeS πŸ“ˆ, a method to solve them with ease. πŸ”₯

🌐: lines-merging.github.io
πŸ“œ: arxiv.org/abs/2410.17146

🧡 1/11
View quoted post on X
Dimitriadis Nikos @nikdimitriadis
Fine-tuning pre-trained models leads to catastrophic forgetting, gains on one task cause losses on others. These issues worsen in multi-task merging scenarios.

Enter LiNeS πŸ“ˆ, a method to solve them with ease. πŸ”₯

🌐: lines-merging.github.io
πŸ“œ: arxiv.org/abs/2410.17146

🧡 1/11
lines-merging.github.io LiNeS We propose a simple algorithm that scales linearly the parameters of the task vector as a function of depth and show that it is effective in many different settings
6 replies 55 reposts 254 likes 31K views View on X
Dimitriadis Nikos @nikdimitriadis
Good morning ICMLπŸ‡¦πŸ‡Ή Presenting today "Localizing Task Information for Improved Model Merging and Compression" with @wangkeml @gortizji @francoisfleuret @pafrossard.

Happy to see you at poster #2002 from 11:30 to 13:00 if you are interested in model merging & multi-task learning!
Dimitriadis Nikos @nikdimitriadis Jun 3, 2024
Wouldn't it be great if we could merge the knowledge of 20 specialist models into a single one without losing performance? πŸ’ͺ🏻

Introducing our new ICML paper "Localizing Task Information for Improved Model Merging and Compression". πŸŽ‰

πŸ“œ: arxiv.org/pdf/2405.07813

🧡1/9
View quoted post on X
0 replies 4 reposts 14 likes 829 views View on X