Twitxy
Saved
@nikaletras's profile picture
View on X

Nikos Aletras

@nikaletras

Professor @SheffieldNLP. I do #NLProc stuff.
Sheffield, England nikosaletras.com Joined September 2009
372 Following 2.1K Followers 1.5K Posts

Latest posts 5

Nikos Aletras @nikaletras
Many congratulations to Miles who passed his PhD viva with minor corrections!

During his PhD, Miles did fundamental work on large language model compression, including a long and successful collaboration with AstraZeneca NLP team in Cambridge.
CDT in Speech and Language Technologies @sltcdt Oct 1
Congrats to Miles Williams on passing his #PhD #viva on "Towards Efficient and Effective Language Model Compression". Thanks to supervisor Prof @nikaletras and viva examiners Dr @PontiEdoardo (Imperial College London) and Professor @EGVasilaki (Sheffield)!
View quoted post on X
0 replies 0 reposts 10 likes 597 views View on X
Nikos Aletras @nikaletras
Pre-pretraining still works when scaling to 7B models but not for the reasons that it works in smaller models (up to ~1B) 馃憞
Atsuki Yamaguchi @_gucciiiii Oct 1
A last piece of my PhD work is out!
arxiv.org/abs/2609.39827

We ask whether "pre-pretraining" (PPT) (i.e., training on synthetic non-natural language data briefly prior to pre-training on natural language data) helps at scale.

This is the largest ever PPT study! (1/5)
Prior work sits at or below 1B parameters and short training horizons. We span four scales, four PT mixtures, five PPT tasks, and up to 100B PT tokens to test the effectiveness of PPT in a more realistic setting.
View quoted post on X
0 replies 1 reposts 11 likes 571 views View on X
Nikos Aletras @nikaletras
Higher eval scores for preferences and diversity might just mean your LLM is becoming a dull copycat, if you train it on preference pairs from a different domain.

Here is a comprehensive recipe book and analysis of transf learning methods for pref tuning, coming at #EMNLP2026馃憞
Constantinos Karouzos @ckarouzos Aug 24
"An Empirical Study on Preference Tuning Generalization and Diversity Under Domain Shift" is accepted at EMNLP 2026! 馃帀
See you in Budapest! 馃嚟馃嚭
Work with @Xingwei__Tan and @nikaletras (@SheffieldNLP ). Powered by @sltcdt!
View quoted post on X
1 replies 2 reposts 23 likes 1.7K views View on X
Nikos Aletras @nikaletras
We will be presenting our MultiHashFormer models at #EMNLP2026

#鈨e "scale" LM heads using a combinatorial hashing architecture.

馃搱MHF beats standard LM heads at 1B and 3B model scales using the exact same Transformer backbone.

馃攲Offers parameter-free vocabulary expansion.
Nikos Aletras @nikaletras Jun 29
Here is MultiHashFormer, our new framework for vocabulary efficient language modelling. Work w/ @HuiyinXue and @_gucciiiii

Inspired by chaotic dynamic memory systems with distributed state spaces, we replace the LM embedding matrix with a modular hashing interface

1/N
View quoted post on X