Twitxy Saved

Dan DiPietro

Verified account Applied Compute

@danyulol

building @appliedcompute. founder, previously at Jane Street.
New York City dandipietro.com Joined September 2022
315 Following 305 Followers 78 Posts

Latest posts 5

Dan DiPietro Verified account @danyulol
SFT as a training technique is generally very underrated. People have gotten this idea that you can’t SFT without catastrophically degrading a model. This is completely untrue. Done well, you can often come close to RL performance with an order of magnitude less data (1/n)
8 replies 12 reposts 415 likes 21K views View on X
Dan DiPietro Verified account @danyulol
Low-precision is the future. Frontier open models like 4.1 Flash and K3 already serve inference with low precision. I think the same thing will happen soon for training. Training actor forwards, backwards, and even optimizer state will move to 8-bit and 4-bit regimes.
0:23
1 replies 5 reposts 41 likes 8.1K views View on X
Dan DiPietro Verified account @danyulol
We would need 35 Enrons to cancel out the public wealth creation from Amazon's early IPO.

SOX is a great example of how seemingly well-intentioned regulation can have horrific unintended downstream consequences, e.g. reducing middle class participation in an entire category of wealth creation.
Dan DiPietro Verified account @danyulol
Sometimes verifiers are actually very expensive to run, and it’s much cheaper to use an LLMaJ instead. One example: suppose you’re RLing a model to build HFT strategies, and each rollout requires an extensive event-driven backtest that takes hours to complete. (1/n)