Twitxy Saved

Shaiyon

Verified account This account is verified. Learn more

@hshaiyon

applied research @ramplabs
New York Joined August 2013
434 Following 256 Followers 249 Posts

Latest posts 5

Shaiyon Verified account @hshaiyon
Incredibly proud to share my work with the world! There is so much alpha in enterprise traces, we're just getting started.
Ramp Labs Verified account @RampLabs 1d
We’re publishing Ramp SWE-Bench v2, featuring 500 tasks built from work shipped to production by over 200 of our engineers, more rigorous verification, and frontier models running in their native harnesses. labs.ramp.com/swebench
View quoted post on X
8 replies 1 reposts 48 likes 2.2K views View on X
Shaiyon Verified account @hshaiyon
It’s an interesting time to be in evals, was fun learning about what everyone’s building!
Kaggle @kaggle Oct 28
@zapier, one of our Bench Summit partners, led a panel discussion on the realities of managing highly influential evaluation signals.

The panel, moderated by Lukas Bergstrom (@lukasb), included insights from Robin Salimans (@SalimansRobin) of Zapier, Greg Kamradt (@GregKamradt) of ARC Prize Foundation, and Shaiyon Hariri of Ramp.

Attendees heard about what defines a successful benchmark, the challenges of maintaining adoption, and critical lessons learned from V1 to V2+.
View quoted post on X
5 replies 1 reposts 27 likes 1.2K views View on X
Shaiyon Verified account @hshaiyon
identify desired behavior → build eval → induce behavior → ??? → profit
Ramp Labs Verified account @RampLabs Oct 15
We had early access to @NVIDIAAI Nemotron 3.5 Super VL and post-trained it for one of our highest-volume workflows: receipt extraction.

One epoch of supervised fine-tuning on 10,000 examples made it competitive with leading open and closed-source models on our receipt benchmark.
View quoted post on X
1 replies 0 reposts 21 likes 830 views View on X