rahul

Julius Research

@0interestrates

CEO @JuliusAI | DeepFinance bench (first long horizon finance benchmark): juliusresearch.ai/blog/deep-fina…

San Francisco, CA Joined October 2020

3.5K Following 69K Followers 5.1K Posts

Latest posts 5

Pinned post

rahul @0interestrates 5d

Today we're releasing DeepFinance, the first ultra-long horizon benchmark for high finance work.

A set of 21 problems that reflect work at top PE and IB firms, DeepFinance is built using real deal data.

Problems are constructed by experts with backgrounds from top firms like Evercore, JPM and UBS, and institutions like HBS and Wharton. Problems represent work that would take human professionals 20-40 hours to complete.

Can AI really diligence a $100M+ deal? Finance benchmarks today are limited to evaluating agents on QA on financial docs or grading the numerical correctness of simple financial models. However, real world high finance work is full of assumptions and conflicting evidence. We built DeepFinance to realistically evaluate agents on those end-to-end finance workflows.

DeepFinance leaderboard: juliusresearch.ai/deep-finance

rahul @0interestrates 9h

Gemini 4 Carbon scores 56.6% on ultra long horizon finance tasks and takes #2 spot on DeepFinance bench

nice work by the GDM team

Logan Kilpatrick Verified account @OfficialLoganK Oct 28

Introducing Gemini 4 Carbon, it's a big jump over Argon which we released just 2 weeks ago.

Gemini 4 Carbon is SOTA on cyber, long horizon tasks, ranks #1 on Artificial Analysis V5, significantly improves on usability in domains like coding, and is more aligned.

View quoted post on X

rahul @0interestrates 1d

never wrong. just early

Ilya Sutskever Verified account @ilyasut Sep 1

Neoclouds have limited cybersecurity. Next time agents successfully go rouge, they'll try taking over a neocloud to run more copies. This is bad.

Thus: neoclouds should greatly strengthen their cybersecurity and every company with strong cyber models should help with that.
View quoted post on X

rahul @0interestrates 1d

a slightly deeper dive on Inkling 2 Small on long horizon finance workflows:

1. on complex tasks, Inkling 2 Small doesn't work sequentially like Claude or GPT. while inspecting trajectories, we found that other models tended to follow a sequence of reading → deciding → doing → verifying, whereas Inkling trajectories tended to go back and forth frequently between those steps. Inkling seems more human like in its problem solving (see color distribution in chart 1 below which shows distribution on an example task)

2. at higher reasoning efforts, the additional tokens went towards checking and building the outputs. however scores on lower reasoning efforts were similar to scores on higher reasoning effort

3. Inkling is good at checking its calculations and many of its misses on DeepFinance came not from miscalculating but actually from checking against wrong priors

rahul Verified account @0interestrates 5d

Inkling 2 Small from @thinkymachines is now the best American open source model on DeepFinance bench

Inkling 2 Small scores higher than GLM 5.3, DeepSeek V4.1 Flash and Nvidia’s Nemotron 3 Ultra on long-horizon high finance workflows

View quoted post on X

rahul @0interestrates 1d

real word finance: "pls fix, thx"

finance benchmarks: "create a detailed breakdown of the cash flow analysis, including key assumptions, projections, and sensitivity scenarios, with a fully linked three-statement model and supporting schedules"

DeepFinance bench fixes this. prompts are realistic, short, and reflect how real world finance works

a sample task instruction:

rahul Verified account @0interestrates 5d

Today we're releasing DeepFinance, the first ultra-long horizon benchmark for high finance work.

A set of 21 problems that reflect work at top PE and IB firms, DeepFinance is built using real deal data.

Problems are constructed by experts with backgrounds from top firms like Evercore, JPM and UBS, and institutions like HBS and Wharton. Problems represent work that would take human professionals 20-40 hours to complete.

Can AI really diligence a $100M+ deal? Finance benchmarks today are limited to evaluating agents on QA on financial docs or grading the numerical correctness of simple financial models. However, real world high finance work is full of assumptions and conflicting evidence. We built DeepFinance to realistically evaluate agents on those end-to-end finance workflows.

DeepFinance leaderboard: juliusresearch.ai/deep-finance

View quoted post on X