Twitxy Saved

Wang shuai (Dylan)

@dylan_wangs

Member of Technical Staff @KeenableAI | Previous Research Fellow at @Ielabgroup | @UWAEMS alumni

RAG, IR, NLP, Model Efficiency

BlueSky: dylanshuaiwang
Brisbane, Queensland shuaiwang.io Joined February 2018
215 Following 232 Followers 184 Posts

Latest posts 5

Wang shuai (Dylan) @dylan_wangs
So glad to be part of Keenable! Seeing the pace, passion, and dedication of the team every day makes me confident that we're destined to build something truly exceptional and deliver the best possible service to our customers; excited for what's ahead!
Artificial Analysis Verified account @ArtificialAnlys Nov 10
Keenable Search 1.0 takes 1st (pro, 83) and 2nd (realtime, 82) on the Artificial Analysis Search Index: ahead of Exa (auto, dynamic highlights) and Perplexity (medium) at 80

We benchmarked both modes of @KeenableAI's Keenable Search 1.0, pro and realtime, in our Stirrup agent harness with Keenable Search as its web_search tool to complete 1700 search-focused tasks.

At $4 per 1k searches, Keenable costs about $0.040 per task in both modes, over a third below the board median ($0.063). It is also one of the fastest products we measure: 15.0s (realtime) and 15.5s (pro) per task, 2nd and 3rd on the board.

Key benchmarking results for Keenable Search 1.0:

➤ Artificial Analysis Search Index: Keenable (pro) scores 83 and Keenable (realtime) 82, up from 67 for both modes when we first benchmarked Keenable in August, and a ~50-point lift over the same model with no search (33). Both lead Exa (auto, dynamic highlights) and Perplexity (medium) at 80

➤ DeepSearchQA is Keenable's strongest benchmark: both modes score 88 F1, 1st and 2nd of 26, ahead of Exa (auto, dynamic highlights) at 86. Pro also ranks 1st on BrowseComp (87%, level with Perplexity (medium)) and on AA-Omniscience (73%, level with Exa (auto, dynamic highlights) and Firecrawl)

➤ Fast search: realtime takes 2.3s of search time per task (2nd of 26) and pro 3.0s (3rd), against a board median of 12.5s. Each search returns in about 330ms (realtime) and 390ms (pro) on average

Key details:

➤ Setup: GPT-5.6 Luna (medium) in Stirrup, Keenable Search API with mode=realtime and mode=pro

➤ Pricing: $4 per 1k requests in both modes (public pay-as-you-go price)

➤ Cards: We disabled Keenable's search cards for testing (disable_cards=true)
View quoted post on X