Twitxy Saved

rohin

Verified account This account is verified. Learn more

@rohinlohe

agent payments @ cloudflare
sf Joined October 2017
505 Following 903 Followers 214 Posts

Latest posts 5

rohin Verified account @rohinlohe
“This joint pilot, which used a participating cohort of ~1% of Cloudflare’s total network, has enabled OpenAI to discover and index hundreds of millions of new web pages, helping people discover more useful content and get more questions answered through ChatGPT.”
Cloudflare @Cloudflare Oct 15
Today, Cloudflare and @OpenAI released the first findings from their joint research pilot, demonstrating how real-time network signals can make AI search drastically smarter, fresher, and safer without overloading website infrastructure. cfl.re/4zUxVPd
View quoted post on X
1 replies 1 reposts 21 likes 1.1K views View on X
rohin Verified account @rohinlohe
Someone has to foot the bill for the compute and bandwidth. Shouldn't it be the (abusive) crawler?

Charge crawlers (e.g. by User Agent): developers.cloudflare.com/monetization-g…

Block crawlers: developers.cloudflare.com/bots/additiona…
developers.cloudflare.com Monetization Gateway · Cloudflare Monetization Gateway docs Charge for protected resources through the x402 protocol.
Josh tried coding Verified account @joshtriedcoding Oct 5
meta's crawler is spamming the hell out of people

we got 700,000+ requests in a single day to Upstash, many of them fetching the same pages hundreds of times over

what the fuck man 😭
View quoted post on X