Common Crawl Foundation

This account is verified. Learn more

@CommonCrawl

Common Crawl is a non-profit foundation dedicated to the Open Web.

San Francisco, CA commoncrawl.org Joined February 2010

1.6K Following 7.8K Followers 1.5K Posts

Latest posts 5

Common Crawl Foundation @CommonCrawl 4d

Steering Committee Election Results Announced netpreserveblog.wordpress.com/2026/11/10/ste…

netpreserveblog.wordpress.com Steering Committee Election Results Announced The 2026 Steering Committee Election closed on Friday, November 6. The following IIPC member institutions have been elected to serve on the Steering Committee for terms commencing 1 January 2027: E…

Common Crawl Foundation @CommonCrawl Nov 8

This is a pretty useful summary guide to Common Crawl. (100% AI written, but it's not bad)

casrai.org/guides/common-…

casrai.org Common Crawl: Open Web Data Corpus Guide — CASRAI Explore Common Crawl for AI and text mining. Learn CCBot crawl mechanics, WARC and WET formats, CDX index queries, AWS S3 access, and research licensing.

Common Crawl Foundation @CommonCrawl Nov 5

Introducing warc2zip

We are pleased to introduce warc2zip, a tool that allows anyone to turn WARC archives into simpler, zipped files that can be inspected with ordinary tools.

Common Crawl Foundation @CommonCrawl Oct 30

The uneven rise of AI-generated web content

arxiv.org/pdf/2610.36631

Common Crawl Foundation @CommonCrawl Oct 24

Common Crawl Foundation will be at IETF 127 in SF, week of November 15. If you are around, we'd love to chat!