We are excited for @motherduck Data Outpost next week!
Looking forward to catching up with familiar faces and meeting everyone. 馃殌
Looking forward to catching up with familiar faces and meeting everyone. 馃殌
Galaxy
Verified account This account is verified because it's an official organization on X. Learn more@getgalaxyio
Building open data infrastructure for the context era.
Try Filament, our data movement engine.
Try Filament, our data movement engine.
349 Following 230 Followers 94 Posts
Latest posts 5
Huge product update:
You can now transform your data in Filament 馃敟
Drop columns, rename them, mask PII, and regex out junk before it hits your warehouse.
All in-memory on Apache Arrow, fast at any volume.
Batch runs today. Apache 2.0, self-host anywhere.
You can now transform your data in Filament 馃敟
Drop columns, rename them, mask PII, and regex out junk before it hits your warehouse.
All in-memory on Apache Arrow, fast at any volume.
Batch runs today. Apache 2.0, self-host anywhere.
1:20
New connectors in Filament:
- Kafka source + sink
- @ServiceTitan source
- @splunk source
- FHIR R4 source
- fxmacrodata + livetennisapi.com 馃幘
Also:
- Slack notifications + connector allowlists
- Plus a stack of MySQL + Postgres fixes
Apache 2.0, self-host anywhere
- Kafka source + sink
- @ServiceTitan source
- @splunk source
- FHIR R4 source
- fxmacrodata + livetennisapi.com 馃幘
Also:
- Slack notifications + connector allowlists
- Plus a stack of MySQL + Postgres fixes
Apache 2.0, self-host anywhere
An old data vendor fired us for moving too much data.
We were moving so much volume at my previous co that they were losing money on us. So we built an early version of our own, in-house. Filament, what we build today, came out of that, and it's free and open source.
Since launch a lot of the teams reaching out have been data-adjacent teams like GTM & HR, often without a data team of their own, plus companies in spaces like home services where data infrastructure is becoming more important.
Every week, I'm going to explain one data movement concept, for them and anyone else who's curious --
001: What does "moving data" mean?
Your data lives in lots of places.
Customer records in your CRM like Salesforce
Product usage in a CDP like PostHog
Campaign results in your marketing tools like Meta or LinkedIn Ads
and probably a few spreadsheets on top of that
To understand anything across all of it (for example, "which campaigns bring in customers who stay for more than 6 months"), you need it in one central place, and usually that means putting it in a data warehouse or a data lake.
Getting it there is "data movement."
The goal is getting data from point A to point B in the most efficient way possible. In practice thats keeping the data up to date without it slowing down or getting expensive as the volume grows.
Once it's centralized, you can start building on it, whether that's a data platform or a context layer.
Filament moves data from databases, APIs, streams, files and webhooks to wherever you need it, fast. You might start with a warehouse or lake, then send the same data to a search system or an ML pipeline as your team builds more.
Next week: what's a data stack?
We were moving so much volume at my previous co that they were losing money on us. So we built an early version of our own, in-house. Filament, what we build today, came out of that, and it's free and open source.
Since launch a lot of the teams reaching out have been data-adjacent teams like GTM & HR, often without a data team of their own, plus companies in spaces like home services where data infrastructure is becoming more important.
Every week, I'm going to explain one data movement concept, for them and anyone else who's curious --
001: What does "moving data" mean?
Your data lives in lots of places.
Customer records in your CRM like Salesforce
Product usage in a CDP like PostHog
Campaign results in your marketing tools like Meta or LinkedIn Ads
and probably a few spreadsheets on top of that
To understand anything across all of it (for example, "which campaigns bring in customers who stay for more than 6 months"), you need it in one central place, and usually that means putting it in a data warehouse or a data lake.
Getting it there is "data movement."
The goal is getting data from point A to point B in the most efficient way possible. In practice thats keeping the data up to date without it slowing down or getting expensive as the volume grows.
Once it's centralized, you can start building on it, whether that's a data platform or a context layer.
Filament moves data from databases, APIs, streams, files and webhooks to wherever you need it, fast. You might start with a warehouse or lake, then send the same data to a search system or an ML pipeline as your team builds more.
Next week: what's a data stack?
0:26
Filament v0.9.0 is out:
- NATS JetStream source + sink (shoutout @synadia!)
- @HubSpot, @mondaydotcom, @Zoho, @meetgranola, @Gong_io, @useapolloio, @InstantlyAI, @Mailchimp, @heyreach_io sources
- HTTP connectors pin their API version
- Theme toggle in UI (priorities 馃槄)
More 馃敟 later this week 馃憖
getgalaxy.io/filament
- NATS JetStream source + sink (shoutout @synadia!)
- @HubSpot, @mondaydotcom, @Zoho, @meetgranola, @Gong_io, @useapolloio, @InstantlyAI, @Mailchimp, @heyreach_io sources
- HTTP connectors pin their API version
- Theme toggle in UI (priorities 馃槄)
More 馃敟 later this week 馃憖
getgalaxy.io/filament