Our partner Stitch is introducing Singer: an open source project for simple, composable ETL. Singer enables any data source to be analyzed in Mode - regardless of whether or not you're a Stitch customer.
Mode is a powerful business intelligence platform for analyzing, visualizing, and sharing all kinds of data. Our customers use Mode to gain insight into many different aspects of their business. That means consolidating data from different tools and sources into a data warehouse. While our ETL partners, like Stitch, integrate with many of the most popular data sources to simplify data consolidation, we know that there are many more places where your data might live.
When our customers need a data source that isn't supported by commercial ETL tools, their only option has been to write a custom script to pull the data themselves. While writing a script for a one-time data pull isn't too hard, we've found that over the long term it creates a big maintenance burden; APIs change, a different data warehouse is chosen, servers go down, cron jobs stop running, API rate limits get hit, and so on. This is why we recommend commercial ETL tools in the first place, but the problem isn't totally solved if only five of your six data sources is supported.
Stitch is introducing Singer to solve that problem. Singer is a better way to write and collaborate on ETL scripts, and it includes three parts:
- Taps, which pull data from sources
- Targets, which send data to destinations
- A JSON-based format for communication between taps and targets
All taps and targets can be mixed and matched, so changing the destination you're loading data into is easy. Since it's all open source, the community can leverage each other's improvements.
Singer integrations can be run on hardware that you manage, but Stitch can run any integration written in the Singer format. When you use Stitch for your Singer integrations, you'll get auto-scaling, a secure infrastructure, credential management, monitoring, and alerting.
