CircleCI accelerates product insights 10x with Mode

When Rommy Ghaly joined CircleCI last year as the head of product analytics and data science, his boss said, “I don't know what your team does, and it’s up to you to change that.”

Fast forward 12 months, and Rommy’s team is supercharging product-led growth – building dashboards in minutes instead of days or weeks, discovering new metrics to power the next wave of user engagement, and increasing the volume of product insights by 10x.

“It’s because of how fast we can run analysis and share insights with the product team,” Rommy says. “You can't believe it's the same team we had a year ago.”

How did this happen? “Largely because of Mode,” Rommy says. Here, we’ll break down the way Mode helps CircleCI’s product team build a user-centered, high-growth product that drives the business forward.

CircleCI is a platform that helps engineers build software faster by enabling them to work on different parts of a complex project at the same time.

“We are a product-led growth company,” says Rommy. At companies like these, the product takes center stage, working as the primary way to acquire, retain, and monetize customers – rather than traditional sales or growth marketing.

Case in point: “Like many product-led growth businesses, we offer a free plan,” Rommy says. As customers use CircleCI more often and invite additional team members to the platform, they graduate to paid plans that support their larger-scale needs. “Our goal is to allow for a one-person startup team to get started on CircleCI and become a major enterprise company on our platform,” Rommy says. 

Since the business grows as more customers use the product, measuring product usage is critical. “By understanding product metrics, we increase engagement and thus revenue,” Rommy says.

How does the product team get growth-driving product insights when they need it? “All our product managers have access to Mode,” Rommy says.

This year, the product team needed a new dashboard in time for a major product launch, so they could observe key metrics while the launch was happening.

“With Mode, spinning up the dashboard literally took us 30 minutes, “ Rommy says. “Even before the product launched, we could see the data flowing into the dashboard. Because of Mode, we got immediate feedback on the performance of new features so we could iterate during the launch period. In fact, we had daily standups to look at the data in Mode to determine where we wanted to focus on optimizing the feature post-launch.”

Mode enables this high-velocity work by letting data teams query the data warehouse directly using SQL, something that’s far less ideal in Looker, for example. Rommy says: “We had access to all the tables in Snowflake, so we could just spin up a really fast and easy Mode dashboard.”

“Looker is highly governed and very reliable,” Rommy says. “But building a dashboard on Looker has to go through a much bigger process within our organization, with many checkpoints.”

Since product usage is the core of CircleCI’s business, the product team pays close attention to how customers use the tool. While there are near-limitless ways to measure product usage (minutes? users? clicks?), selecting the relevant metric – the one that shows how to create growth – requires insight from skilled analysts. That’s the work that data teams do best, and that’s what Rommy’s team used Mode to do.

Analyzing the product data with Mode, Rommy’s team identified an important and previously-overlooked metric. While the metric itself is proprietary, tracking it allows the teams to accelerate the customer onboarding process so engineers using CircleCI can get to work even faster.

The product team is now re-orienting its strategy to speed up new-user onboarding with this metric in mind, Rommy says. “Engagement has now shifted very much to a user-focused mindset.”

This could have huge impacts on the business. “As soon as you figure out the engagement problem, you've solved the biggest problem for your product-led growth business,” Rommy says. “Deep engagement makes acquisition costs more efficient, since ad spend without engagement is just burning money. Plus, when engaged customers start paying for the product, you're going to significantly improve monetization.”

The entire product organization can view and interact with this new metric in Mode. “Visualizing the data in Mode was much more effective. Building the same thing in Looker would need several different charts to visualize the data,” Rommy says.

“When I joined, my team was sharing two or three analyses per month, and now, we’re running upwards of 15 to 20 analyses per month, largely because of Mode,” Rommy says.

“We've now become the team that is recognized at CircleCI for our ability to share data quickly, to share insights quickly, and to drive decisions and strategy,” he adds.

Mode makes this happen by allowing analysts to do more work in fewer steps, and then share their insights with one click.

“Historically, sourcing, analyzing, and sharing data quickly was challenging because of the lift it took,” Rommy says. “You would run the query locally, you would export the result to a Google sheet and build your charts there. Then you would screenshot those charts or copy them elsewhere before you could write up the analysis and share it. Because of that, analyses took longer and were less frequently accessible or shared broadly.”

With Mode, it’s a different world. “Now, we can do quick analysis in notebooks in Mode, doing SQL in the exact same place,” Rommy says. “Then, we can combine the analyses into a single report for anybody in the product organization to access.”

Analysis is an iterative process where product and data leaders work together in a positive feedback loop – to define key questions, review the analysis, and prioritize next steps.

Accelerating this process is the major reason why CircleCI continues to partner with Mode. Rommy says: “Mode helps analysts and product managers analyze deeply and get quick answers. Speed to feedback is our core value.”