As an industry, we’ve published countless articles, papers, videos, and diagrams about the modern data stack—describing what it is, what it isn’t, where it’s going, and how it’s evolving. As we embark into the next phase of BI, data leaders are asking themselves again:
What should the modern data stack do for my team today, and how can it drive the business forward for years to come?
These questions require data leaders to think critically about the experiences that the modern data stack should provide across lines of business, people, processes, tools, and other software. What would it take to turn your data into a true competitive advantage?
Have we achieved operational efficiency; can all of my analysts quickly iterate across technical methodologies and languages to perform ad hoc analysis, and deliver consumable data and insights to stakeholders?
Is my organization actually data-driven; can business teams really self-serve their own reports and gain insights from clean, governed data to make both daily and strategic decisions?
Is our data team revered as the source of truth, living at the center of the business and built on a tight, architectural stack that is abridged to maximize ROI in today’s marketplace?
Chances are, you quietly said, “no” to at least one or all of these questions when reading. Here’s why: within the data community, business intelligence (BI) is still treated as a window dressing that makes data more palatable to non-technical audiences, all while restricting the capabilities of analysts to do their work in their preferred languages and methodologies. Chances are before reading this paper, you may not have even considered all of the above functions and their outcomes as modern BI.
The core root of this dilemma is that data is spread across a massive range of people, processes, tools, and platforms. It is not consolidated in a way where it can be governed and used effectively at scale (more point solutions are not the answer). Data teams and business teams simply haven't been able to work well and fast enough in a single tool that meets both parties’ needs.
Most BI solutions simply haven’t cleared the path forward for a better, more impactful data experience.
Over time, this constant push-and-pull dynamic between data teams and business teams has eroded confidence in the validity of data at organizations, and most BI solutions simply haven’t cleared the path forward for a better, more impactful data experience.
The modern data experience is the promise of data fully realized at an organization: trusted self-service analytics, operational efficiency, and data maturity—all made possible by centering the data team.
The modern data experience requires a full reframing of people, processes, and tools where data teams sit in the center, sharing their guidance upfront and working in a tool that prioritizes everything they need in an analytical workflow for speed. Think: data teams at the helm—less service desk, more influence.
Driving from the front office–not the back—data teams should own, create, and maintain the data hub that everyone within the organization operates from. In this new world, the data team’s work (in collaboration) informs every decision inside the business because they are the data traffic control center. The data team oversees the main highways by verifying data sets, metrics, and permissions so the business team can work with pre-validated data that is standardized with their expertise, and relevant modifications.
Ad hoc analysis and self-serve are two sides of the same coin when data teams are at the center.
A modern data experience gives analysts the ability to curate guardrails, so business users have a tangible, and realistic starting place. And when business-oriented data teams can influence revenue through all functions of a business in this manner, they become indispensable. This is the pinnacle data experience that today’s stack should enable.
A modern data experience lets data teams do all of this in the same place where they also perform the game-changing ad hoc analysis with speed and efficiency. In fact, ad hoc analysis and self-serve are two sides of the same coin when data teams are at the center. Each process informs the other, and by bringing data experts and domain experts closer together to collaborate in one place, the true value of the modern data stack is realized.
Loving this content? Get the full guide
Here are 10 foundational principles for achieving a modern data experience across your organization.
These principles are particularly important because this experience won’t be built by a single vendor, but by an entire ecosystem. Though we’ll never be perfectly aligned or agree to the same ideas, these principles can help keep us rowing in the same direction. Modern BI should:
Enable everyone to do their job rather than asking them to be an analyst. We don’t just hand people data and ask them to analyze it; we incorporate it into the format (no code, or SQL, Python, and R) or operational systems where they already live. Data should help people do their jobs, rather than add a new job for them to do. This accelerates speed to insights across the organization and helps you stay ahead of the competition.
Operate from one central data hub. Anyone should be able to transition seamlessly from being able to view a key metric sourced from a well-vetted data catalog to exploring that metric with groupings and filters to incorporating it in deep technical analyses. Business stakeholders consuming data should never have to fully leave one system and start over in another. This eliminates cost-switching between tools and helps maintain accuracy.
Makes status and trust explicit to empower confident decision-making. Every data asset should show if upstream processes are operating abnormally, out-of-date, or in some state of development or disrepair. Our goal should be to spend more time debating what to do because of a number on a dashboard than we spend verifying if that number’s right. This will increase the number of data-driven decisions made across our organization without overburdening my in-demand data team.
Remember what we’ve learned to scale through every phase of data maturity. To make sure our time is spent exploring new territory rather than retracing old steps, a modern data experience should remember and catalog what we learn and what we say about it.
Govern business logic globally for data democratization at scale. Business logic—instructions for transforming data and computing metrics—should be centralized such that it’s accessible anywhere data is consumed, whether that’s a BI dashboard, a Python notebook, or an operational ML pipeline.
Adapt and display data across an ecosystem of tools. To data analysts, data is made up of tables and relational structures. To everyone else, data is ever-changing. Sometimes it’s a time series of a single metric, sometimes it’s a complex pivot table, and sometimes it’s a document of explanatory narratives. People should be able to search for, ask questions of, and explore data in these terms, not just as tables and columns.
Bridge data from past and future tools as the industry evolves. The modern data stack is not a leap: It’s a transition and some uncomfortable anchors are coming with us. Most notably, Excel isn’t going away. A modern data experience has to negotiate with it and not treat it as an outdated pariah.
Go backward and forward. We’re borrowing this one from Tristan Handy at dbt. Analytical work requires a lot of revisions, and trial and error. Tools should support this iterative process, allowing people to easily go from point A to point B and back again. We add to Tristan’s point by articulating that A to B and back to A often means a cycle between self-serve by business teams and ad hoc analysis by data teams – keeping those flows in the same tool helps inform the other and speeds up cycle time.
Be elastic and not rigid for operational efficiency. The modern data experience should be emergent—able to start small and grow into new, unforeseen territory. Rigid experiences and systems create debt that will quickly come due.
Transform relationships between people, businesses, and data. Technical modularity can’t tempt us to build more walls. A modern data experience needs to break down walls by encouraging collaboration and conversation between business, data, and engineering teams. It needs to let you discover new ways of understanding and adapting to different workflows for teams across disciplines.
Get the full guide
How to Lead from The Center: A Guide for Data Leaders in a New Era of BI Transformation
What does it take to turn your data into a true competitive advantage?
As an industry, we’ve exhausted ourselves describing what Modern BI is, what it isn’t, and how it’s evolving. As we embark into the next phase of BI, data leaders are asking themselves again, ‘What should the modern data stack do for my team today, and how can it drive the business forward for years to come?’