We recently went through a common process at Mode; we considered bringing on a new tool for our data infrastructure. While those of us on the analytics team have a deep understanding of our data stack, we're not experts in security. Before we bring any new data tool into our stack, we ask Mode's head of security, Rafae Bhatti, to help us out with a risk assessment.
Would you believe me if I told you that he simply referred us to Mode's security policy documentation, a few previous reviews he'd conducted, and told us to figure it out?
Of course you wouldn't. Rafae walked us through the issues to consider and made sure we understood how to evaluate the decision. He later followed up directly to make sure the right protocols were followed. And though I'm proud to say that Rafae is one of the best security leaders I've ever worked with, his approach here isn't what makes him stand out. Nearly any security officer worth their salt would do the same thing in this situation.
Why? Because security officers like Rafae see themselves as being responsible for security outcomes, not just inputs. Rafae didn't just give me the information I needed - he followed up to make sure the decision I made was the right one.
Now, imagine this parallel scenario: Our head of marketing is evaluating which of our marketing campaigns need more investment and which need to be cut. She has a few questions about how each one is performing. While she's familiar with our marketing programs and goals, she's not an expert in analytics or our data stack. She wants to make the right decision, so she asks an analyst for help.
A typical response from an analyst would be to point her to a marketing dashboard or a tool for accessing campaign data, and leave her to make the decision on her own.
This hands-off approach would seem reckless for questions about security. And yet that approach is not just the norm for analytical questions in most organizations; it's often the ideal. Many analytics teams aspire to enable as much “self-serve” as possible - in other words, to remove themselves from as many decision-making processes as they can. And many vendors hold this up as the promised land: build a dashboard or report, hand it off to the marketer or product manager to “self-serve” their data, and step away.
Self-serve isn't analysis
This approach often develops for understandable reasons. Companies have to make a lot of decisions, and analysts can't be involved in all of them. Self serve seems like the only way to scale.
But as data scientists, we should be clear-eyed about our primary responsibility: to turn data into insight, and insight into action. Attempting to hand off these problems doesn't change that. Providing a tool to someone to get the data doesn't offload that responsibility. If the data is easily misunderstood, or if the person making the decision hasn't been trained to interpret it, it's on us for creating that situation.
To put it another way, the only things self-serve helps scale is SQL code and a general understanding of business logic. But knowing how to write a query, or how our company defines a new customer, isn't what makes an analyst. We're analysts because, when there are problems, we know which rocks to look under, how to question what we find, and how to distinguish between what we've learned and what we haven't.
The more we focus our role on delivering data access and remove ourselves from delivering outcomes, the more we waste this skill - and the worse off our organizations will be.
