October 5, 2022
NaN minute read
This article was informed by our Co-founder and Chief Analytics Officer, Benn Stancil.
What are the best questions to ask when interviewing for a data analyst? When you search for this question on the web, you’ll find that there are a lot of articles written for interviewees, but hardly any for those doing the interviewing. You’ll also find that many of these articles feature lists and lists of questions that can be quite overwhelming.
This piece is written to share frameworks and principles that our Chief Analytics Officer Benn Stancil has used to interview data scientists that you can adjust for your company (every business is different). Our goal is to share what we’ve learned so that you can quickly hire the best data analyst for your business.
In assessing for skills, we start with some general questions first to understand what type of analytical role they’re interested in.
Starting with a general warm-up question can be helpful. Analyst roles can be very different. A general warm-up question not only helps break the ice, but it can often reveal how the candidate sees the role of analyst, and what they want it to be. It will also show if a candidate truly understands the nature of the role, and hasn’t only read about it. Below are types of warm-up questions that we’ve used before:
Tell me about what brings you here?
What is it that you are looking for?
What excites you about this role?
When interviewing for analysts, you’ll also want to look for soft skill traits that include:
The nature of an analyst's role includes dealing with ambiguous issues at a business. This could look like someone coming to an analyst with a question that they’re not sure how to analyze, or analysts being introduced to a business scenario that doesn’t have a singular question but are tasked with figuring out how leaders should think about the problem and the available options.
How well do they handle this ambiguity? How vague of a situation can they be thrown in and figure out what to do? For example, questions that Benn Stancil has asked when interviewing candidates are:
We’re thinking about hosting a conference, should we do it?
We’re considering launching a new product line in early December or in late January; how would you help us decide which date is right?
These questions have no right answers. Instead, look for how candidates handle the problem. The traits and abilities we look for are below.
Does this person get frustrated or turned off from these types of questions? Because these types of questions are ones analysts deal with on the day-to-day, knowing that a candidate is excited, interested, or empowered by these types of questions will shed light on how their attitude would be on your team.
Analysts will be asked tons of questions and will need to share their findings with a variety of stakeholders–some technical, many not. Can this person talk through their thought process? Can they explain the observations that they’re seeing in a coherent way?
These questions should focus more toward uncovering their level of quantitative comfort and problem-solving ability than if they always compute the right numerical outcome.
Which metrics would you use to measure your decision? What might that metric miss?
What’s the difference between mean and median? And which one is better?
These types of question test comfort level and experience—have they seen problems like this before? Typically you’ll hear the answer “it depends,” which you might follow up with “If it depends, when is the case you would use one vs. the other?”
For example, in the mean and median question, most candidates will know that you use the median when something is badly skewed, like income distributions. But people struggle with identifying when they’d choose the mean over the median. Following up with “When would you choose the mean over the median?” pushes people to think more mathematically. It’s an unfamiliar question that, when answered, will show both their experience and how comfortable they are thinking through an unfamiliar twist on a familiar problem. Drilling down on these types of questions are key to differentiating skill level.
Some questions just won’t help as much as others, Benn says. This can vary a lot by the specifics of the role, but here are questions we avoid:
Avoid questions about SQL problems and coding
There are different philosophies on whether to include these in an interview or not, but generally they are unhelpful for understanding how candidates approach analytical problems. We think problem solving is more important than technical skills, which can often be taught or refined on the job.
Avoid vague questions about datasets, like “Have you worked with a large dataset?”
This question is too vague and one that’s used to be a stylistic filter. People may ask these questions to see whether a candidate has worked with data in databases and not just in Excel files. Though those differences matter, it rarely matters if their database had hundreds of gigabytes, hundreds of terabytes, or petabytes of data in it. What does matter is their ability to analyze that data.
After talking to hundreds of data teams, here's what we found were common stages of data team growth.
What determines the crossover from junior analyst to senior analyst?
How does one differentiate interview questions for senior data analysts vs interview questions for junior data analysts? You really don’t, says Benn. At the root of every analyst’s job is data analysis. So questions don't necessarily need to be different, but based on the candidate's level, their answers should be. What sets analysts apart and highlights their seniority level is their ability to translate business needs into analytical problems without being told directly what to solve for.
"The further away you can get from someone saying ‘I want this number on this chart to tell me this thing’ —that’s really the step up the senior ladder."
Vicky Yu, previously a Senior Lead Analyst at Myfitnesspal and Senior Data Analyst at Prezi, states “A junior analyst simply provides the data requested. A senior analyst goes one step further to learn the company KPIs and how their results relate to KPIs that impact the business.” We’d agree. The shift in thinking from junior to senior analyst
With the most junior analyst, you might go: “I need a chart that shows this on the x-axis and this on the y-axis” and they’ll go crunching numbers.
A level up from that, you might say: I need to figure out conversion rates for marketing emails; I can translate that into numbers.
A level up from that, you might say: Someone is saying “Help us make marketing better,” and you might go improve conversion rates for emails.
A level up from that: Nobody says anything but you recognize that the company is running a bunch of emails and think ‘we can improve those.’
Be prepared to walk the path of the interviewee on their data analysis process. With interview questions that are more ambiguous and don’t lend themselves to set responses, you have to be prepared to journey with the candidate. The data interview questions we listed are designed for a candidate to unpack their process and describe how they stepped their way through it.
You might ask “How would you measure this?” The candidate will respond and then you can follow up with “How would you measure it this way?” or “Suppose you can’t get this information, then what do you do?.” Adding these additional complications will require candidates to think about the problem in different angles. Likewise, your interviewee might drive conversations with additional questions or thoughts.
"One of the hardest parts of interviewing data analysts is having the conversation–walking the path with them. Though we keep our initial questions standard, we don’t always ask the same follow ups. The interviewee is trying to solve a problem, and you have to let them lead."
Give attention to those with untraditional backgrounds. The best analysts are people who get vague problems and have analytical instincts of what to do about them. These types of people don’t necessarily come from stem backgrounds. They come more from the social sciences—people who have to build novel experiments and think about questions like “How do I measure the effects of something in politics? How do I construct a way to measure this qualitative phenomenon with the data that I have? ”
"Should we host a conference is not a thing you can solve a math problem to figure out; it’s a thing where you try to figure out how in the world you would measure this very immeasurable thing?"
Make sure people from untraditional backgrounds get a real look and don't get filtered out because they don’t have experience that fits exactly to the job description or a standout resume
Asking the right type of interview questions is essential to finding a great data analyst fit for your team. Consider crafting questions that highlight how they deal with ambiguity—and look out for not only their response to the questions, but their level of enthusiasm for it.
Be willing to dive deeper and journey with the interviewee. Stay away from surface-level questions, and make sure to not miss out on candidates just because they come from an untraditional background.
Join this webinar to learn more about the flexible graphing system, Visual Explorer, and how it can support your data-driven storytelling.
Work-related distractions for data enthusiasts.