Framing Stakeholder Questions

The Gap Between the Question Asked and the Question Needed

Stakeholders rarely ask precise questions. They arrive with a symptom, a hunch, or a request that sounds specific but is not: "Why did sales drop?" "Can you pull some data on user engagement?" "What's our conversion rate?" The analyst's first and most important skill is not writing SQL. It is closing the gap between the question asked and the question needed.

The question asked is what the stakeholder says. The question needed is what would actually help them make a decision. These are often different. A VP of Sales who asks "Why did sales drop last week?" may actually need to know whether the drop is a tracking artifact, a seasonal pattern, a competitive response, or a genuine problem requiring intervention. The analyst who immediately starts querying sales data without clarifying which of these possibilities matters will produce an analysis that is technically correct but decision-irrelevant.

The translation process has four steps:

1. Identify the decision. Every analysis serves a decision, even if the stakeholder has not articulated it. Ask: "What will you do with this answer?" If the stakeholder says "I just want to know," probe further. "Knowing" is not a decision. The decision might be: whether to increase the marketing budget, whether to roll back a product change, whether to investigate a specific channel, or whether to do nothing because the drop is within normal variance. The decision determines the scope, depth, and urgency of the analysis.

2. Define the terms. "Sales" is not a definition. Is it gross merchandise value, net revenue after refunds, number of transactions, or new customer acquisitions? "Drop" is not a definition. Drop relative to what: last week, the same week last year, the trailing four-week average, or a forecast? Each definition leads to a different analysis. The analyst must propose a definition, confirm it with the stakeholder, and document it so the result cannot be misinterpreted.

3. Specify the scope. Scope has three dimensions: time (what period?), segment (which users, products, or regions?), and depth (surface-level trend or root-cause investigation?). A stakeholder who asks "Why did sales drop?" may be satisfied with a one-page summary showing the drop is concentrated in one region, or they may need a full root-cause analysis with statistical testing. The analyst proposes a scope based on the decision's stakes and confirms before starting.

4. Set expectations. Before starting, communicate: what you will deliver, when you will deliver it, what data you will use, and what limitations the analysis will have. "I will have a preliminary answer by Thursday using data from our warehouse. I cannot isolate the impact of the pricing change from the seasonal effect with the data available, but I can show you the drop by segment and channel." This prevents the stakeholder from expecting a definitive causal answer when the data only supports correlation.

The Analysis Brief

The analysis brief is the document that captures the translated question. It is the contract between the analyst and the stakeholder, and it prevents the most common failure mode in analytical work: delivering a thorough answer to the wrong question.

A brief has six components:

1. Context. Why is this question being asked now? What triggered it? "Sales dropped 12% week-over-week, and the leadership team wants to understand whether this is a trend or an anomaly." Context prevents the analyst from investigating a symptom without understanding the disease.

2. Decision. What decision will this analysis inform? "The decision is whether to launch a retention campaign for the affected segment or to investigate a potential data quality issue first." If no decision can be identified, the analysis may be premature. Sometimes the right response to a vague question is to help the stakeholder figure out what they actually need to decide.

3. Question. The specific, testable question. "Did sales drop significantly in the 25-34 age segment in the Midwest region during the week of March 8-14, and is the drop concentrated in a specific product category?" This is not the question the stakeholder asked. It is the question that answers what they need.

4. Scope and constraints. What is in scope and out of scope. "In scope: sales data from January 1 to March 14, segmented by age, region, and product category. Out of scope: marketing campaign performance, competitor pricing, and customer support ticket volume." Constraints include data availability, time, and analytical methods. "We have 3 days and access to the sales warehouse. We do not have experiment data from the recent website change."

5. Success criteria. What does a good answer look like? "A good answer identifies whether the drop is statistically significant, which segments are affected, and whether the drop correlates with any known events. It does not need to prove causation." Success criteria prevent scope creep and help the stakeholder evaluate whether the analysis met their needs.

6. Deliverable and timeline. What will the stakeholder receive and when? "A written summary with charts, delivered by Thursday EOD. The summary will include a conclusion, evidence, and recommended next steps."

The brief does not need to be a formal document. For a quick request, it can be a Slack message with these six components. For a high-stakes investigation, it should be a shared document that the stakeholder explicitly approves. The formality should match the stakes. The discipline of writing the brief — even briefly — is what matters. It forces the analyst to think before querying and gives the stakeholder a chance to correct misunderstandings before hours of work are invested.

Common Framing Failures

Even experienced analysts fall into predictable framing traps. Recognizing them is the first step to avoiding them.

Failure 1: The data dump. The stakeholder asks for "user engagement data," and the analyst exports every engagement metric they can find. The stakeholder receives a spreadsheet with 47 columns and no insight. The failure is not the query; it is the lack of translation. The analyst should have asked: "What decision are you trying to make with engagement data? Are you trying to decide whether to invest in a new feature, or whether the current feature is being used?" The answer determines which metrics matter.

Failure 2: The premature deep-dive. The stakeholder mentions a drop in sales, and the analyst spends three days building a sophisticated decomposition model before confirming that the drop is real and not a tracking artifact. The first hour of any investigation should be a sanity check: verify the data, check for known issues, and confirm the premise. If the drop is a tracking bug, the deep-dive is wasted.

Failure 3: The unanswerable question. The stakeholder asks, "What would happen to revenue if we changed our pricing?" This is a causal question that requires either a controlled experiment or a strong quasi-experimental design. If no experiment has been run and no natural experiment exists, the analyst cannot answer this with observational data. The correct response is: "I cannot answer that directly with the data we have. I can show you how revenue varied when we changed pricing in the past, but that correlation is not a reliable prediction of what would happen if we changed it again. To answer this properly, we need to design an experiment." This is not a failure; it is honest scoping.

Failure 4: The moving target. The stakeholder approves a brief, the analyst begins work, and the stakeholder sends three messages adding new questions. Each new question seems small, but together they double the scope. The analyst should acknowledge the new questions, assess their impact on the timeline and scope, and either extend the deadline or push back. "I can add the channel breakdown, but it will push delivery to Friday. Alternatively, I can deliver the original scope on Thursday and follow up with the channel analysis next week."

Failure 5: The answer in search of a question. The analyst finds an interesting pattern in the data and presents it to the stakeholder, who has no idea what to do with it. The pattern may be statistically real but decision-irrelevant. Every analysis should connect back to a decision. If the stakeholder cannot act on the finding, the analysis was exploratory, not actionable, and should be labeled as such.

Framing in Practice: A Worked Example

Watch the framing process transform a vague request into a testable brief.

The request: A product manager messages you: "Can you look into why our activation rate is so bad? I think it's the new onboarding flow."

Step 1: Identify the decision. You respond: "Happy to dig into this. What are you trying to decide? Are you considering rolling back the onboarding change, or trying to improve it?" The product manager replies: "If the new flow is actually worse, we'll roll it back. If it's fine, we'll keep iterating on it."

Step 2: Define the terms. You ask: "How are you measuring activation? Is it completing the setup wizard, or performing a key action like creating a project?" The product manager says: "Completing the setup wizard." You confirm: "And when you say 'the new flow,' you mean the version launched on February 20th, correct?" They confirm.

Step 3: Specify the scope. You propose: "I'll compare activation rates for users who entered onboarding before February 20th versus after, controlling for acquisition channel and device type. I'll look at the two weeks before and two weeks after the launch. I won't be able to isolate the onboarding change from other factors like seasonality or marketing mix, but I can tell you whether the drop is large enough to be concerning." The product manager agrees.

Step 4: Set expectations. You add: "I'll have a preliminary read by Wednesday. The main limitation is that this is observational, not experimental — other things changed around the same time, so I can show correlation but not prove the onboarding change caused the drop. If the drop is large, we should consider an A/B test to confirm before rolling back."

The brief, summarized:

• Context: Activation rate appears to have dropped after the onboarding change on Feb 20.
• Decision: Whether to roll back the new onboarding flow.
• Question: Did activation rate (setup wizard completion) decrease significantly for users entering onboarding after Feb 20, compared to before, controlling for channel and device?
• Scope: Two weeks before and after Feb 20. Segmented by channel and device. Observational, not experimental.
• Success criteria: A clear answer on whether the drop is statistically significant and large enough to warrant a rollback decision.
• Deliverable: Written summary with charts by Wednesday.

The analysis: You query the data, compute activation rates for the two cohorts, and find that activation dropped from 68% to 61% — a 7 percentage point decline. You check for confounders: the drop is consistent across channels and devices, which strengthens the case that the onboarding change is the cause. You also check whether the drop is concentrated in a specific step of the wizard and find that step 3 (connecting an integration) has a 40% drop-off rate in the new flow versus 15% in the old.

The recommendation: You present the findings: the drop is real, significant, and concentrated in step 3. You recommend rolling back step 3 to the previous design while keeping the rest of the new flow, and running an A/B test to confirm. The product manager has a clear, evidence-based decision.

This is what framing looks like in practice. The analyst did not just answer the question. They defined the question, scoped the analysis, stated the limitations, and connected the findings to a decision.