Exploratory & Ealuative Research @ Shopkick
Plugging the Leaky Bucket: Shopkick's 76% User Drop-off Challenge
Overview
As team-of-one researcher, led comprehensive onboarding redesign that addressed a 76% user drop-off rate within 30 days. Through data-driven insights and cross-functional collaboration, reduced onboarding time from 2 minutes to 45 seconds while establishing trust-building framework for new users.
Key Impact: Transformed onboarding experience from confusion-inducing barrier to guided introduction, directly addressing the core retention challenge facing the business.
The Challenge: When Growth Doesn't Stick
Business Context
Shopkick faced a classic "leaky bucket" problem: 50,000 weekly app installs, 8,300 activations, but only ~2,000 users remaining after 30 days—a devastating 76% drop-off rate. While the marketing team could drive downloads, the product couldn't retain users long enough to demonstrate value.
The Core Problem
The Symptoms: Users were confused about the value proposition, couldn't find earning opportunities, and most critically, didn't understand how the app worked after onboarding.
My Hypothesis: The onboarding experience was creating barriers instead of building confidence, leading to immediate abandonment.
The Stakes: Without solving retention, all marketing investment was essentially wasted.
My Approach: Research as the Foundation for Product Strategy
Building Research Infrastructure as Team-of-One
Since I was the only researcher, I had to establish both research processes and strategic direction simultaneously:
Process Innovation: Implemented research guidelines and frameworks to enable cross-functional teams to incorporate user insights into their work
Strategic Leadership: Led vision and goal setting sessions to align teams around user-centered metrics beyond just downloads
Reframing the Problem Through Data
Instead of: "How do we make onboarding faster?"
I asked: "What's causing users to lose trust before they experience our value?"
My Multi-Method Strategy:
Quantitative foundation: Collaborated with Data Engineers to identify drop-off points
Cross-functional knowledge capture: Led brainstorming sessions to surface internal assumptions
Competitive intelligence: Analyzed how other complex apps handle first-time user experience
Qualitative deep-dive: 1:1 interviews to understand user mental models and trust barriers
Link to beginning state of buyflow.
Critical Discovery: Trust Must Be Earned, Not Assumed
1. The Data Told a Story About Trust
Key Finding: The biggest drop-off occurred at "gender and age" screen, users were abandoning when asked for personal information without context.
The Insight: Users weren't just confused about functionality; they were actively distrusting our intentions.
Quote: "Why does a shopping app need to know my age and turn on my Bluetooth? This feels like they're tracking me for reasons they won't tell me."
2. Mental Model Mismatch
Critical Discovery: Users expected rewards apps to work like loyalty programs (earn points, redeem rewards), but Shopkick's location-based earning was completely foreign to them.
The Problem: Our onboarding explained what to do but not why it worked or how it benefited them.
Impact: Users completed onboarding but had no framework for understanding their next actions.
3. The "First Action" Paralysis
Key Finding: Even users who completed onboarding were paralyzed by choice on the main screen, they couldn't identify their "first action."
Why This Mattered: We were losing users not because they didn't like the app, but because they couldn't figure out how to start using it successfully.
Cross-Functional Collaboration: Turning Insights into Action
Leading Through Influence, Not Authority
As team-of-one researcher, I had to drive change through collaboration rather than mandate:
With Data Engineering: Partnered to identify quantitative patterns that supported qualitative insights
With Legal: Negotiated to reduce required information collection until trust was established
With Design: Facilitated "How Might We" sessions to convert research findings into design solutions
With Product Management: Established new success metrics focused on user understanding rather than just completion rates
The Brainstorming Session Strategy
My Approach: Before conducting external research, I captured internal team knowledge and assumptions through structured brainstorming.
Why This Worked: It created buy-in for research findings and helped the team recognize the difference between their assumptions and user reality.
The Output: Clear alignment on what we thought we knew versus what we needed to validate.
Strategic Thinking: Solving for Long-term Success
My Framework for Onboarding Redesign
Trust Building: Only ask for information users understand the value of providing
Mental Model Alignment: Explain concepts in terms users already understand
Progressive Disclosure: Introduce complexity gradually as users experience success
Clear Next Actions: Remove ambiguity about what users should do first
Addressing Systemic Issues, Not Just Symptoms
The Temptation: Fix the obvious problems (long onboarding, confusing screens)
My Approach: Address the underlying trust and comprehension issues that caused multiple symptoms
The Insight: Users weren't just confused, they were actively protective of their personal information and skeptical of unfamiliar earning mechanisms.
Impact & Measurable Outcomes
Immediate Results
Onboarding time reduced from 2 minutes to 45 seconds
Information collection streamlined to build trust before asking for personal details
Walkthrough experience implemented to address first-action confusion
Trust-building copy that explained the "why" behind each request
Strategic Impact
Established research-driven culture in a previously assumption-driven organization
Created framework for evaluating all user-facing features through trust lens
Aligned cross-functional teams around user understanding metrics
Built foundation for addressing other retention challenges (personalization, earning opportunities)
What I Learned
1. Research as Organizational Change Agent
As team-of-one, my role wasn't just to conduct studies, it was to build research-informed decision-making into the company culture.
2. Data Partnership Amplifies Impact
The most powerful insights came from combining quantitative drop-off data with qualitative trust insights. Neither alone would have driven the changes we achieved.
3. Internal Knowledge Has Value
The brainstorming session revealed that the team had valuable intuitions about user problems, they just needed validation and prioritization through research.
4. Trust Is the Foundation of User Experience
Before users can appreciate your app's unique value, they need to trust your intentions. This is especially critical for apps that request location data or personal information.
Systems Thinking: Retention as Strategic Advantage
The Bigger Picture Impact
User Journey: How does improved onboarding affect long-term engagement and lifetime value?
Business Model: How does better retention change the economics of user acquisition?
Competitive Position: How does user understanding create defensible advantage in the rewards app market?
Team Dynamics: How does research-driven culture affect product development velocity and quality?
Future Considerations
Personalization Strategy: How can we use onboarding insights to improve the core app experience?
Trust Framework: How can we apply trust-building principles to other features?
Measurement Evolution: What metrics best capture user understanding versus just behavioral completion?
Validation Through Implementation
The redesigned onboarding experience directly addressed the core retention challenge, providing a foundation for tackling other "leaky bucket" issues like personalization and earning opportunity discovery.