MY ROLE
I worked on this project as a lead and sole designer and collaborated with a team of product managers, front-end developers, back-end engineers, and data scientists. My primary areas of responsibility included:
• Product vision and design strategy
• End-to-end UX workflows
• User research
• User flows, wireframes, prototypes, high-fidelity mocks
• User testing and validation
01. Overview
Zulily is an event-based e-commerce platform with millions of customers. We have vast information on customer demographics, shopping habits, and engagement metrics. Zulily events are highly curated and discounted shopping experiences that only last a few days so it’s critical that we’re utilizing our data effectively to make sure event campaigns are going to the right customers.
Marketing teams struggled with accessing and utilizing our data, but now we've developed a new tool that makes it easier than ever to create customer segments and run successful campaigns.
02. The Challenge
How can we improve the experience of creating high-value, targeted audiences for our non-technical marketing team members? How can we democratize the vast amount of data we have at our disposal and make it more accessible?
High-level objectives
1️⃣ Make complex data manageable and audience creation simple
2️⃣ Integrate seamlessly into existing workflows
3️⃣ Increase revenue through higher conversions, click-throughs, and purchase rates
4️⃣ Simplify workflows to allow Marketing Specialists to focus on more high-value, creative tasks
5️⃣ Decrease the number of engineering support tickets
03. SOLUTION
How can we create a customer dimension-based tool that will allow our marketing teams to combine attributes at scale and generate highly targeted segments to improve the customer experience and generate revenue?
The current means of creating an audience required writing SQL queries in a code editor. This created a barrier for most Marketing Specialists.
Research
I facilitated user research sessions with cross-functional partners to identify user needs, business requirements, and technical constraints. These sessions helped us prioritize our focus for an MVP and create a shared perspective on what a solution could look like.
Key takeaways:
🤨 Creating an audience is too technical, inefficient, and poorly documented
🤔 Customer attributes are ambiguous with poor naming conventions and taxonomy
📉 Engagement campaigns are not being deployed to optimal audiences, leaving millions in potential revenue on the table
🤩 High anticipation by marketing team members for a new tool as they could easily see the potential payoffs of being able to quickly generate unique audiences for their campaigns
Existing Audience Analysis
With the assistance of an engineer and data scientist, we analyzed the hundreds of existing audiences that have been created in the past. We used results to determine which attributes we should prioritize in making accessible for our users.
Results of the existing audience analysis
Creating a Taxonomy
Organizing customer attributes into logical, easy-to-navigate groupings was crucial to the successful launch of the Audience Builder. I ran a virtual card sorting exercise with 10 potential users of the tool, the results of which would act as a baseline for the final attribute menu taxonomy.
Similarity matrix from attributes card sort exercise
Initial attribute menu taxonomy
Ideation
With the goal of making our customer attributes easily accessible and turning a complex process of writing SQL queries into a visually intuitive one, I sketched out some rough ideas for how we might achieve those goals.
Some potential concepts we explored included:
💡 Drag-and-drop interface
Users can choose what attributes they want to include in their query and drag them onto a query canvas
Users can choose what attributes they want to include in their query and drag them onto a query canvas
💡 Sequential form-based UI
Provide users with a sequential series of form and input fields to specify the query components step-by-step. Each form would represent a part of the query.
Provide users with a sequential series of form and input fields to specify the query components step-by-step. Each form would represent a part of the query.
💡 Natural language input
Allow users to naturally write out what they want their query to include and use natural language processing to interpret user intent
Allow users to naturally write out what they want their query to include and use natural language processing to interpret user intent
Concept sketches for form based UI
Concept sketches for drag-and-drop UI
Wireframes & Concept Validation
As a team, we decided on which concepts we felt were most promising, then created some quick wireframes to validate with our users. The drag-and-drop concept struck the right balance of providing the flexibility our users needed with the technical feasibility that would allow our developers to build out the solution.
Audience Management View
Query Builder Concept
Query Builder Concept
Wireframe User Feedback
The core of this new tool was going to be the query builder itself. I honed in on the query feedback to iterate and improve on the solution I was proposing.
Wireframe user feedback
User Flow
The audience builder is part of a larger campaign creation workflow, so we needed to map out how this tool would fit into that workstream.
User flow for creating an audience for a new marketing engagement campaign
Prototype & User Testing
Based on our research and wireframe feedback, I created a functional rapid prototype to validate concepts with users. I ran user tests with 6 Marketing Specialists and had them go through a series of tasks that would be required to build an audience.
What we hoped to learn:
𝐐. What is the users' initial reaction to this screen? What do they do first?
𝐐. How are users going about finding attributes they want to use? Search? Browse?
𝐐. Is the drag-and-drop interaction model understood and intuitive? Do users understand how to use this functionality to build a query?
𝐐. Is the structure of the query readable and do users clearly understand what they are building?
Prototype used for user testing
Positive/Neutral feedback
• 6 out of 6 reacted positively to the clean and familiar look and feel of the UI.
• 4 out of 6 users' first instinct was to use the search functionality for attributes.
• 6 out of 6 users easily discovered the drag-and-drop functionality of the attributes.
Negative feedback
• Drop targets were not very obvious and caused users to fail to successfully add attributes in the right locations.
• The visual functions made sense to users, but the query structure itself was still a little confusing for users. "And/Or" conditional indicators and visualization of groups were the biggest sources of confusion.
"This is so much easier to use and feels like a safe way to experiment with quickly making new audiences."
- Marketing Specialist test participant
"This will definitely get users unfamiliar with creating audiences up to speed quickly and will reduce training time for new hires."
- Marketing Specialist test participant
04. Launch
We evaluated the user needs, business needs, and technical requirements to prioritize the final list of features we needed for the V1 launch of the tool.
V1 Requirements
✅ Ability to manage and modify created audiences
✅ Provide a drag-and-drop UI to visually build a query
✅ Visualize functions for easy manipulation in the UI
✅ Ability to schedule audiences to run on specific dates/times
✅ Ability to assign usage context to an audience
✅ Create a searchable index of attributes
✅ Provide an easy way to see attribute definitions
Post-launch Follow Ups
➡️ Real-time ID counts
➡️ Attribute conflict validations
➡️ Smart attributes based on current attributes in use
➡️ More robust audience performance reports
Final Screens
Although we were adhering to our existing design system, this project required several new components and design patterns to be developed. Working closely with my engineer partners, we determined what existing design system patterns and components we could leverage, and what functionality would require new components. Below are examples of the final screens and components that we launched with.
Audience List
Audience List Scheduler
New Audience Modal
Query Builder Default View
Query Builder Canvas with Functions
Query Builder Scheduler Component
Query Builder Tag Component
Audience Query Builder Demo
Screen capture of the launched Audience Builder in use
05. Results
The Audience Builder was quickly adopted by our Marketing Specialists after launch and this gave us an opportunity to perform quick analysis on the success of the tool.
After launching, we observed the following outcomes:
High adoption and usage
Dozens of new audiences were being created in the first few weeks after launch. The feedback was overwhelmingly positive.
Dozens of new audiences were being created in the first few weeks after launch. The feedback was overwhelmingly positive.
Unlocked the potential for more robust audience testing
The customer dimension-based approach allows Marketing Specialists to observe user trends, react quickly, and make adjustments to the marketing segments.
The customer dimension-based approach allows Marketing Specialists to observe user trends, react quickly, and make adjustments to the marketing segments.
Increased open and click-through rates
The ability to more precisely personalize target outbound messaging based on a deep set of customer dimensions led to improved metrics across marketing engagement campaigns.
The ability to more precisely personalize target outbound messaging based on a deep set of customer dimensions led to improved metrics across marketing engagement campaigns.
Workflow optimization and new opportunities
With the friction removed from audience creation, new audiences could be validated quickly and customers can be engaged at the right moments in their Zulily shopping experience.
With the friction removed from audience creation, new audiences could be validated quickly and customers can be engaged at the right moments in their Zulily shopping experience.