Expediting User Testing @ Meta / Facebook Advertising
Emphasizing: Standardizing / scalable processes, concurrent testing, unmoderated UserTesting, and to-the-point highlight videos.
Ads Manager (AM) is the core of Facebook’s (now Meta) revenue generating business model. AM powers the experience of small and large advertisers who want to expand their reach through Facebook, Instagram, and WhatsApp platforms. The business tools have more than 100,000 business users (advertisers) on a daily basis. These range from mom-and-pop shops to global fortune-500 brands.
In 2020, we had the opportunity to supercharge the existing team of 20+ researchers by continuing to build out a rapid user testing platform. The goal and result was to ensure the product teams were speaking / interviewing / prototyping with their end users on a weekly basis. We supplied the ability to identify the target user group, form the testing plan, execute, synthesize and deliver insights & recommendations. All within 2-week turnaround. Often with 3-4 projects running at a time.
- Launched Rapid Research program to supercharge Facebook’s core advertising business
- Improved speed and scale of 20+ embedded researchers (and associated product partners)
- Increased monthly engaged business user task completion – Website set: +37.89%, WhatsApp linking: +100.34%, custom CTA creation: +100.62% (project specific)
- Conducted 30+ qualitative studies (moderated & unmoderated) with 150+ business users across desktop, mobile, and native app advertising experiences
- Established framework for high-impact video highlight reels to support recommendations
First, our mission (as it often is) was to establish foundational working relationships with our product partners in the organization. We had to communicate our team makeup, levers of support, key deliverables, and approach to partnering. In this case, our rapid research team was meant to support embedded researchers within their own product teams (Designer, Engineer, Data Science, Product Manager, Content Strategist, Marketing Manager, and Researcher). We supported 20+ researchers who had worked on various products (advertising onboarding, lightweight ads, mobile ads, native app ads, desktop Ads Manager, etc.). One of the absolute best learnings from this job was working closely with so many researchers to trade tips, tricks, and insights.
Second, we tasked ourselves with setting up a repeatable research processes. Our team had access to some very useful industry tools, such as: UserTesting, UserZoom, dScout, Optimal Cardsort, and more. However, in order to take full advantage, we had to employ scalable internal tools. To do this, we spent a portion of our time as a team to create & manage research protocols, recruiting screeners, task management, project tracking, and key deliverables. This helped us be streamlined and aligned in our approach to tackling new projects. During this position I was able to go deep on the development of frameworks that I’ve since used in other project environments. Simple screener practices, share-out structures, interview guides, tactical question approaches, and master synthesis spreadsheets are some examples of these tools in my belt for future engagements.
Third, we not only had to deliver value to the project / product teams with which we partnered, but we also were tasked with boiling up opportunities for the broader organization to tackle or consider. We had monthly engagements with senior leaders that looked to our lean / nimble research team for new opportunities we were seeing across all the products we engaged. As I mentioned before, we had the privilege of supercharging researchers and their product teams across a multitude of user touch points. This gave us broad-based insight as well as on-the-ground experience of what was working, or not in our business user experiences. We took advantage of our position in the organization to highlight larger concerns and areas that need investment. This is something I look to do at any organization for which I work. Not only provide value at the tactical product level, but also to senior leadership / strategic level.
Finally, near the end of my tenure, we were tasked with scaling to support even more teams. This underscores the business value we were driving. Leaders were beginning to ask our team to support more product teams with their respective business and user touch points. We took this as a compliment; however, it intensified the need to identify more efficiencies in our processes. Some specific examples of changes we made as a result of taking on more projects was continued refinement of our intake method (Google site questionnaire and automated response system) and developing a cadence of interchangeable job roles within the team (lead intake, lead tracker, lead presenter, lead synthesizer, etc.). As a result of this job’s experience, I’m more confident to onboard to new teams with ideas already in mind on how to scale and operationalize UX research.
“I’m amazed to see how quickly you immersed into the problem space we were trying to solve, for moving feedback into actions while creating the research plan, for being a great facilitator, for helping my team to see the value of research during the playback and for showing me the world of unmoderated research.” – FB user researcher
“#thanks Paul for your very thoughtful insights and recommendations on the PCM project. Loved working with you throughout – you take so much initiative, responsibility and go really beyond what’s been asked. The team will benefit immensely from both, the immediate term recommendations and the food for thought you have planted in your presentation.” – FB user researcher
“I took Paul’s insights database framework and immediately applied it to my business. My research lead skips some of the steps that Paul tackles in every project…and I’ve gone back to the basics to make our practice more organized. I’m really grateful for his willingness to methodically walk us through his methods.” – UX research mentee