Case study
Prospect
Connecting two sides of the business
Context
In November 2017, I joined the QuickBooks Self-Employed (QBSE) team as part of a two-person ‘start-in’, tasked with experimenting into a zero-to-one, network effect marketplace. We were to connect the six million small businesses and one million self-employed customers that Intuit served but had, thus far, not brought together in a meaningful way.
Our team was comprised of a product manager and a product designer (me). We were given free-reign to test into a marketplace that helped self-employed people find work and, in turn, enable small businesses to expand their book of business, allowing them to take on more work. Prior to our joining the company, this concept had come out of multiple working sessions conducted by senior leadership, and resulting in a ‘six-pager’—a truncated version of what is commonplace in companies like Amazon. In the form of a press-release, describing a product release, as though just unveiled to the world, and its impact on Intuit’s business and the economy, it formed the foundation and initial guardrails for our process.
Premise
In short, our initial leap of faith assumption was that Intuit could build on it’s thriving network of self-employed workers and small businesses to create a platform that connected and helped them find work / workers.
We immediately dug in, interviewing internal stakeholders and researchers, who themselves had reams of data that spoke to the needs of the self-employed in helping them find work. We spoke with multiple small business owners and self-employed people in in-depth interviews, listening to their unique journeys and the pain points they experienced. We defined archetypes that gave us an initial grouping, enabling us to understand the needs of each side and the phases that their journeys mapped to.
Intital customer problem
I am newly self-employed, trying to follow my passion.
I am trying to find a regular stream of work and income,
but I don’t know where to find reliable clients who pay well and on time,
because I don’t have a network to rely on,
which makes me feel anxious and that I might fail.
Progress
We were able to pull out themes that allowed us to narrow in on our first customer problem and ideal state statements. We did competitor analysis of job seeker experiences and found gaps in highlighting mutual connections. I sketched out a quick prototype that looked at how QuickBooks Self-Employed customers, who used the app to track their business expenses, could activate a job seeker profile. I tested it via twelve, in-person tests, the results of which were… underwhelming. I created an ideal state prototype showing a thriving jobs marketplace and tested it with customers—we were making progress but a little unsure of where we were headed.
Profile prototypes
Mentorship
The founder of Intuit, Scott Cook, is still very active in the company, helping mentor teams who are in the early stages of experimenting into new business. Scott is the primary advocate for Intuit’s D4D, Design for Delight process. Many companies have their own flavor of this, Intuit’s being strongly based on the Lean Startup model, and is the prescribed approach that all projects, regardless of phase, should follow.
Being new to the company, we were following a process that we were comfortable with. Both myself and my PM partner had successfully released award-winning products in the past, following the typical product process. Our first mentor session with Scott, however, set us on a path that would change the way we would approach product, from that day forward.
After taking Scott through all the work we had done, describing the pain points we’d heard from research participants, and our comp analysis, Scott not only immediately understood it, he told us where we were going wrong. Looking back, it felt a little like a grilling—but it wasn’t. Getting to know Scott over the next year (and on subsequent projects), his no-nonsense approach in mentorship is purely in service of getting to the heart of the customer problem, and providing the team with clear next steps. Up until that point, we were of the opinion that helping self-employed people find work was the focus. Scott immediately reframed it, pushing us to focus on the hirer—the small business. He quickly recognized that our data on small businesses spoke to a clear customer problem—help hirers find people they can trust to hire.
Re-framing
We had research that pointed to a pattern that hirers would follow, using their network to find people to hire—people they could trust. This trust was founded in the connections they had with people—obvious, I know. Where we found the problem to be was when personal network connections, or ‘word-of-mouth’ was exhausted. Trust would fall dramatically and recruitment services for hirers were seen as a last resort. We heard stories from many small business owners about hiring through services where they had had less than good experiences, having jobs go badly because of bad hires—greatly impacting their business and brand.
After the first session, Scott sent us away to align on a new, narrowed customer problem statement, a primary leap of faith assumption (if found to be false would mean you don’t have a business), and challenged us to come up with ten hypotheses and behavioral experiments to run, digging into the initial core of our problem—what trust signals are needed to make a hirer feel that they have enough to hire someone they have no connection to. We were told to prioritize them in order of what was achievable, based on level-of-effort.
Customer problem
I am a hirer looking for someone to help with my business,
I am trying to find a person I can trust to represent my brand,
but once my personal network is exhausted, I don’t know who I can trust to hire,
because there is little to no accountability on the job seeker, if I have no connection to them,
which makes me feel like I may not be able to meet the demands of my growing business.
Leap of faith assumption
In the absence of word-of-mouth recommendations from their network, hirers will use a service that leverages data to find trusted workers.
Initial hypothesis
If we offer hirers variations in trust signals on a jobseeker profile, we will learn which combination of signals resonate the most.
Freedom
We sought advice on how to ship behavioral tests. With no engineering resource in our team, we spoke to numerous people. Our assumptions about Intuit (having been warned about the legalities of running experiments inside the Intuit ecosystem) was that it was a laborious process to get tests up. Period.
One interaction, however, opened our eyes, giving us the freedom to do whatever we wanted. A leader from the research team asked us, ‘what’s stopping you from getting a test up today?’. Their push was that we didn’t need to test within the Intuit ecosystem. We were testing a new business. We simply needed to gather data on our hypotheses.
First test
Test
Google AdWords to Unbounce landing page
Goal
3% of visitors submit their email address
Result
14.79% conversion rate
399 page visits
Experimentation
With the question of ‘what can you learn today’ perpetually spinning in our heads, we did analysis of the types of trust signals products used to trigger transactional behavior. Star ratings, reviews, repeated engagement, etc. We stood up a landing page, branded ‘Prospect’, scrappily built using Unbounce, offering ‘designers (we thought we’d start with a familiar industry) that you could trust to hire’. We used Google AdWords to drive traffic via keywords that spoke to our value prop. We launched experiment after experiment.
Local corkboard experiment—0% conversion
Craigslist ad
Multivariate test
Control, 4.63%
11.23%
4.9%
5.17%
6%
5.69%
29% open rate, 5.4% click through, 0% message / hire rate
Trust signals
We quickly found that proof of past success was the signal that won out in our multivariate test. We sent follow-up emails to visitors that had provided their contact email, looking for designers. I added fake profiles of colleagues (that agreed to be part of the experiment, including myself) with the understanding that, if we received contact, I would pose as that person to learn about the hirer’s needs. After a short time, the experiment was yielding results. We were seeing relatively good engagement but we were also noticing that our one-page site would quickly lose visitor trust if they tried to dig deeper on the business. The home to our trust test was losing customer’s trust.
Destination built using Squarespace
Established
We quickly moved to a template in Squarespace, faking an established business with Home, About and Contact pages. We replicated the designer preview pages from our Unbounce emailers and pushed traffic, via AdWords to our new offering, getting into bidding-wars on keywords with UpWork and ZipRecruiter. Within days, we were receiving contact volume to our designer profile pages. A big insight we gathered was that our previous iteration, offering three designers that fit the hirer’s needs, saw drop-off. A hypothesis I posed was, based on past research where we heard that businesses just wanted to be given a recommendation through word-of-mouth—I suggested we just offer one designer. We’d suggest that we’d done the work and found the right person. Nothing else was needed.
Email 67% open rate, 42% click through rate, 17% hire rate
Squarespace
We tried it, sending customers a single candidate for them to hire, and it worked. I was hired twice. I actually did the work for one job, a YouTuber looking for brand design, and a window company, based in California, looking for a designer for a month long contract. The second role I couldn’t take but in talking to the hirer, I was able to ask about their experience with ‘Prospect’. The response was fascinating—“…I like it. I was looking for someone for this role. They did the work and they sent me you.” This wasn’t validation but an indication that we were on to something.
Other surprises and key learnings from getting hired:
- Location is critical for some hirers, even for jobs that could be done remotely
- As a new contractor, I had no idea what to charge
- I thought I wasn't going to be paid, after multiple follow-up emails
- Communication varied widely from hirer to hirer, from email, to text, to phone
Hired
After personally getting ‘hired’ twice, we wanted to close the loop, matching a self-employed person with a small business looking for someone they could trust to hire. We quickly expanded our site, allowing self-employed people to fill out a form with their details to be matched with hirers. Again we used AdWords to draw in the contractor side of the marketplace. With the basic information we gathered, we pulled together profile pages for them, manually pulling portfolio imagery and sending their profiles out to hirers who had contacted us for help. We were getting close...
Our internal stakeholders, who we had had regular check-ins with, and excited about our progress, were keen to see some results—in-product. We were asked to create an ideal state, based on what we knew to be true from our research. Our mentors (Scott included) were pushing us to stay the course with our research but we were asked to stop and bring a concept to product. We did. We showed an end-to-end, manifesting in QuickBooks (hirers) and QuickBooks Self-Employed (contractors) being connected in a thriving marketplace.
Ecosystem
This generated a great deal of excitement, even though we had not yet closed the loop through our experimentation. Post share out, we were paired with an internal team who were looking at unravelling the data we had across our six million businesses and one million self-employed customers. In many ways, this made sense as we were still to prove out one of our internal leap of faith assumptions—that intuit has the data to make meaningful connections, when word-of-mouth has been exhausted.
For better or worse, being brought back into the ecosystem and having to experiment in the confines of the brand, our pace of experimentation was slowed dramatically. Our focus was on proving out an unproven marketplace, inside the QuickBooks ecosystem. We made slow progress, testing with limited internal tools. We tested completion rates of profile creation on the self-employed side and hirer engagement with job posting. The results of which were inconclusive.
Data connections
Not too long after, I moved to a different team but the project continued, driven by engineering and, although progress slowed, the initiative has since been incorporated into Intuit’s five-year strategy. A huge achievement for a scrappy start-in!
As mentioned before, we were in regular AdWord bidding wars with UpWork and ZipRecruiter. At that time, no one was talking about using data connections as a replacement for word-of-mouth. As time went on, these companies, and others, repositioned their value props to focus on data—driving connections, helping hirers find people they can trust. It was fascinating to see this evolve, and for our research to land on an unsolved customer problem.
Although our work didn’t end in a marketplace being released to customers, the unearthing of, through behavioral experimentation, the true customer problem was a success, but just the beginning.