Would I feel disappointed I could no longer use Slack?
In our company, Transistor, Jon and I use Slack every day to communicate. I'm also a member of numerous groups that use Slack.
When I initially started using Slack, I loved it. The interface felt playful and fun. Plus, all my peers were using it.
But now, like many others, I've grown weary of it.
However, my weariness isn't enough to make me stop using it. I wonder: how disappointed would I be if it went away?
In 2009, Sean Ellis used this same question to determine if Dropbox had product/market fit:
"How would you feel if you could no longer use the product?"
He gave respondents four choices:
Not disappointed (it really isn't that useful)
N/A (I no longer use the product)
His benchmark for product/market fit was if 40% (or more) of respondents replied that they would be "very disappointed if they could no longer use the product."
Using the survey is considered a "best practice" for Product Managers, analysts, and founders. I've personally used it for different startups I've worked with. I've also recommended it here, in the past:
It seems especially helpful for early-stage products.
But, as I observed above, the results might not be as useful for more mature products.
The survey has limitations that product people should be aware of.
First, it asks customers to guess about their future feelings.
As Rob Fitzpatrick points out in his book, their answers could lead you down the wrong path. He points out that humans are generally bad at forecasting their future behavior.
When a customer responds to your product survey and says: "I'd be very disappointed if your product went away," they're guessing. In the future, if your product does go away, they might find that it doesn't bug them that much!
For example, even though I rely on Slack every day, I might be relieved if it went away. Not having constant interruptions and notifications might make my life better.
Likewise, I remember thinking I'd be upset when Microsoft acquired Sunrise Calendar and announced they'd be shutting it down.
But, when they did remove my account, I just moved to Google Calendar, and I was fine. Any feelings of disappointment disappeared pretty quickly.
In "The Mom Test," Rob advocates for asking a different type of question:
"Ask about what they already do now, not what they believe they might do in the future."
He doesn't mention the Product/Market Fit survey directly, but I believe the principle applies here as well.
Instead of asking, "how disappointed would you feel?" we could ask:
"Has the product ever been unavailable?" (Due to an outage, or other reason)
"How did that make you feel when you couldn't use it?"
"How did it affect your productivity?"
"Did you search for alternatives?"
An outage is a real event in the past. We don't have to guess about how we'd feel, we know.
You can search "Slack is down" on Twitter and see tons of examples of how people react.
The most common reaction to Slack being down?
"Oh cool, I can get some work done!"
This is the kind of sentiment that may not show up on Product/Market Fit Surveys.
It sounds like many folks would be “relieved” if they could no longer use Slack.But that doesn't mean they don't have product/market fit. Their customers will continue to use Slack because it’s entrenched in their company culture and processes.
My friend Derrick Reimer learned how rooted Slack users were in their current behavior when he tried to build a competitor:
The gap between interest and implementation was of canyon-like proportions. Small teams didn’t seem that compelled by Level. In follow-up conversations, I discovered that Slack was at most a minor annoyance for them. Suboptimal? Yes. Worth going through the trouble of switching? Probably not.
To me, this "allegiance to the present" is a clear sign of product/market fit. Folks are annoyed by Slack, but they're not willing to give it up.
Our allegiance to products is often irrational:
People will buy a Tesla because they're fans of Elon Musk.
I've continued to use Adobe Fireworks CS5, out of habit, despite it being 10 years old.
Dan Ariely is a behavioral economist at MIT. In his book, he spells out our fallibility:
Standard economics assumes that we are rational. But, we are far less rational in our decision making. We all make the same types of mistakes over and over, because of the basic wiring of our brains.
We often design surveys assuming that humans will answer honestly. But Ariely's viewpoint is congruent with Rob Fitzpatrick's:
Individuals are honest only to the extent that it suits them (including their desire to please others).
When doing product research, we need to assume that folks are being dishonest (or at the very least, can't predict how they'll behave in the future).
So if we can't rely exclusively on the product/market fit survey, what other tactics can we use?
The most important step is to listen to your market. Grab a copy of The Mom Test, and start asking questions like:
What solution are you currently using?
How did you find it?
What else did you try?
What convinced you to try it?
Don't ask future-based questions ("Would you use this?"). Ask about what they're already doing, or have already tried.
You're looking for lots of evidence that people are searching for, trying, and buying solutions that perform a similar function to what you've built (or are thinking of building).
Also: don't lead the witness! Markets pull solutions towards themselves naturally; you shouldn't have to pull the market towards your solution.
In most cases, if you don't have enough information, it's because you haven't had enough conversations with potential customers.
Brian Balfour recommends looking at three business metrics:
If you're growing at a healthy pace, retaining most of your users, and are seeing recurring usage from those users, Brian thinks that's a good sign of product/market fit.
I like this framework because it shows the importance of choosing the right market:
Top-line growth comes from a market that's hungry for a solution.
Retention and usage are both signs that you've built a good product for that market.
I hope this is helpful!