To message or not to message new users, that was the burning question facing the team at Magoosh, so they used Intercom to find out what would be more effective.
What follows is a guest post by Sharat Akasapu, a product manager with Magoosh, an online test-preparation startup located in Berkeley, CA.
Student feedback fuels our company forward. As the product manager at Magoosh, a provider of online test-prep services, I can tell you from personal experience that we learn from our students everyday — how to improve our product, what to add, and what to change. Without them, we’d be lost when deciding on what next steps to take and we’d probably build a product based solely on intuition, something we never want to do. That’s why it’s crucial that we maintain regular contact with our customers. We touch base with them every day to ask for feedback, ratings, and suggestions. Intercom ensures we are able to do that in an efficient and organized way.
How do we use Intercom?
We send students emails asking them to use our mobile apps, and we’re able to tailor each message based on whether they have an Android or iOS device.
Intercom shows us when students have answered more than 50 questions on our app, so we can send them follow-up messages asking them to rate their experience.
We can track how they’re using Magoosh, which allows us to spot more active students and send them access to free resources, like study plans and extra guidance.
We can also run quick and easy product experiments using Intercom, which gives us extra in-app feedback on how to most effectively communicate with our students. And that’s huge! More customer contact = more feedback = new and improved products.
In fact, a recent product experiment with Intercom taught us how to change our customer onboarding tactics and resulted in more conversions. Here’s how we did it:
Background:
Magoosh has “free trial” users and “premium” users. Our free trial users get access to a portion of our premium materials, but not all of it — like a preview. It’s our hope that they’ll enjoy using the free materials so much that they’ll decide to go ahead and purchase the full product later on. But creating those conversions requires a bit of customer outreach work on our end, as well.
So, we came up with a hypothesis: If we send trial customers a welcome onboarding message when they first log into a Magoosh product, they will be more likely to purchase premium accounts in the future.
And we put it to the test.
Issue we faced:
We knew Intercom allowed us to test the effectiveness of one message vs. another message (we even read their blog post about it here), but it didn’t seem like there was a way to test the effectiveness of showing a message vs. not showing a message. In other words, we could send a “Welcome to Magoosh!” email to five new students, while not sending any message to five others, but how would we know which tactic was more effective?
How we solved it:
Instead of spending valuable engineering time developing a brand new A/B testing feature, we added another attribute to Intercom and called it “Even”. If the student’s user ID was “Even,” they were assigned a YES. If it was “Odd,” they were assigned a NO.
Then, we sent a welcome message to only students that had a YES, or our “Even” students. This way, we had split our users into two separate groups.
What happened:
Over the course of two weeks, we found that students who saw the welcome customer onboarding message (our “Even” students) converted 17% better. That was awesome news! That meant more revenue for our team as a result of a quick message.
How this could benefit other companies:
Intercom can be used to run simple A/B experiments without having your internal engineering team spend valuable time building new features. If your experiment works out, your team can then evaluate whether it makes sense to spend more time to build it directly into your product.
Give it a try yourself and drop us some feedback in the comments. If you want to learn more about our culture and how we build products at Magoosh, check out our blog at magoosh.com/blog.
Learn more about how Intercom can help with customer onboarding.
Got A Great Use Case?
If you’ve achieved success with Intercom and would like to share it on our blog, drop a mail to editor@intercom.com and we’ll get talking.
Comments