Peter Drucker famously quipped “if you can’t measure it, you can’t manage it” and that’s especially true when it comes to feedback from your customers.
But what if your surveys are so long they distort your metrics? Or, suppose only outliers take your surveys so your data doesn’t represent the majority of your customers? Measurement is not the end in itself. Your metrics need to capture what’s actually going on. One way to get more accurate data is to shorten your survey.
Case Study: Alaska Airlines
I recently flew Alaska Airlines from San Jose to Portland. Afterward, they sent me a request to take their survey. Because my seat was filthy, I figured taking their survey could be a good way to let them know. Besides, I have often thought, Alaska is pretty good, if only they were more attentive to cleaning.
The email request said, “this should take about 15 minutes of your time,” and when opening the survey, it called the survey “brief”.
The exact quote from the first screen of the survey was: “The fifteen-minute survey will help Alaska Airlines improve its customer satisfaction and service levels”. It’s not a strong opening, but it got much worse.
94 Questions is Way Too Long!
This survey had multiple flaws, but here I’ll unpack only those related to survey length.
With a staggering 94 questions (more if I had answered some questions differently) completing the survey in the purported 15 minutes would require maintaining a no-distractions-pace of a few seconds per question.
In reality, it took me about half an hour to complete the survey, but I probably stopped a few times. After all, 94 questions is daunting!
If you’re thinking that your company would never fall victim to lengthy surveys, think again!
Two years ago we completed the Point-of-Purchase survey (PoP) Survey Study for the Retail Sector. In it, we covered retailers such as Ace Hardware, Nordstrom, and Walmart. One of our findings was that most retailers desperately needed to shorten their surveys.
Why Long Surveys Happen
I’ve studied surveys for over a decade, and I’ve found that there are two reasons surveys balloon out of control. The first, is that every department has ‘just a few’ questions they want to ask. With each department adding their two cents, you can see how quickly things can get out of hand.
The second, is that as much as executives say they want to listen to their customers, the reality is that it’s highly unlikely Alaska’s CEO Bradley Tilden has anything to do with his customer survey program. If he knew what the Alaska survey experience is really like, he might demand a shorter survey.
The Problems with Painfully Long Surveys
Aside from a bad user experience, there are several reasons your company should shorten your survey.
- Low Response Rates: Not many customers are willing to take a long survey, so you miss out on a lot of data! They might open the survey and quickly close it or remember from the last time how long your surveys are and vow to NEVER take your survey again.
- Low Completion Rates: Fatigued customers will give up in the middle of the survey or may start randomly choosing answers just to get through it. You either miss out on data or the data you get is junk!
- Respondent Bias: The data you do get is from customers who have the energy and time to take such long surveys.
- Tarnished Brand: A poorly-designed survey shows your customers that you’re not invested in truly listening to them and their concerns.
Fortunately, with a little bit of effort you can shorten your survey, prevent survey fatigue, and get high-quality data. Consider these solutions:
4 Solutions to Shorten Your Survey
Solution #1 | Incorporate Operations Data
Sometimes companies ask questions when they already know the answer. If a company has your email, they have at least your basic CRM data. However, many surveys still ask these types of redundant questions. For example, Alaska Airline’s survey asked for basic facts it already knew such as:
- Did you fly first class, premium, or economy?
- How late did your flight arrive or was it on time?
- Did you pay for a checked bag?
Unquestionably, Alaska Airlines knows the answers to these questions. What they (and other companies) could do instead is customize the survey and link operations data to the survey itself. When companies don’t take the time to incorporate operations data, they ask customers to spend time telling them things they already know.
We customize surveys every day at Interaction Metrics, and other companies do this too. So it’s hardly asking a lot of Alaska to do this, it’s more like rising to customer’s expectations of what a survey should be.
We customize email survey requests, the survey text (inserting incident numbers, etc.) and even the questions are customized into each survey link. This small investment in best practices helps customers want to take your survey (think high response rate). In addition, it keeps them engaged with your survey (think better data and higher completion rates).
Solution #2 | Apply Logic Gating
Efficiently-used logic gating is one of the most helpful tools for maintaining relevance with customers and asking the fewest possible questions. Logic gating simply means that your survey is dynamic and adjusts questions based on customers’ answers. For customers who have limited areas they want to comment on, logic gating dramatically reduces survey length.
Alaska does use some logic gating but misses several key opportunities to maximum its potential. For example, one section asks the customer to select all competitors they’re familiar with and then rate each of those competitors. There are many better ways to handle this.
Solution #3 | Avoid Repeated Questions
While it’s important to focus on the individual aspects of a customer’s experience, it can be easy to get too detailed.
In Alaska Airline’s survey, one screen asked respondents to note every part of the plane they felt wasn’t clean. The survey listed 6 different areas of the plane, asking respondents to remember how they felt about details like the walls or the tray tables.
While this question was only asked if you reported having a negative experience with aircraft cleanliness (good logic gating!), asking customers to answer questions about their environment down to the seatback pocket is excessive. This entire screen could be replaced by an open-ended question asking why exactly they felt it wasn’t clean.
Another question asked customers to rate the boarding process. Then on the same screen asked them to rate “waiting at the gate before boarding”. Do you really need that level of granularity? Do your customers really think that hard about what you mean by the difference between the boarding process and the boarding gate?
Solution #4 | Recognize Human Frailty
Remember that respondents are customers with limited time and mental energy to devote to your survey.
One question in the Alaska Airlines survey asked respondents to remember the last 10 flights they had taken in the last year and which airlines they flew with. It can be hard to recall specifics about what happened a few days ago, let alone in the past year. As I often say ‘’customers don’t think about you nearly as much as you think about you”. Keep it simple! Ask relevant questions about recent experiences to avoid draining your customers.
Checklist to Shorten Your Survey
- When developing your customer survey, make sure you take it yourself. Can you make it all the way through, or do you lose focus? Your customers have less patience for your survey than you do, so take that into consideration.
- More detailed information is typically better but comes at a cost to your customers. That’s why you need to think hard about the purpose behind each question.
- Can you ask it more succinctly?
- Does it overlap with a question you’ve already asked?
- Are you sure you will use the information you are asking for to improve the customer experience?
Bottom Line
Shorten your surveys to get the most accurate, relevant data. If CEOs like Bradley Tilden knew about the length of his survey, we think he would intervene immediately. So start now and shorten your survey today!
Let’s make the world better with better customer listening!
