Skip to content
Fast Company Executive Board

The Fast Company Executive Board is a private, fee-based network of influential leaders, experts, executives, and entrepreneurs who share their insights with our audience.

A primer on finding a high-quality online survey sample

It’s imperative to ensure your results aren’t tainted intentionally or by accident.

[Andrey Popov/Adobe Stock]

Profile photo of Sumit Aneja

Online surveys are one of the fastest and most cost-effective ways to get customer feedback that can impact what happens in your business. The quality of the survey sample has to be high, however, for these tools to achieve their maximum value. There are offerings like online research panels, which equip you with everything you need to ensure your samples are high-quality.

THE HURDLES THAT CAN LIMIT SURVEY QUALITY

One of the biggest challenges professionals run into when they’re trying to get valid survey data is bots. These are sophisticated software programs that drive fake traffic to surveys and can complete answers like a human would. People often take advantage of them to complete surveys that offer compensation. A White Ops pilot program found these kinds of bots can cost top brands $30 million in marketing spend in just 90 days.

Secondly, people can submit answers to a survey multiple times. This can happen when an individual seeks to intentionally throw off the results, wants another shot at the compensation offered, or experienced some technical issue and resubmitted by accident.

Both bots and people who fill out surveys multiple times reduce the validity of survey results. To quantify the size of the problem, understand that somewhere between 15% and 30% of all market research data is fraudulent. Company leaders end up making decisions based on inaccurate or misleading data. Those misinformed decisions can cost businesses millions of dollars.

FOUR TOOLS TO IMPROVE THE ACCURACY OF YOUR SURVEYS

Even though survey falsification and duplication is widespread, you have multiple options for combating them:

• IP detection: An internet protocol (IP) address is a numerical label that identifies the device interface you’re using when you’re online (e.g., this is Joe’s iPad). It also reveals exactly where you are located on a network so providers know where to send the data you request. You can’t connect to a network, including the internet, without an IP.

Allowing multiple survey responses to come from the same device or IP address isn’t always a bad thing—for example, maybe you’d like everyone in a family to be able to participate. But limiting users to one survey per IP address can ensure people don’t duplicate responses or create a multiple panelist account. IP detection tools can identify where suspicious responses are coming from so you can take action to block the user.

• Bot detection: Probably the most well-known tool in this category is a completely automated public turing test to tell computers and humans apart (CAPTCHA). CAPTCHA requires a user to respond to a challenge question or test, such as choosing all the images out of a set that include a specific item. You can embed these across your survey to stop bots from filling the survey out, or at the point of registration to stop bots from signing up.

Other forms of bot detection include open-ended survey questions, tracking time and date stamps, and flagging responses that have been completed suspiciously fast or that seem to be based on access to information the respondent shouldn’t have.

• Proxy detection: Normally when you’re online, you connect directly to whatever website you want to go to. A proxy server, however, acts as a middleman to direct traffic from you to the site and from the site back to you. It can protect you by filtering content or blocking sites. Virtual private networks (VPNs) function similarly, but they also encrypt your information.

Fraudsters who manipulate surveys or other data can use anonymous proxy servers and VPNs to hide their device or IP address. Proxy/VPN detection tools analyze a user’s connection to determine whether they’re trying to cloak their identity.

• IP/connection risk score: An IP address can reveal lots of data about a user, such as where they are operating geographically or who their internet service provider (ISP) is. Companies now have access to tools that translate the data into a score that reveals how risky it is for the user to connect to or interact with the business. You can decide based on the score whether to flag a user or accept their response.

• Audience control tools: These tools allow marketers and other professionals to create their surveys and build specific survey panels limited to truly relevant respondents (e.g., male, 45-50 years old, from Wisconsin). They can combine with artificial intelligence and other services such as IP detection to make sure responses are coming from real people, and eliminate common problems like survey duplication. They also allow companies to assess responses in real time for both insights and security. Being more precise with the way the survey is built and who it reaches reduces the odds that fraudsters will gain access.

TO MAKE SMART DECISIONS, AIM FOR HIGHER SURVEY QUALITY

Online surveys are not perfect. They’re still prone to biases and require excellent identification of the target population, and it’s imperative to ensure your results aren’t tainted intentionally or by accident. Nevertheless, with the right precautions, such as IP detection or audience control tools, the validity of your results can improve. Invest in the best options that are within your budget, because accurate insights ensure intelligent decisions for your business.


Sumit Aneja is CEO of Voxco, an omnichannel software survey platform and a global market leader in the multi-modal survey software sector.