Data collection and quality
Surveys
Once you have launched your survey, it is important to actively manage the data collection process to ensure the information you gather is reliable and useful. Quality control at this stage will reduce the risk of problems later on during analysis.
A few ways to strengthen your data quality are to pilot test your survey before launching, ensuring consistency across different methods of completion (e.g., make sure your questions are asked in the same way regardless of how people complete your survey), and keep a log of any errors or issues so you can make improvements next time.
Monitoring your survey data
Tracking progress whilst your survey is live is essential. This will allow you to identify and troubleshoot any problems early on.
A few things to look out for:
Response rates: How many people have started/completed your survey? Consider whether you need to promote your survey more widely or remind respondents to complete.
Sample characteristics: Are the responses representative of your target sample or the groups you want to hear from? Consider alternative distribution methods if your sample is missing representativeness.
Drop-off points: Are people quitting the survey at the same point? If so, you might want to consider making your survey shorter or changing the wording of your questions to make them easier.
Managing false and disingenuous responses
If your survey is shared widely (e.g., on social media) or there is an incentive for completing the survey, you may experience some disingenuous responses.
There are some signs of disingenuous responses:
Looking for patterns or repeated answers, particularly in open-text responses. For example, a response where every question is answered to with ‘strongly agree’ or identical responses across multiple surveys.
Check how long it took for respondents to complete your survey. Very short completion times might indicate a false response.
Be cautious and think carefully about responses you think are ‘unusual’ as these may reflect genuine responses and respondents’ real experiences. Ask a colleague for their opinion and only remove responses if there is strong evidence to suggest it is false.