Digital
Designing a good survey
February 17, 2017 by Jono Ellis No Comments | Category Digital Public Services
This is a blog post by Philip Kirk, one of our user researchers.
A survey can be an effective way to get information on who is using your service. They can investigate the characteristics, behaviours and opinions of a group of people and, if designed well, can help you understand your users and what they need.
A poorly designed survey can undermine the results and findings. When beginning work on a new survey, ask yourself these questions:
- Is the goal of the survey clear?
- What is the question I am trying to answer, and is a survey the right research tool to do it?
- Do I have an overall ‘research question’ or survey goal that each survey question is helping to answer?
- Do I have a clear picture of who I need to answer this question and a plan to reach them?
- Is it clear how I will gather and analyse the results?
- How will each question feed into the survey goal?
- How will the responses be categorised and then interpreted?
- If using numbered scales or open questions, do I have the resources and skills to analyse and make sense of them?
- Are the outcomes of the survey actionable?
- How will I action the insights I get from the survey?
- Is the research goal and data giving me relevant insights?
If you don’t know how the answer to a question will help make changes to the work you’re doing, there is no need to ask that question.
Survey questions
Designing the questions is the most difficult part of making a survey. Each question should relate to the survey goal and be written from the perspective of the person answering the survey.
Questions should:
- relate to the survey goal
- be written from the perspective of the reader
- be obvious
- be interesting
- be appropriate
- be reliable
- be valid
Questions should not:
- lack purpose
- use abbreviations and acronyms
- be written in passive voice
- have a strong point of view
- be unnecessary
- cause distraction or confusion
Reliable – written so if the same group answered the questions at different times, they would give the same responses.
Valid – the question is answering what you think it is asking.
Question types
Questions can be open or closed. Open questions allow the person to write their answer to a question in their own words. Closed questions have a list of responses that the person chooses from. Each type of question has positives and negatives:
| Open | Closed | |
| Positives | Increased reflection | Easier to answer |
| Accurate responses | Easier to analyse | |
| Unanticipated answers | Less time and thought | |
| Shows up bad questions | ||
| Negatives | More interpretive | Adds more bias |
| Harder to analyse | Answers not clear and defined | |
| Require more time to answer | Not all answers are included |
Survey structure
A good survey should have four sections; introduction, consent, the questions and a thank you. It should be clean, simple and have a logical flow.
- Introduction – a short explanation of the survey and its goal. It should clearly describe, in a few sentences, what the reader needs to do and how long the survey will take to complete.
- Consent – this is essential and you need to comply with the Data Protection Act. You should get consent at the start and this needs to be explicit. Consent can’t be inferred from pre-ticked boxes or from starting the survey. Consent has to be a positive indication that personal information can be processed, such as an un-ticked box.
Consent should clearly tell the reader:
- what they have to do
- what the data will be used for
- who will see the data
- how data will be kept secure
- how long data will be kept, including the date this starts from
- how the data will be disposed
A survey isn’t a tool to collect email addresses. You can’t use email addresses from a survey for any purpose other than what was set out in the consent. If you want to follow up from the survey, explicitly state this in the consent with information about what the follow up will be.
- Questions – these should be set out in a logical order. The number of questions needed depends on your survey goal. Questions should be checked to ensure they are in plain English and ideally at a reading age of 11 years old. There are many tools to check this, such as Hemingway Editor.
- Thank you – thank the person for filling out the survey. You may want to add your contact details if you want any feedback or questions from the audience directly.
TIP: If using closed questions and are not able to statistically test them then:
- don’t use the same wording in the questions as an answer
- have the most negative answer on the left side of the scale
Do you agree this question is biased?
Agree ☐ Neutral ☐ Disagree ☐
Is this question biased?
Disagree ☐ Neutral ☐ Agree ☐
Finally, take the survey out and see how people use it. Is it the way you expect them to? This is the best way to make sure your questions make sense to your audience.
Tags: survey design, user research
RSS
Email
Leave a comment