What you need to know about scales questions

Overview

This article provides comprehensive information about using scales questions in surveys, including best practices for rating scales, matrix questions, and balanced scales.

Information

Learn how to effectively use rating scales and matrix questions in your surveys to gather accurate and meaningful data from respondents.

• Using rating scale questions
• The scale must be balanced
• Matrix questions

Using rating scale questions

When it comes to rating scale questions, there is no evidence suggesting that one model is superior to another. However, it is recommended to follow these guidelines:

• Start with the least attractive answer and end with the most attractive answer
• Use a simple "yes/no" or "good/bad" format when a rating is not necessary
• Tivian recommends using a number scale from 1-6, where 1 is the lowest and 6 is the highest value

Reasons for using a 1-6 scale:

• "Dice roll ratings" are commonly used for evaluations
• Reflects the grading scale used in schools
• Is pedagogically correct (lowest score for lowest value)
• Provides the opportunity to calculate the average score
• No natural midpoint, forcing respondents to choose either the "negative" (1-3) or "positive" (4-6) half of the scale

The scale must be balanced

When using a scale as the response option, ensure that it is balanced with an equal number of negative and positive values. Here are some key points to consider:

• Use definitions for the outermost values in the scale (e.g., "to a very small degree" and "to a very large degree")
• Avoid leading question text formation in the scale
• Include both positive and negative values in the question (e.g., "How much do you agree or disagree with the following statements...")
• Use the same type of scale throughout the survey
• Make a conscious choice between a 5, 6, or 7-point scale, as averages may not be comparable across different scales

Matrix questions

Matrix questions can be useful for shortening questionnaires and improving the flow of the survey. Here are some guidelines for using matrix questions effectively:

• Limit the number of questions in a matrix to 8-10 to prevent respondent fatigue
• Use random rotation or shuffling of response options to avoid bias towards options at the top of the list

FAQ

Why is it recommended to use a 1-6 scale?

A 1-6 scale is recommended because it reflects common evaluation methods, aligns with school grading systems, allows for average score calculations, and forces respondents to choose between positive and negative halves of the scale without a neutral midpoint.

How many questions should be included in a matrix?

It is recommended to limit the number of questions in a matrix to 8-10 to prevent respondent fatigue and maintain the quality of responses.

Why is it important to balance the scale?

A balanced scale with an equal number of negative and positive values helps avoid bias and provides respondents with a fair range of options to express their opinions accurately.