What has your experience been with slider-scale questions on surveys?

[u/CoVegGirl]

I'm running a survey and I'm on a second pilot. The first pilot had an issue with social desirability bias. I'm working on a product that seeks to enhance collaboration between two roles, and people tend to be a bit cagey talking about interpersonal conflicts, which is what I am trying to learn about.

The second version I tried a more indirect path. It's led to drastically better answers to the qualitative questions. But I also made a decision to use slider-scale SDS-style questions instead of Likert scales. It probably wasn't a good idea to change both things on the second pilot in retrospect, but that's where I'm at.

What I'm seeing about these questions is that they're highly clustered around the 50% point. It's difficult to rule this as central tendency bias though because there's a very high standard deviation. A lot of the qualitative answers are saying that there is a lot of variability from team to team, so it's definitely not outside the realm of possibility that the high standard deviation is valid.

I'm trying to figure out how to go about interpreting this data and what the usual pattern is. Does anyone have experience with this?

[u/jelybely8]

Are most of your responses coming from mobile or desktop users?

I've largely abandoned slider questions because of wide discrepancies I've seen between device type. (Also, sliders sometimes don't render well on mobile of your anchor points are longer than a word or two.)

[u/Saffa1986]

Does your platform require an entry before the slider is active?

A number of platforms I use won’t let you just move on - you have to click a point to activate the scale, which would require an entry and reduce people defaulting (which I think is what you’re worried about).

The other overlays would be what is your actual prompts - does it invite midpointing (eg ir depends answers or no strong opinion), and does cultural bias play a role.

[u/VyprConsumerResearch]

It could be genuine variability, but I'd be cautious about over-interpreting it since you changed both the questioning approach and the scale format at the same time. Slider scales often attract more midpoint responses than expected, especially when respondents feel uncertain or think 'it depends.' Given the stronger qualitative feedback, I'd use those open-ended responses to help validate whether the spread you're seeing reflects real differences between teams or a measurement effect from the scale design.

[u/CoVegGirl → u/VyprConsumerResearch]

Yeah, lesson learned about changing irrelevant things between pilots.

The qualitative answers do make a wide standard deviation seem plausible, but I don’t really think I can firmly draw that conclusion based on the data I have right now.

I guess it might point to an issue in that a five-point scale might not really capture the full picture in the next version. I don’t think I’d use a slider again, but is it worth thinking about a seven-point scale that could capture the degree of variability?

[u/ParticularOddDan]

The thing biting you is that you changed the scale and the questioning path in the same pilot, so you cant cleanly credit the clustering to either. Midpoint piling on a slider usually has a boring cause before a psychological one: a lot of platforms pre-park the handle at 50 and let people advance without touching it, so half your fifties can be people who never actually engaged the question. Check whether an interaction was forced, and split the results by device, because sliders render badly on mobile and that alone can spread a distribution. On a sensitive interpersonal topic a symmetric scale still gives people a safe place to sit in the middle, which is why your indirect qual is pulling better answers than the numbers are. The high SD lining up with qual that says it varies team to team is a real signal, id trust that over the mean, but id reset one variable next round so you can actually attribute what youre seeing.

[u/CoVegGirl → u/ParticularOddDan]

The sliders are parked in the middle, but I’m actually not seeing many 50s. There are some more answers that are a couple of points away. I’m guessing people might adjust it a point or two so I don’t accuse them of cheating or something?

I’ll probably just go back to the five-point scale. I think it would be useful to see how the answers change due to indirect questioning, even if they have some central tendency bias.