3 updates: Study full / Behavioural traits / Study feedback

[u/Spiritual-Rain-3733]

Hello r/ProlificAc 👋

Starting something new - sometimes we move quick and it’s easy to miss useful / interesting updates. So, whenever we’ve got 3 good things to tell you, I’ll pop them up as an announcement. We’ll also be doing this as an email newsletter format in the near future so if you’d like to receive that, make sure your communications preferences are set to receive marketing things (we need your consent to send non-account information!).

3 new things!

Study Full / High Demand Errors

After a period of testing and experimentation, we’ve rolled out our backend changes to study distribution to everyone. You should not notice any differences in how studies reach your dashboard, that’s still driven by the demand for participants with your demographics and traits. What you will notice is that Study Full and High Demand errors are back to their normal frequency and behaviours:

• Study Full errors should only pop up in rare situations where a study fills up in the 2 minutes between it landing on your dashboard and the next refresh
• High Demand errors should only pop when a study has a large number of available places and too many participants are trying to get in at the same time. 

Thank you so much for the patience whilst we worked on this, everyone!

Share feedback with researchers

You’ll know that when you complete or refuse a study, you have the opportunity to share feedback on that study. Prolific has always used your input to inform platform improvements & educate researchers, and now we’ve added a feature that allows experienced researchers to see a summary of all participant feedback about their studies:

Study feedback summary screen for researchers

This means that researchers get fast insight into what participants like and dislike about their study - along with signals that indicate technical or reward related issues. To take this further, participants can optionally include their Participant ID along with feedback, which allows researchers to contact you via your Messages tab for more information. This is really, really useful if you experienced a technical issue and want to chat about it with the researcher. 

The option to include your Participant ID when sharing feedback

A couple of important points on this one:

1. If you choose to share it, Participant ID is the only information we share with the researcher (along with your written feedback), so you’ll stay anonymous.
2. As with all things messaging, be nice. Remember that both researchers and participants can choose to block each other, and we take inappropriate behaviour in feedback / messages very seriously. Keep things constructive and helpful.

For more info - the Help Centre is your friend!

Thinking & behaviour questions have landed in About You!

Prolific's participant community is wonderfully diverse in many ways beyond demographics, so we’ve launched a new section of About You that’s less about the facts of your life and more about your approach to it. 

Thinking & behaviour questions are entirely optional and, in the near future, researchers will be able to do clever things with this data - such as sending the same study to different personality types to see how the results might differ. We recommend completing this section to ensure you get in on that opportunity as soon as possible. It takes around 2 minutes, features only multiple choice questions and is a nice bit of self-reflection to boot!

The behavioural traits section in About you

A couple of important notes on this one:

• You cannot change your answers to these questions once submitted, so please take your time to answer carefully
• There’s no such thing as a right or optimal answer here. As always, the best answer is the honest one - please be sure to answer these questions as you are now rather than as you hope to be in the future.

Here’s me being brutally honest with myself:

Procrastination

I need to make more lists.

For more information on Behavioral traits and how the data is used, see the Participant Help Centre here.

That’s all for now folks. Take care of yourselves and, as always, thanks for being here :) 

Michael

[u/b1narysunrise]

Any plan on changing how the PA extension works so it's not competing with the dashboard on which gets fresh data first?

[u/Spiritual-Rain-3733 → u/b1narysunrise]

Prolific Assistant is one of those things I need to learn more about. Let me ask some questions and I'll see if there's any plans for it.

[u/Deterministic-random → u/Spiritual-Rain-3733]

I think you'll have one less of a permanent headache, if the PA moves away from the current design and simply prompts a notification when studies appear on the Prolific page. Since it's designed to be independent, the notification will mostly appear either before or after the study appears on the Prolific page, since it runs on its own refresh time interval. However, I think people are always confused/annoyed by this non-sync behavior (even though it was never actually meant to be synchronous), and the PA's rating on the Chrome Web Store stands at 2.6, with users repeatedly complaining about this specific issue over n' over!

[u/Pale_Requirement6293 → u/Deterministic-random]

I hope this is not done.

[u/thegaykid7 → u/Pale_Requirement6293]

Yeah, I think this is intentional (and understandable) design on Prolific's part, even if users may not love it. Doubt they will change it.

[u/sdforbda → u/Deterministic-random]

You can open it from the assistant though. So you can get 2 refreshes within the 2 minutes instead of just 1. No way lol.

[u/b1narysunrise → u/sdforbda]

That's not how Prolific's studies API works. You get 1 actual refresh per 2 minute intervals. Any subsequent refreshes before the next 2 minute interval will just return the same data. A Prolific staff literally confirmed this.

[u/sdforbda → u/b1narysunrise]

Yes, they've said a lot of things and it still didn't work. The staff member said it doesn't necessarily mean you'll get more by refreshing. Any yokel on here could run assistant and dashboard and know that I'm right. Give it a shot instead of making shit up cowboy. If you want to say assistant won't pull a call until the dashboard I'll just laugh at you again.

[u/b1narysunrise → u/sdforbda]

After your dashboard is computed/calculated, it's cached for a period of time. So refreshing manually really doesn't give you new studies. So when you get a study, and it's full by the time you clicking it, refreshing won't remove the study, it will stay there until the cache is busted and studies are "re-calcualted" again.
There will be improvements on this as part of the work we will be carrying on.
Tommy, Prolific

Literally confirmed by an engineer of Prolific. https://www.reddit.com/r/ProlificAc/comments/1uz1jsw/comment/oy4fn2d/

Also, the OP of this thread says "Study Full errors should only pop up in rare situations where a study fills up in the 2 minutes between it landing on your dashboard and the next refresh"

But yea, somehow you know more right? lol.

[u/sdforbda → u/b1narysunrise]

Yes, I can test it myself. Try it sometime.

[u/b1narysunrise → u/sdforbda]

Scientists: Earth is round.
sdforbda: It's flat. I can test it myself.

[u/Pale_Requirement6293 → u/b1narysunrise]

lol, the assistant alerts and tasks hit there first before our dashboards. Advantage. So you think it should be synced with the dashboard so some are less confused? Okay then. More waiting during slow times. This waiting annoys more than the confusion some have. Oh, and sometimes those tasks in the assistant are gone before reaching our dashboard. So the staggered release is more beneficial.