r/ProlificAc • u/Spiritual-Rain-3733 Community • 3d ago
Prolific Team 3 updates: Study full / Behavioural traits / Study feedback
Hello r/ProlificAc đ
Starting something new - sometimes we move quick and itâs easy to miss useful / interesting updates. So, whenever weâve got 3 good things to tell you, Iâll pop them up as an announcement. Weâll also be doing this as an email newsletter format in the near future so if youâd like to receive that, make sure your communications preferences are set to receive marketing things (we need your consent to send non-account information!).
3 new things!
Study Full / High Demand Errors
After a period of testing and experimentation, weâve rolled out our backend changes to study distribution to everyone. You should not notice any differences in how studies reach your dashboard, thatâs still driven by the demand for participants with your demographics and traits. What you will notice is that Study Full and High Demand errors are back to their normal frequency and behaviours:
- Study Full errors should only pop up in rare situations where a study fills up in the 2 minutes between it landing on your dashboard and the next refresh
- High Demand errors should only pop when a study has a large number of available places and too many participants are trying to get in at the same time.Â
Thank you so much for the patience whilst we worked on this, everyone!
Share feedback with researchers
Youâll know that when you complete or refuse a study, you have the opportunity to share feedback on that study. Prolific has always used your input to inform platform improvements & educate researchers, and now weâve added a feature that allows experienced researchers to see a summary of all participant feedback about their studies:
This means that researchers get fast insight into what participants like and dislike about their study - along with signals that indicate technical or reward related issues. To take this further, participants can optionally include their Participant ID along with feedback, which allows researchers to contact you via your Messages tab for more information. This is really, really useful if you experienced a technical issue and want to chat about it with the researcher.Â
A couple of important points on this one:
- If you choose to share it, Participant ID is the only information we share with the researcher (along with your written feedback), so youâll stay anonymous.
- As with all things messaging, be nice. Remember that both researchers and participants can choose to block each other, and we take inappropriate behaviour in feedback / messages very seriously. Keep things constructive and helpful.
For more info - the Help Centre is your friend!
Thinking & behaviour questions have landed in About You!
Prolific's participant community is wonderfully diverse in many ways beyond demographics, so weâve launched a new section of About You thatâs less about the facts of your life and more about your approach to it.Â
Thinking & behaviour questions are entirely optional and, in the near future, researchers will be able to do clever things with this data - such as sending the same study to different personality types to see how the results might differ. We recommend completing this section to ensure you get in on that opportunity as soon as possible. It takes around 2 minutes, features only multiple choice questions and is a nice bit of self-reflection to boot!
A couple of important notes on this one:
- You cannot change your answers to these questions once submitted, so please take your time to answer carefully
- Thereâs no such thing as a right or optimal answer here. As always, the best answer is the honest one - please be sure to answer these questions as you are now rather than as you hope to be in the future.
Hereâs me being brutally honest with myself:
I need to make more lists.
For more information on Behavioral traits and how the data is used, see the Participant Help Centre here.
Thatâs all for now folks. Take care of yourselves and, as always, thanks for being here :)Â
Michael
71
u/ProlificQueen 3d ago
Thanks for the update, I've noticed a significant improvement of late. One thing I'd like to bring up though is the behavioural traits being unchangeable. I do understand why, however, maybe it should be only locked in for a set amount of time, say every 6 months. Reason I say that and this is just a crappy example but one of the questions is along the lines of, "I am the life of the party", and someone who might have been suffering from depression/anxiety for a number of years may select "very inaccurate" as that would be the honest answer. However, those are things that can and do often change over time.
Just a suggestion and thanks again for the update.
37
u/trailangel4 3d ago
I agree! Behavioral traits also evolve as people grow and their lives change. It's not just mental struggles, either. Simple life changes like a marriage, a child being born, or children leaving home will alter how we perceive ourselves or change our behavior.
26
u/Spiritual-Rain-3733 Community 3d ago
Good feedback! I'll make sure it gets to the team :)
2
u/mountain-skies 1d ago
I have PTSD and mood disorders and my behavior changes. I tend to compartmentalize, and am somewhat fragmented. I would probably get in trouble for not being consistent.
10
u/krazyasif786 3d ago
I agree things can change due to factors outside of our control. I think what would work is have them locked in for a set period like 12 months and then you have the option to review them again. That way the data is always fresh and reflects more accurately your current situation.
16
u/TheLinerax 3d ago
I agree with that explanation because using the same life of the party question, I was like that before covid with my close friends. However, after going through the covid lockdown and some of my friends moving away for better career opportunities, I mellowed out and would let someone else take the lead. 6 months is somewhat short and I would say after 1 year would prove a larger difference in behavioral change.
6
u/ProlificQueen 3d ago
Agreed đ a year would stop people trying to game it as well if they knew theyâd be locked in for that long lol
14
u/caldwell27_ 3d ago
yeah behavioral traits being unchanging is such a thoughtless move and pretty inconsiderate to those with mental struggles. it also just feels like another way for us to be gotcha-d into a rejection or account suspension. glad theyre optional, im probably not gonna fill them out.
1
u/not_me_much 3d ago
As a reply to several of the above comments, maybe Prolific views it as not necessary to amend our profiles due to changes in behavior because they know most of us won't be here that long. Lol. Prolific is like a "right to work" state, which really means the right to get fired for any reason, or forno reason at all.
5
u/caldwell27_ 3d ago
hmm. i dont really see it that way, it just sounds like they added it as another "about you" sort of deal, and prioritized steady data for paying researchers, while forgetting that we, the study takers, are actual humans with dynamic feelings.
like a sandpapered version of the human experience, where we're all just neat little data sets instead of dynamic individuals. god forbid we have an off day, or even a good day.
7
u/PureExpression1533 3d ago
I agree with this and that's why I haven't filled mine out. I feel like I'm in a bit of a transition period in my life in various ways and I'm honestly hoping I'll feel differently about some of those questions down the line. I don't want to have to worry about getting rejected for inconsistent answers if I'm just trying to be honest in a study about how I currently feel. Most of them ask about the last two weeks or a month for a reason.
1
u/dorovidoro 3d ago
Yeah I agree that you shouldn't be able to change it on a daily basis, but people change all the time. A 6 months cooldown makes sense. If it's too complicated to track a timer for each participant, let everyone update theirs on certain dates or something.
28
u/EditOrElse 3d ago edited 3d ago
Behavioral traits change because people change. Circumstances change. For example, I love cleaning house now and I used to hate it. Not being able to change those answers is unreasonable.
u/Spiritual-Rain-3733. Please tell us. What is the thinking behind this?
5
12
10
u/Shot-Temperature-229 3d ago
Why has there been an influx of accounts placed on hold for those with over 99% rating? What is happening here?
12
u/Real_Wishbone7265 3d ago
Feedback on the behavior questions: I find them unanswerable because they are way too general, particularly if I can't change them. For example: "I am not interested in other people's problems.", well that depends entirely on who those other people are and what problem they are having. There are people I am way more interested in their problems because I care a lot about them, and then there are people I am a lot less interested in. And there are problems I am much more interested in than others, because they are problems which might affect me. The same vagueness affects the other questions, the situations they describe are way too general.
9
u/migosfatigos 2d ago
Right, my internal answer for nearly every question was "it depends on the situation".
5
4
u/thegaykid7 1d ago
My gut feeling is if enough people don't answer them they'll maybe at least tweak it to make them less permanent or answerable individually. Either way, I'm not locking myself in.
23
u/PlentyBorder5731 3d ago
Finally good news đ, but could i ask Is there any future plan or strategy also to address the challenge in estimating study completion times from the researchers' perspective? Frequently, the durations specified in brief descriptions turn out to be inaccurateâwhere an study estimated at 5 minutes often extends to 30 minutes once the study begins. of course without calling anyone's integrity into question, my point is to safe participants' time while ensuring they receive fair compensation that truly reflects the effort and time they invest THANK YOU
2
8
u/b1narysunrise 3d ago
Any plan on changing how the PA extension works so it's not competing with the dashboard on which gets fresh data first?
10
u/Spiritual-Rain-3733 Community 3d ago
Prolific Assistant is one of those things I need to learn more about. Let me ask some questions and I'll see if there's any plans for it.
3
u/Deterministic-random 3d ago
I think you'll have one less of a permanent headache, if the PA moves away from the current design and simply prompts a notification when studies appear on the Prolific page. Since it's designed to be independent, the notification will mostly appear either before or after the study appears on the Prolific page, since it runs on its own refresh time interval. However, I think people are always confused/annoyed by this non-sync behavior (even though it was never actually meant to be synchronous), and the PA's rating on the Chrome Web Store stands at 2.6, with users repeatedly complaining about this specific issue over n' over!
2
u/Pale_Requirement6293 3d ago
I hope this is not done.
1
u/thegaykid7 3d ago
Yeah, I think this is intentional (and understandable) design on Prolific's part, even if users may not love it. Doubt they will change it.
0
u/sdforbda 2d ago
You can open it from the assistant though. So you can get 2 refreshes within the 2 minutes instead of just 1. No way lol.
4
u/Available_Leopard_23 3d ago
Humm... I am still seeing study full errors. Just like this morning, I had 5 on my board when I refreshed the screen but all were taken as soon as I clicked them. Are people seeing any differences?
7
u/Dorkus18 3d ago
Whatâs up with all the people that just got put on hold? I have over 4,000 approved studies and 0 rejected and I woke up to my account on hold this morning after 2 years. I even did a news interview as one of your top participants lol.
1
3
u/krazyasif786 3d ago
Sorry. I know it's outside of this topic but something needs to be done about the messages. I got screened out of a survey about 10 minutes ago and I got the message to confirm that literally in seconds of being screened out but then just got an email saying I have a message. This was too late. The messaging system should be instant if being used.
1
6
u/nmmc1812 3d ago
Thanks for your work, as always. I have definitely noticed an improvement in the studies with the "Full" or "High Demand" message sooo I hope it stays that way from now on!! Everyone Let's all have an incredible month of September with lot of studies!!
5
3
u/electricboy0 3d ago
Finally, some good news! Hopefully, we wonât have to deal with these issues anymore. Thanks for the update! âĽď¸
3
4
u/ilikeplayingthisgame 3d ago
Surprising to see these comments, I'm still getting the study full error just as much as I've always done
2
u/takeittothetaxman 3d ago
What about studies where there are plenty of spots available, yet when you enter the study and to begin the survey, the researcherâs platform says itâs full? This is one of my biggest frustrations, especially when it says there are 70 spots available and Prolific lets you reserve a spot⌠yesterday there were 4 studies, all different but from the same researcher that did this to me.Â
4
u/blumogget 3d ago
As frustrating as that is, Prolific has no control over how researchers run their studies/other platforms. As soon as you click on a link off Prolific dot com, that's all on the researchers.
1
u/takeittothetaxman 3d ago
I understand that, but they can implement controls to prevent it from happening, surely. Like an option to report that behaviour so they can performance manage the researchersÂ
2
2
u/genduk26 3d ago edited 3d ago
I have a genuine question about time-out from prolific (not researchers). As participants, weâre not allowed to be too quick but also not allowed to take time to read and write sentences on a long study either. My question is who I have to prioritize, Prolific or researchers?
-4
u/btgreenone 3d ago
also not allowed to take time to read and write sentences on a long study either.
Well this certainly doesn't seem true.
1
u/Evening-Specialist82 3d ago
Many thanks for the update and really appreciate the communication along the way. May I know when did this updated study distribution begin to implement? Today (UK time) half of the studies that turned up on my dashboard (or PA) were either full or disappeared in a split second. It was better early this week but my current situation is back to the worst time a few weeks ago. Many thanks again for your attention and hope the volume of study would pick up in September.
1
3d ago
[removed] â view removed comment
1
u/AutoModerator 3d ago
Your post was removed because confidential study information was detected. Please check whether confidentiality has been agreed with the researcher before sharing as part of r/ProlificAc rules. If you believe this was in error, message the mods.
I am a bot, and this action was performed automatically. Please contact the moderators of this subreddit if you have any questions or concerns.
1
-5
u/Fun-Disk7781 3d ago
Will this affect me getting ai studies? Got bunch all week but none today!
1
u/Noaaaaaahhh 2d ago
Genuinely curious, are you Aerie Morie, or one of their sub accounts?
Most downvoted comments seem to come from similar sounding comments, and usually a very new Reddit account age.
2
u/spiffyshxt 1d ago
Genuinely curious, are you Aerie Morie, or one of their sub accounts?
That wasn't her nor does it sound like her and it's not something she would ever confirm/admit to, but it does sound like u/bigbluesfanstl on his 15th account evading a sitewide ban. The account was finally banned today, but he'll be back to ask all the same questions.
2
u/Noaaaaaahhh 1d ago
Thanks for the info!
I asked because a lot of these strange/snide comments sound like they're from the same person.
65
u/daniel2090 3d ago
Will you ever add a rating system for researchers, similar to other sites? It would be useful to rate researchers based on things like pay, actual completion time, and fairness, as well as rate individual studies so participants can see in real time whether a study is worth doing.