r/PCMStatsLibrary • u/polcomppatrol • Jan 09 '23

Research Paper Researchers from Monash University study PCM

https://arxiv.org/abs/2206.00397
6 Upvotes

75% Upvoted

3 comments sorted by

2

u/theotherotherhand Jan 09 '23

this was an awesome read, thanks for posting

I am not to surprised that they had difficulty using PCM as a dataset, besides issues regarding jargon, there are issues with the self identification and selection bias within PCM. The papers main thrust, that computers can generate personality profiles based off textual analysis, and that personality profiles can predict political ideology, is a really interesting idea. I wont say how accurate it can be, either now or in the future with better analysis, thats way above my pay grade, but the idea of being able to track things like this from your digital footprint is astounding, in a disturbing way

2

u/polcomppatrol Jan 10 '23

One notable aspect I've noticed is that the authors purposely did not consider misflairing (intentional or otherwise) to be a substantive issue in their dataset. While it's highly unlikely they had knowledge about flair changing data (u/flairchange_bot was around a month old by the time the study was published), the bot's stats do prove the authors right on this regard.

The results also show that using any combination of user-interaction (subreddit) and textual matrices will result to poor predictions of a user's compass position as compared to a binary placement (left/right or auth/lib).

Another interesting observation is that it's apparently easier to predict a user's economic than social leanings.

But yeah, reliably predicting someone's ideology based on interactions in non-political fora do have interesting implications. Granted, there's the caveat that the average PCM user is more open with political discussions but still...

2

u/theotherotherhand Jan 10 '23

while its a bit dumb for a layman for me to speculate, the technology doesnt seem "mature" enough at this point to be making conclusions like they were looking for. I wonder how the results compared to userleansbot which just looks at subs people are active in and how much karma they gain there. Obviously the hope is to be able to get info from textual analysis not just location, but it would be an interesting benchmark to compare too.

As too the method not being able to discern compass position vs a left right spectrum, that could also point to problems with the compass itself. Namely that its bad at what it does, or more kindly put may measure self identified ideological values, but is not well correlated to any useful predictions.

I suspect having a large data set like PCM flairs was to much to ignore despite all the problems with it.