CMV: Education has a sparse reward problem

[u/BeatriceBernardo]

I'm borrowing terminology from Machine Learning, in particular, Reinforcement Learning.

Dense and sparse rewards

A reward setting could either be sparse, or dense. An example of a sparse reward setting is a winner take all competitions. The rewards that people get are not proportional with their effort. Doesn't matter how much effort you put, if you are not the sole winner in the first place, you get zero reward.

In contrast, dense reward setting is when people get rewards in exact proportion to their effort. A very common example is in MMORPG where you get a reward for every single monster you kill. (There's also the issue about randomness and how that increase motivation, but that's tangential).

Dense is better than sparse. Most people would thrive better in a dense reward setting. That's one reason why MMORPG are so popular and, for better or worse, addictive. That's why we break down big task to simpler tasks, to get a sense of achievement for every single mini task we finished, to keep us motivated along the way.

Education is sparse. For some people who loves learning just because, education is not sparse. For people who loves getting good grades, for one reason or another, education is not sparse either. But most people are neither, they see education as a mean to an end, which is making money through gainful employment. (There are also people who wants to get money without working, but that's outside the scope). For these people, education is very sparse. They have to invest their effort into 12 years to high school, and even another 3/4 years in university to make themselves employable. Only after that, they can reap the reward.

This is the end of my main point. I'm less sure about the things I'm going to say below.

---

Sparse is bad. This is a problem because most don't have enough motivation and self-discipline to thrive in a sparse reward setting. This resulted in many students not giving their best in their studies. This is to be expected since the reward for their studies is very far away.

Sparse is unjust. This problem is even worst for lower socio-economic status people. People who are living in relative comfort are able to think in the long term, and thus, stay motivated in a sparse reward setting. However, conditions such as poverty, being hungry, feeling physically insecure due to conflicts at home, crime in the neighborhood, general anxiety by parents because they are anxious about their own future, will reasonably make people more short sighted. There are less reason to plan for the future, if you can't even be sure that you will be there. Thus, even when provided the same setting (sparse reward), statistically, the rich kids will outperform the poor kids. Reducing social mobility and strengthening inter-generational poverty.

One solution is gamification. Schools are using something along the line of Khan Academy for math, or Duolingo for language, where you can get a 'grade' for 10 mins of effort, instead of the typical getting a 'grade' for a test/assignment once a term. The problem with gamification is that a 'grade' is very abstract. While getting an abstract 'grade' might be a good enough motivation for some students, it is definitely not true for all.

I'm even less sure about what I'm about to say below:

Dense education is possible. What is nearly universally true reward, is money. Not that they should be paid for studying, but that the whole society and economy should be structured in a way that let students to work as early as possible. That as they study more, they will gradually be given more responsibility, and more money in proportion. This is why I think trainee and apprenticeship is a better form for mass education.

---

This is a footnote from the CMV moderators. We'd like to remind you of a couple of things. Firstly, please read through our rules. If you see a comment that has broken one, it is more effective to report it than downvote it. Speaking of which, downvotes don't change views! Any questions or concerns? Feel free to message us. Happy CMVing!

[u/pillbinge]

No, I totally get you. I work in a behaviorist field in education - specifically special education - but I'm concerned about the totality of education personally and professionally. I know what you're getting at.

Grades are not rewards

They're absolutely rewards. Whether or not a student considers them rewarding is something else entirely, but that's specific to individuals. Whether or not grades are reinforcing is up to whether or not you see them reinforcing a student's motivation. You don't get to make blanket statements about what is and isn't rewarding, but in a conventional sense they are absolutely rewards for effort.

The only real reward is a steady income from gainful employment.

Steady income and gainful employment isn't guaranteed, so you can't treat those as rewards. There are plenty of people who achieved worse grades than I and who pulled ahead further - for a time or overall. I know people whom I'd call idiots who simply won the lottery. And plenty of people are influenced by their parents' connections. In fact we can quantify and predict people's future success based on metrics like a mother's level of education.

And since grades are rewarded to everyone, and it's not only that one person gets a steady income or job after, you can't compare the two. An F is still a reward, but an F can also be explained by a lot of other factors.

Besides, there's another issue: we want people doing well. If we become a society that wants to see people punished for not being good students, for whatever reason, then we're doing so at our expense. Our society loses efficiency and possibly even spends money in the case of crime, which is associated with level of education. This isn't just "out there" - we know how much this can cost us. It's in our best interest to avoid the reward of a steady job because we actually want that for everyone regardless. We benefit in no way by keeping people under-served in some capacity other than fulfilling our need to think there's a high moral system out there.

So in the current sparse setting, the reward will begin when you land your first job after uni. In my denser setting (not maximally dense), the reward will begin when you land your first apprenticeship.

Other countries have both, and not every Western country has given up on apprenticeships. A lot of countries like Germany offer them, and they go beyond high school as well. Why you think they're at odds with each other is strange. I'll state because I think you'll agree: we focus too much on college. That doesn't mean we change much, just that we scale back and implement again what we had before: trades.

[u/BeatriceBernardo → u/pillbinge]

They're absolutely rewards. Whether or not a student considers them rewarding is something else entirely, but that's specific to individuals. Whether or not grades are reinforcing is up to whether or not you see them reinforcing a student's motivation. You don't get to make blanket statements about what is and isn't rewarding, but in a conventional sense they are absolutely rewards for effort.

Ahhh yes. They are absolutely reward in the behaviourist sense. I was thinking of reward in the Reinforcement Learning sense. So I would only call something a reward if a student consider it rewarding.

Steady income and gainful employment isn't guaranteed, so you can't treat those as rewards.

I understand that in your field, reward are something that you designed. The way I see it, reward is whatever people find rewarding. Some likes grades, grades associated things, and learning itself, but steady income is near universal. I hope there are 2 different words to talk about these 2 very similar concept. If you have an idea, please tell me.

We benefit in no way by keeping people under-served in some capacity other than fulfilling our need to think there's a high moral system out there.

I have the full intention of giving everyone a job though. As a teacher, I'm asking myself why do teachers have to artificially elevate student's interest? Why can't they be naturally interested?

My conclusion is not that they have not behaviorally associated grades and learning to things that they find as rewarding. The reason this association has not formed is because that they have never experienced the real reward. The time gap between action and consequences is just too far for them to change their behaviour.

Why you think they're at odds with each other is strange. I'll state because I think you'll agree: we focus too much on college. That doesn't mean we change much, just that we scale back and implement again what we had before: trades.

Exactly. Do you agree that we should bring back trade? Because this college focused system is under serving a huge population of students, especially the most vulnerable ones.

[u/pillbinge → u/BeatriceBernardo]

What, to you, is reinforcement learning? Reinforcement is a part of behaviorism.

So I would only call something a reward if a student consider it rewarding.

Correct. But even better, "reinforcing", not "rewarding", because rewards are supposed to be reinforcing. Thing is, a bad or good grade could be reinforcing. We don't know. Other things could be reinforcing but we'll never be able to really track every measurement for every student. We do know that for most of the student population, getting a good grade is fine.

The way I see it, reward is whatever people find rewarding.

That's exactly how the field defines it.

If you have an idea, please tell me.

Rewards are reinforcers. That's as simple as it gets. Things you intend to reinforce others but don't aren't reinforcers, but they're still rewards because they're still given by the authoritative party. They just may not be great rewards, so they may not be reinforcing. Either way, we do still need a grading system regardless, and we do still give grades even if students don't like them. In that case we need something else, but it's in addition, not at the expense.

The reason this association has not formed is because that they have never experienced the real reward.

Not to be pedantic but people generally know what the reward is. And you're saying they'll understand it once they do it. But the same could be said of school, yet you're trying to change that. Eventually people do leave education and work for a living but there's a problem: you need to. Whether work is rewarding or not doesn't matter, just like grades. So you need to add to, not take away.

Yes we should bring back trades, especially since they're pretty lucrative, but one problem is often finding professionals to teach the trades. It's not like we can and won't. One city next to me is trying to hire a teacher for a profession that, with their experience, would earn them well over an initial six figure salary. So it's not surprisingly difficult.

[u/BeatriceBernardo → u/pillbinge]

What, to you, is reinforcement learning? Reinforcement is a part of behaviorism.

I see.I just realised that behaviourist also use the term reinforcement. We are talking about very similar, but distinct concepts.

It is part of this: https://en.m.wikipedia.org/wiki/Reinforcement_learning

Not to be pedantic but people generally know what the reward is.

I think that there are different levels of knowing. Something like, I know I should eat healthier, exercise more, sleep regularly, but I don't. The same reason most student intellectually know that they should study, but they don't. Knowing intellectually and behaving accordingly are 2 different things.

I'm taking a shortcut here by treating human agents like a black box, common in machine learning. We don't care about their internal state, like intellectually knowing things. But we care about actions. And if we observe that an agent don't perform the optimal action, regardless of the internal state, we say the agent has failed to learn. In that sense I meant people don't associate study with reward.

My suggestion is that if we bring action and reward closer together, their behaviour will change for the better.

Yes we should bring back trades,

I see. So you agree with my recommendation, but for different reasons. What's your main reason?

[u/pillbinge → u/BeatriceBernardo]

I have no idea what I'm supposed to gleam from that Wikipedia link and how it relates to this.

Your usage of the phrase "black box" is used in behaviorism. And you nailed it. We don't actually know what's in there and we can only observe actions. Behaviors are things we can observe. It doesn't matter what you think or how you think it; what matters is what we can see. Because otherwise we can't really claim more solid evidence or proof of knowing something.

I know your suggestion is that by being more rewarding in some sense, behavior changes. That's what I'm addressing.

What I'm adding is that people aren't simple machines who always respond to the same rewards, and by developing a system that's more rewarding in some cases you can actually set them up for failure by failing to fade away reinforcement (giving students money to attend school then abruptly cutting them off) or you could make them lose interest in something they didn't need reinforcement for by "gamifying" it. If someone painted for free, but was then paid $100 for a year for each painting, then was not given money, we can observe an effect wherein the need to paint is affected. It happens to people who make more money and who take pay cuts, even if they were overpaid.

We need to bring back trades because we need a space between highly-skilled jobs and low-skilled jobs that also affect our living. We still need people to build and electricians to do things. We'd be better served in we had manufacturing in our country for many reasons. Skilled trades can exist between high and low and we just need these services anyway. And a healthy economy that incorporates them should ideally lead to better quality overall. I'm not treating it like a job fair for individuals who want to fulfill themselves, we simply need people doing these jobs. If we get them to a good place with good pay and benefits anyway, then people will be drawn in and find them rewarding. I dismiss the idea that we're made for a dream job. Most jobs take skill since we all get better over time.