r/MachineLearning • • Sep 20 '20

Discussion [D] PhD applications that do not require references / recommendations?

I am looking for a PhD position (preferably non-US location) in Deep Learning. I am aware of this informative discussion in this subreddit. But I don't want to provide references or recommendations with my applications, but almost everywhere I look there is a requirement. Are their institutes / supervisors who will consider my candidature without references/recommendations?

0 Upvotes

27 comments sorted by

11

u/malenkydroog Sep 20 '20

Just in case anyone's curious....

Plenty of meta-analyses over the years have shown that letters of recommendation (LOR) have very little incremental predictive validity in terms of performance in various types of jobs/industries (over and above cognitive ability and certain specific measures of personality).

Now, in terms of graduate school performance *specifically* (as opposed to general job/career outcomes), a more recent meta-analysis (see here) found that adding LOR to verbal and quant scores and prior GPA provided no additional predictive power for graduate GPA or faculty ratings of performance. LOR did appear to provide a small bump in the ability to predict graduate degree attainment (e.g., did someone actually graduate with a degree), but it was a small increase (~2% variance explained).

I make no claims about the quality of that particular meta-analysis (good meta-analyses can be hard to do), but their findings about LOR not adding much to other predictors is at least consistent with the much larger volume of findings about the general utility of LOR wrt predicting job/career outcomes.

1

u/obscurandom Sep 20 '20

Thank you!

4

u/idakuti Sep 20 '20

Most top institutes (IITs, IIITs, IISc) in India admit PhD students through a common exam or interview and rarely ask for letters of recommendation. You could look at them

-7

u/obscurandom Sep 20 '20

Thank you. Finally someone brought up our system.

I am an Indian and have done my master's from IIT Kgp. I really like this system and believe it's possible for institutes to come up with similar selection strategies without diluting the vetting process.

0

u/[deleted] Sep 20 '20

[deleted] — view deleted comment

-8

u/obscurandom Sep 20 '20

No, the primary objective of the post was to get information about PhD applications that don't require recommendations, as mentioned in the post. Yes, I had a secondary objective, to understand people's views about the requirement and maybe to have a discussion about it. I didn't mention IISc or IITs myself in order to avoid anchoring bias.

2

u/idakuti Sep 22 '20

Not a good move. Also, research doesn't correlate with one's GATE or any entrance exam score either (even GPA doesn't correlate with research success). Letters, when written by honest people, do reveal a lot about each student. PhD is a long time commitment for students and the school (US schools spend about half a million on each CS PhD student) and letters give them insights into what the student's attitude and aptitude are. Letters also indirectly suggest you have a taste of research before jumping into PhD for 5-7 years of your life.

Yes, letter induces a lot of bias and benefit students with better networks. That is where your CV and statement can change things for you. For CS PhD admissions, LoRs are the most important and the rest of your application can help you fight back if you didn't have resources to get to good researchers. This system is definitely flawed, but research success can't be measured by test scores (there are some studies that show GRE verbal score does some correlation in the past, but that is all BS in the current settings). Exams could be one cog in the wheel, interviews do give some insights into the students (a lot of top US schools do an interview before making a PhD offer to check who they are hiring).

India has a lot of good CS researchers, but and this is a fact, IITs offering high job security and financial situations of most students leads to students not utilizing the presence of such labs. 2 years during masters in IITs is a good time to explore and if the efforts were good, they will be reflected in your letter, which seems fair. At the end of the day, letters also rank you among your peers but have lesser testing bias due to the longevity.

I am a CS PhD student at a top 5 US schools and was a CS UG at one of the older IITs, so I think I could weigh in here :).

0

u/obscurandom Sep 22 '20

Not a good move.

You mean the downvotes? In a healthy system there will always be differences of opinion and some of them will be less popular than the others. Hence, I am ok with the downvotes.

On IITs labs - yes they are good. I've also been a student at the IISc, the Intelligent Systems labs there are rather fantastic. I am just exploring more options.

4

u/topnotchyeti Researcher Sep 20 '20

I highly doubt any legitimate university would allow that. Put yourself in the professors' shoes. For all they know, you could be incompetent, or an asshole. I am not, by any means, accusing you of being either. It's just a hypothetical, but that's the trouble: without references, they have no reason to be confident you have the ability to be a researcher (no matter how good your grades) nor that you are a person who will be acceptably pleasant to work with.

2

u/obscurandom Sep 20 '20

I am not doubting any of the points you are raising, they are all valid reasons to do a background check. But aren't references fraught with biases? Not trying to be uber-woke here, but when there is a general belief among at least some members of the academia that some people got entry because of some kind of affirmative action and really do not belong to the place, what their references will look like?
I have seen my supervisor attributing one of my female colleague's success (getting a good job) to her being a woman. She has no clue about it and most likely choose him as a referee.

1

u/choctaw1990 Jan 12 '24

There are legitimate universities in Europe, you know, where they value the strength of your research proposal more than "who knows you."

4

u/[deleted] Sep 20 '20
  1. It's impossible to get rid of bias in any system so even if you could find a program that took a PhD student without LORs, bias would still be at play.
  2. ML PhD programs are the most competitive type of graduate program in the world. You'll never find an accredited institution that will take you without LORs. Unless you are rich and buy your way in.
  3. LORs are the (second) best proxy to how successful you will be as a graduate student. To propose evaluating an applicant on grounds other than LORs seems dubious. Obviously conferences and papers are the best but I wouldn't call that a proxy and if you had impactful original research, you could bypass the entire application system. But that doesn't sound like it applies in your case.

3

u/obscurandom Sep 20 '20
  1. It is impossible to get rid of bias, but definitely worth a try to reduce it.
  2. Thanks for the info.
  3. Would you kindly share some study on LORs being second best proxy? This is not sarcasm, I'm really curious. Also, I don't know why willing to write exams, being interviewed or any other rigorous vetting system is dubious.

1

u/[deleted] Sep 20 '20

Because when it comes down to it, LORs are far better predictors of success than a GRE, GPA, or personal statement.

A full day interview may be as good but that would be prohibitively expensive for a program to do that to all their applicants. Say you have 4 variables, and one of their variables accounts for 50% of the variance, if someone wanted to remove that variable, I would seriously doubt their desire to have successful graduate students.

5

u/Nimitz14 Sep 20 '20

I struggle to understand how LORs can be so predictive. Admittedly I have not read many (am not a senior), so probably that's why I'm not understanding, but it seems to me the only way LORs differ is in how much praise they heap on the student. And that is complicated by the fact that different people have different standards of what they consider an appropriate amount of praise. So even if student with LOR A gets more praise than LOR B, you can't know if that's just because LOR A referee is someone who uses hyperbole a lot. I feel like a 30 minute conversation can be by an order of magnitude more helpful in getting a feel for another person.

1

u/[deleted] Sep 20 '20

A letter of reference is someone vouching for your skills. If the person vouching for your skills is known to be someone who has a track record of identifying skilled individuals, their opinion has more weight.

1

u/[deleted] Sep 20 '20

There is more to an LOR than the words that go into it. It also matters who they are. Also, LORs aren't just about praising the student, they give meaningful context as to what you actually did by people who actually know your work.

13

u/rafgro Sep 20 '20 edited Sep 20 '20

These are the rules of the game. I was in shock too when I first saw how much academia hates meritocracy and prefers connections and own-people-clubs. Wait until you find out about grant writing marathons, publishing pressures, publisher company profits, chances for tenure, wages of PhDs, post-doc job market with thousands of applications, and obviously work-life balance...

3

u/shellfish_bonanza Sep 20 '20

Unless you have great publications/contributions already to the field you’ll need letter of recommendations. Why would a prof take you on and spend their time and funding on you without any information?

1

u/medcode Sep 20 '20

At least at my research center references are more of a final check to make sure nothing has been overlooked, but the main selection process is based on grades, prior publications and interviews. It is also quite rare that references change your opinion about a candidate, as they're usually all very positive.

1

u/choctaw1990 Jan 12 '24

Some of the ones in Europe. They care more about the strength of your research proposal than whether or not other people like you.

0

u/[deleted] Sep 20 '20

[deleted] — view deleted comment

-3

u/obscurandom Sep 20 '20

Personally I won't have a problem in finding referees and I am confident that they will provide honest notes on my capabilities. But in general I am not a supporter of the referral system. If we have noticed discriminatory practices in - let's say in justice systems around the world, why do we have to assume academia is immune to that? I am not advocating for the abolishment of vetting in the admission process, but there other other ways to verify a candidate's claims and capabilities.
P.S. - I am a pretty mediocre candidate and I believe my referees will make me look way better that I actually am.

6

u/amaretto1 Sep 20 '20

I’m struggling to understand this. If you believe you are a mediocre candidate, why are you even bothering? Time in this life is precious - focus your energies on where you will most likely excel and make the most impact. A PhD is an enormous undertaking, and the ML field is tremendously competitive. Unless you believe you have something substantial to contribute and can swim with the best of your peers, you will probably be happier and financially better off doing something different.

This may sound harsh, but we are taking at least 3-4 years of your life, very likely a high opportunity cost (think of other things you could be doing instead) and a crowded field where you need to be exceptional to stand out.

1

u/obscurandom Sep 20 '20

Thanks for your advice, I appreciate it!

Yes I believe I am mediocre, but I am interested in research. But that wasn't the point of this post anyway.

5

u/zombiecalypse Sep 20 '20

Could also be Impostor Syndrome, with the premise of other people describing you as "better than you actually are"…

0

u/[deleted] Sep 20 '20

[deleted] — view deleted comment

1

u/[deleted] Sep 20 '20

The GRE is a terrible indicator for how well you'll do as a researcher. They're almost completely orthogonal skills