Low quant GRE: successes and failures

[Much Anxious Very Waiting]

 

And of course the system helps high scorers better than low scorers and if (I) "don't do well on the GRE, someone else will". Just what are you trying to argue here?

 

 

I'm saying it helps the good scorers more than it hurts the average / slightly worse scorers, or so I understand.

 

And I think test-taking is a related skill to many useful ones, such as study skills or performance under pressure, though of course it doesn't have direct real world value.

 

I'm not 'arguing' anything.... I'm saying, whether we like it or not, we're being compared to people who do very well on the test, no matter how well we do. So we can say that it's a BS measure, or that it's a great measure, or w.e. But either way, some people will do awesome on it, and those are included our 'competition.' 

[spunky]

I think ETS should feature items on the GRE that women and minorities consistently outperform white males in. just to see how society reacts

 

Well… they sorta tried this before with the SATs back in the 60s during the “great score decline” with somewhat mixed results. I believe Roger’s book Social foundations of testing: A multicultural perspective mentions something about it. The main problem was they were not able to find enough items in which minorities/women/choose-your-group-of-choice outperformed white males to make any significant contributions to the total scores. Usually, if minorities/women/choose-your-group-of-choice performed well in a set of items, white men did so as well… although that was not the other way around and nobody could figure out why. It’s important to keep in mind that the ecological/contextualized paradigm of testing (also known as the “third generation” of testing) was just starting to gain traction so this baffled a lot of people in ETS and other testing companies. Eventually, they just decided to stop right there because they were about to open another massive can of worms had they gone further, opting instead for doing normed scores (the ones where your scores are calculated with reference to other people of similar demographic characteristics).

 

I still feel, however, that the problem is not so much on the tests themselves but on the use the tests are given by profs/AdComs/university authorities. 

[TheMercySeat]

Well… they sorta tried this before with the SATs back in the 60s during the “great score decline” with somewhat mixed results. I believe Roger’s book Social foundations of testing: A multicultural perspective mentions something about it. The main problem was they were not able to find enough items in which minorities/women/choose-your-group-of-choice outperformed white males to make any significant contributions to the total scores. Usually, if minorities/women/choose-your-group-of-choice performed well in a set of items, white men did so as well… although that was not the other way around and nobody could figure out why. It’s important to keep in mind that the ecological/contextualized paradigm of testing (also known as the “third generation” of testing) was just starting to gain traction so this baffled a lot of people in ETS and other testing companies. Eventually, they just decided to stop right there because they were about to open another massive can of worms had they gone further, opting instead for doing normed scores (the ones where your scores are calculated with reference to other people of similar demographic characteristics).

I still feel, however, that the problem is not so much on the tests themselves but on the use the tests are given by profs/AdComs/university authorities.

Agreed on the last comment-- ETS is very explicit in advising again the use of cut-scores and advocates for holistic evaluation.

Historically women used to dominate males on SAT V, and now males dominate females on the SAT and the GRE V. I'm surprised to hear that ETS and The College Board (allegedly) cannot figure out how to build assessments that favor females once again.

At any rate, I wish AdComms would just bluntly acknowledge the obvious inferences that they make from the GRE. They can publish all they want on stereotype threat, test bias, differential prediction, and so on, but it really doesn't mean a damn if they systematically block women and minorities from higher education and access to elite institutions through admissions assessments.

One final point: I get paired with a lot of ESL PhD psych candidates who struggle with basic writing, speaking, and spelling in English because they struggle with scientific communication. Many of them are going to or went to Ivy League institutions... How the hell does that work?

do American schools waive GRE V scores for English language learners? They're damn bright people, and it gives me opportunities to make significant contributions to manuscripts... But it's still odd when I know how the GRE locks native English speakers out of programs.

Edited February 17, 2015 by TheMercySeat

[morningsun]

I had an extremely low quant. score (37th percentile) and I've been accepting into two programs so far (I haven't been rejected from any yet). That being said, I have a really high GPA, great stats grades and did well on the rest of the GRE.

[TheMercySeat]

I feel like I can chime in a little more...

 

I had a terrible GRE Q (48%)

 

I got interviews with more than half of the programs that I applied to, including 5 R1s. I preface this with a few caveats:

(1) I have a MA.

(2) I have 2.5 years of full time, paid RA experience in psychometrics (ironic)

(3) I did not apply to programs that boasted having an average GRE Q above the 70th percentile. In retrospect, I think I low-balled a little bit.

(4) One program -ironically NOT a R1 and, in fact, a program that waited until the interview to tell me that they cannot afford to take on a student- took issue with my GRE Q and brought it up during the interview. The R1s and more prestigious departments didn't care about my GRE Q.

(5) My other GREs (V, AW, and subject) were in the 80th and 90th percentiles, respectively.

(6) I did not apply to clinical programs. I got invited to interview at social programs with an acceptance rate as low as 2%, though. 

 

If you're willing to work like a dog to compensate for a weak GRE Q with a strong research CV, you'll stand a chance, and you will not necessarily be restricted to R2s (PS- I maintain that there is nothing wrong with R2s). Don't lose hope!

Edited March 7, 2015 by TheMercySeat

[clinicalapplicant]

I was accepted into a doctoral program in Clinical Psychology and my quantitative score was 144. That's the 18th percentile.

My AW was 93% and Verbal was 81% but it didn't really affect my admission too much. It was the last question asked during my interview and he just wanted to know if statistics would be too much of a struggle for me.

You can still make it as long as you show how much you're committed to the actual program.

Edited March 7, 2015 by clinicalapplicant

[TheMercySeat]

I was accepted into a doctoral program in Clinical Psychology and my quantitative score was 144. That's the 18th percentile.

My AW was 93% and Verbal was 81% but it didn't really affect my admission too much. It was the last question asked during my interview and he just wanted to know if statistics would be too much of a struggle for me.

You can still make it as long as you show how much you're committed to the actual program.

PhD?!

That's pretty epic!

[SocialHealth]

I was accepted with a 150 quant 157 verbal and 4 aw 

[clinpsy2015]

I just got into *my personal* top PhD clinical program with a quant score around the 50th percentile.

 

From what I understand, 50th percentile is pretty terrible for prospective clinical applicants. I took the GRE twice and actually scored lower on quant AND verbal the second time (after studying for another month, mind you!). You can't imagine how awful I felt after I took the GRE the second time and failed to improve. It was seriously disheartening; I felt like giving up the whole process of applying because I was sure I would be rejected. I have extreme testing anxiety but I think my AW score (90th percentile approx.) helped. 

 

Quant is only one piece of the puzzle. I may be speaking solely from my own experiences, but my undergraduate advisers told me that they use the GRE solely to "weed out" applicants. However, a high GPA (and high grades in stats classes) and strong letters of rec can bolster weak quant scores.

 

My low quant score did not stop me from getting interviews. In fact, I was never asked about my quant scores (or the GRE at all). To cover my weaknesses, I made sure to ask faculty members - and my POI especially - what kinds of topics were covered in first and second year stats courses. I think showing interest in stats may help you if your quant score is not so good. 

 

In sum: don't give up! You're worth a lot more than your score. Make sure to own that when you interview. 

Edited March 8, 2015 by clinpsy2015

[TheMercySeat]

Phew!

The biggest reason why I went the non-clinical route was because my GRE Q was garbage and I thought it would lock me out of all clinical programs.

Edited March 8, 2015 by TheMercySeat

[TenaciousBushLeaper]

Phew!

The biggest reason why I went the non-clinical route was because my GRE Q was garage and I thought it would lock me out of all clinical programs.

But have you already accepted an offer for a non-clinical program? 

[TheMercySeat]

But have you already accepted an offer for a non-clinical program?

Not yet. I have two offers and I'm waiting until I visit the one university before I make my decision.

I'm also waiting to hear back from the other places I was interviewed for. While rejection is presumed, I'm still hopeful because the one university that offered admission is in an extremely undesirable location, while the one I'm waiting to visit might not have as much to offer.

[when]

I was accepted into a doctoral program in Clinical Psychology and my quantitative score was 144. That's the 18th percentile.

My AW was 93% and Verbal was 81% but it didn't really affect my admission too much. It was the last question asked during my interview and he just wanted to know if statistics would be too much of a struggle for me.

You can still make it as long as you show how much you're committed to the actual program.

Pretty much the same here!

[Journey2015]

My quant score was in 20th percentile. I was accepted into one PhD program and expect a decision Fr. My top choice on Friday. I, too, was asked about my low score at the end of a day full of interviews. However, my scores were a few years old and I've completed a masters with quantitative thesis, so it wasn't held against me.

[Journey2015]

Rcvd second acceptance today! Its possible!

[MyDogHasAPhD]

Rcvd second acceptance today! Its possible!

Yay!! Congrats. You give me hope!!

[TheMercySeat]

My quant score was in 20th percentile. I was accepted into one PhD program and expect a decision Fr. My top choice on Friday. I, too, was asked about my low score at the end of a day full of interviews. However, my scores were a few years old and I've completed a masters with quantitative thesis, so it wasn't held against me.

Congrats!!!

That rocks!!!

Having a GRE Q scraping below the 48th percentile, I seriously thought I wouldn't get in anywhere, and so I let GRE thresholds dictate which schools I applied to. I also applied to a very rigorous university on a whim and got accepted. I hope perspectives for the next cycle who struggle with GRE Q see our victories and don't get discouraged! If we didn't come out of the woodwork, I would have been left thinking that all grad cafe psych applicants scored above the 70th percentile in everything.

[ed_psy]

What if M+Q doesn't make 300pts mark and a GGPA isn't 3.5+. Anyone got in phd programs? I remember reading over at SDN schools automatically eliminate 50% of the applicants who did not meet their minimum requirements for admissions.

[Engali]

I know people are passionate about the topic, but as future academics we really need to have at least some amount of intellectual honesty when it comes to any topic under discussion. This is particularly salient when we're talking about an area within the province of psychology (i.e., psychometrics and psychological testing).

 

The bottom line is nothing is perfect as a predictor of any outcome. That's just a fact of life. What we have is good evidence that the GRE in general, including the GRE-Q, is a good predictor of grad school "success" across a wide range of operationalizations. The one that has been nitpicked to death in this article:

 

http://internal.psychology.illinois.edu/~nkuncel/gre%20meta.pdf

 

is time to completion. It can be easily argued that how quickly one moves through a program is a poor index of "success" as long as the length isn't excessive. For example, my professor advisor suggests doing six instead of five years in my PhD program because it gives you a year to dedicate to research that you wouldn't have if you were a junior faculty member having to mentor students and teach classes. Is the opportunity cost in first year wages worth the potential long-term gain in landing a better job in a better school because of a stronger CV? Well, it's working out really well for my professor advisor to put it mildly. 

 

The larger issue is that the world isn't perfect. The very arguments used to question the *validity* of the GRE (which, btw, isn't actually put into question because the data already shows otherwise) in this thread conflict. Yes, the subject GRE is a slightly better predictor than the general GRE on the one hand, but on the other hand others have mentioned paying for even the general GRE is burdensome. The requirement set forth by the *graduate schools* of universities demand a general GRE--in this regard, the psychology departments hands are tied in that they must at least require these test scores. How can you then suggest using subject GREs when it would further disenfranchise those who are already struggling to pay for the general GRE in the first place? The problem is that we live in a limited world with limited resources and many moving parts. This is life and you deal with it.

 

The overarching concern I have with all that I have read in this thread is this contempt for the GRE and dismissal of this validity because you don't *like* what it stands for in your minds. This constant refrain of "well grad schools need something to winnow down the list" and "it's just a way to narrow the list down." The validity of the GRE is an *empirical question* and isn't subject to your whims or values. This is how science needs to be approached in general. Can it be better? Sure. Is it perfect? See above. Does it predict grad school success? Arguably better than most any other standardized way of comparing candidates and predicting success. As aspiring scientists I would expect you all to approach these types of empirical questions with some restraint in how it affects you personally. What's particularly ironic is that you all want to be part of a field that spawned these types of tests in the first place. 

 

There is good reason the GRE is used as a way to select candidates: it predicts performance. Are there other things that also predict performance? Sure. Personality measures have incremental validity in predicting performance. But if your personality profile worked to your detriment in getting accepted somewhere, would you then question the validity of those tests? If any test or predictor worked to your detriment, would you then question the validity of those tests? Again, intellectual honesty is key to scientific progress, which may not always work in your favor or to your benefit. You might want to check how willing you are to put your personal concerns aside when if you want to pursue psychology using the scientific method.

[TXInstrument11]

I know people are passionate about the topic, but as future academics we really need to have at least some amount of intellectual honesty when it comes to any topic under discussion. This is particularly salient when we're talking about an area within the province of psychology (i.e., psychometrics and psychological testing).

 

The bottom line is nothing is perfect as a predictor of any outcome. That's just a fact of life. What we have is good evidence that the GRE in general, including the GRE-Q, is a good predictor of grad school "success" across a wide range of operationalizations. The one that has been nitpicked to death in this article:

 

http://internal.psychology.illinois.edu/~nkuncel/gre%20meta.pdf

 

is time to completion. It can be easily argued that how quickly one moves through a program is a poor index of "success" as long as the length isn't excessive. For example, my professor advisor suggests doing six instead of five years in my PhD program because it gives you a year to dedicate to research that you wouldn't have if you were a junior faculty member having to mentor students and teach classes. Is the opportunity cost in first year wages worth the potential long-term gain in landing a better job in a better school because of a stronger CV? Well, it's working out really well for my professor advisor to put it mildly. 

 

The larger issue is that the world isn't perfect. The very arguments used to question the *validity* of the GRE (which, btw, isn't actually put into question because the data already shows otherwise) in this thread conflict. Yes, the subject GRE is a slightly better predictor than the general GRE on the one hand, but on the other hand others have mentioned paying for even the general GRE is burdensome. The requirement set forth by the *graduate schools* of universities demand a general GRE--in this regard, the psychology departments hands are tied in that they must at least require these test scores. How can you then suggest using subject GREs when it would further disenfranchise those who are already struggling to pay for the general GRE in the first place? The problem is that we live in a limited world with limited resources and many moving parts. This is life and you deal with it.

 

The overarching concern I have with all that I have read in this thread is this contempt for the GRE and dismissal of this validity because you don't *like* what it stands for in your minds. This constant refrain of "well grad schools need something to winnow down the list" and "it's just a way to narrow the list down." The validity of the GRE is an *empirical question* and isn't subject to your whims or values. This is how science needs to be approached in general. Can it be better? Sure. Is it perfect? See above. Does it predict grad school success? Arguably better than most any other standardized way of comparing candidates and predicting success. As aspiring scientists I would expect you all to approach these types of empirical questions with some restraint in how it affects you personally. What's particularly ironic is that you all want to be part of a field that spawned these types of tests in the first place. 

 

There is good reason the GRE is used as a way to select candidates: it predicts performance. Are there other things that also predict performance? Sure. Personality measures have incremental validity in predicting performance. But if your personality profile worked to your detriment in getting accepted somewhere, would you then question the validity of those tests? If any test or predictor worked to your detriment, would you then question the validity of those tests? Again, intellectual honesty is key to scientific progress, which may not always work in your favor or to your benefit. You might want to check how willing you are to put your personal concerns aside when if you want to pursue psychology using the scientific method.

First, you need to get off your high horse. We actually did discuss the predictive validity of the GRE, including the analysis of a Quant psych student who worked for ETS. Before reading the article you posted, I had simply never seen high predictive validity for the GRE before.

However, how did they control for selection bias? Good schools who can demand higher GRE scores typically have superior resources and training for their students. Due to reputation, they also tend to attract the best, most ambitious students.

Second, you act as if you are immune to bias, which is a blind spot in itself. Let me guess - you did well on the GRE? People who do well on the SAT subsequently put more stock in it and people who do poorly on the SAT do the opposite (what you assume of us) . I highly doubt that you went into researching the GRE's validity as a bastion of purely objective, scientific thought. Your own bias doesn't change the numbers - which look good here-but it does mean you are being a sanctimonious ass.

[Engali]

First, you need to get off your high horse. We actually did discuss the predictive validity of the GRE, including the analysis of a Quant psych student who worked for ETS. Before reading the article you posted, I had simply never seen high predictive validity for the GRE before.

However, how did they control for selection bias? Good schools who can demand higher GRE scores typically have superior resources and training for their students. Due to reputation, they also tend to attract the best, most ambitious students.

Second, you act as if you are immune to bias, which is a blind spot in itself. Let me guess - you did well on the GRE? People who do well on the SAT subsequently put more stock in it and people who do poorly on the SAT do the opposite (what you assume of us) . I highly doubt that you went into researching the GRE's validity as a bastion of purely objective, scientific thought. Your own bias doesn't change the numbers - which look good here-but it does mean you are being a sanctimonious ass.

 

Just because I disagree with you doesn't mean I am on a high horse. I never claimed intellectual/moral/etc. superiority. I simply pointed out that the evidence is there to look at is more objective than the rash of overly emotional opinions on the matter. The issue was, and is as evidenced in your post, that you can't have a simple discussion about this topic without resorting to petty ad hominems and emotional appeals.

 

Your selection bias argument actually works against your point because the range restriction that would occur from only receiving people who scored highly on the GREs lowers the correlation that would occur if we included the full range of scores.

 

Second, I did not act as though I am immune to bias; it's inherent in being human. The point is to recognize that, which I didn't see in the knee-jerk reactions to GREs and frankly pure rationalizations about it being an invalid predictor of grad school success. The larger point is that the research is already out there--but most of you were more interested in tearing the test down instead of looking for objective evidence of the test's validity. That is the problem. 

[TXInstrument11]

Just because I disagree with you doesn't mean I am on a high horse. I never claimed intellectual/moral/etc. superiority. I simply pointed out that the evidence is there to look at is more objective than the rash of overly emotional opinions on the matter. The issue was, and is as evidenced in your post, that you can't have a simple discussion about this topic without resorting to petty ad hominems and emotional appeals.

Your selection bias argument actually works against your point because the range restriction that would occur from only receiving people who scored highly on the GREs lowers the correlation that would occur if we included the full range of scores.

Second, I did not act as though I am immune to bias; it's inherent in being human. The point is to recognize that, which I didn't see in the knee-jerk reactions to GREs and frankly pure rationalizations about it being an invalid predictor of grad school success. The larger point is that the research is already out there--but most of you were more interested in tearing the test down instead of looking for objective evidence of the test's validity. That is the problem.

No, it is quite clear in your post that you see yourself as taking the moral and intellectual high ground. You don't see how condescending your entire tirade was because you think you are there. Edited March 19, 2015 by TXInstrument11

[TenaciousBushLeaper]

How can you then suggest using subject GREs when it would further disenfranchise those who are already struggling to pay for the general GRE in the first place? The problem is that we live in a limited world with limited resources and many moving parts. This is life and you deal with it.

 

The overarching concern I have with all that I have read in this thread is this contempt for the GRE and dismissal of this validity because you don't *like* what it stands for in your minds. This constant refrain of "well grad schools need something to winnow down the list" and "it's just a way to narrow the list down." The validity of the GRE is an *empirical question* and isn't subject to your whims or values. 

 

I too made a point similar to the upper portion of this quote, I'm biased in this because I'm perfectly happy with my GRE Q score. Coming from what I think would fit in the category of a "disenfranchised" background and having friends who also come from this background who haven't done so well on the GRE Q section(friends who I would argue are far more intelligent than I am) I see it as a bit of a cop-out.

Also, faculty from 2 universities have explicitly told me, if the applicant isn't at least at the 50% mark on either the GRE Q or V sections most of the time they aren't even considered. Usually it takes a faculty member who's championing for these students to have them considered along with everyone else who did well on the GRE sections. 

Edited March 19, 2015 by TenaciousBushLeaper

[TheMercySeat]

I, for one, remain underwhelmed with that -.08 correlation!!!

For such a high-stakes exam, I think it is reasonable for one to expect stronger correlations. It looks like things get a little funky after 1st year GGPA, too.

The financial burden argument makes no sense... There's a simple solution: universities can scrap the general GREs and use the subject GREs. Such a policy change would save students $35.

Consider also that women and minorities have lower means on all subscales of the GREs. If the GRE truly predicts graduate success, then wouldn't it be a lost cause to let women and minorities into graduate school? How can one justify having nonwhites and women in higher education?

Edited March 19, 2015 by TheMercySeat

[Engali]

No, it is quite clear in your post that you see yourself as taking the moral and intellectual high ground. You don't see how condescending your entire tirade was because you think you are there.

Think I am where?

 

So your argument amounts to re-assertion of your point? I made a claim that there was a lack of intellectual honesty when it came to examining the *validity* of the GRE in predicting grad school success. The point wasn't whether or not who was more or less biased or smart, but the willingness to look up research and look at the data/results. FFS the Psych Bull article was free on Google Scholar and published over a decade ago in one of the best journals within psychology.

 

The constant nitpicking about how it doesn't predict all indices of performance perfectly ignores the larger context of nothing be a near perfect predictor or the fact that nothing thus far predicts grad school performance better as a weighted composite of predictors. Just look at table 9 and see how much more predictive UGPA is beyond the GRE.

 

It's like when you questioned the "selection bias" of the article without thinking through the implications of what that would mean. You wanted to prove your point so you threw something out there that worked against your argument because you didn't think through what the "selection bias" would actually do to the data. This is what I am talking about regarding intellectual honesty: thinking through what the data and methods are actually telling us instead of what we want it to tell us.