Welcome to the GradCafe

Hello!  Welcome to The GradCafe Forums.You're welcome to look around the forums and view posts.  However, like most online communities you must register before you can create your own posts.  This is a simple, free process that requires minimal information. Benefits of membership:

  • Participate in discussions
  • Subscribe to topics and forums to get automatic updates
  • Search forums
  • Removes some advertisements (including this one!)
Sign in to follow this  
Followers 0
  • entries
    4
  • comments
    37
  • views
    31,251

The Inconvenient Truth about the GRE - Part 2 - Some Evidence

TheFez

17,087 views

blog-0674886001360639690.jpgAh ye of little faith… I said this blog would be one of inconvenient truths.

You will come to trust the Fez, but ‘til then there is plenty of evidence to back this up...

Here’s link to a February 2007 Science article (Kuncel, N.R., Hezlett, S.A., Standardized Tests Predict Graduate Students' Success, Science, Vol 315 No. 5815, pp 1080-1081) that examines the GRE and GMAT:

https://apps.cla.umn.edu/directory/items/publication/292812.pdf

“Four consistent findings emerged: (i) Standardized tests are effective predictors of performance in graduate school. (ii) Both tests and undergraduate grades predict important academic outcomes beyond grades earned in graduate school. (iii) Standardized admissions tests predict most measures of student success better than prior college academic records do. (iv) The combination of tests and grades yields the most accurate predictions of success.”

Concluding that:

“Results from a large body of literature indicate that standardized tests are useful predictors of subsequent performance in graduate school, predict more accurately than college GPA, do not demonstrate bias, and are not damaged by test coaching. Despite differences across disciplines in grading standards, content, and pedagogy, standardized admissions tests have positive and useful relationships with subsequent student accomplishments.”

There are plenty of other studies that validate the GRE, (like one published in 2001 in the Psychological Bulletin (Vol. 127, No. 1) But the 2007 study has the prettiest graphs.

As for the GRE’s measure of innate intelligence, you could trust the guys at Mensa who have accepted high GRE scores in lieu of a 132 IQ score on the Stanford Binet IQ Test, but for more scholarly proof you check out Carvajal and Pauls, (1995), “Relationships among graduate record examination scores, Wechsler Adult Intelligence Scale-Revised IQs and undergraduate grade point average”. College Student Journal, 29, 414-416.

Here’s a link to the original research thesis in case you don’t have easy journal access

https://esirc.emporia.edu/bitstream/handle/123456789/1709/Pauls%201994.pdf?sequence=1

They find a strong positive correlation between the GRE and IQ. Full scale IQ with GRE-V (r=.63) and with GRE-Q (r=.71)

But my point really was – with enough effort can score well enough on the GRE for most programs, who don’t accept people based on their GRE or GMATs, but they often use them to help make their first cuts.

Fez Out.




12 Comments


You seem to be ignoring the fact that the effectiveness of said test to measure ones ability in a graduate program drops off significantly at anything over 70th percentile. I would venture to guess that the difference between someone who scores 70th percentile on verbal and 90th percentile on verbal is negligible at best. I say that as someone who scored over 90th percentile on verbal. 

Share this comment


Link to comment

You seem to be ignoring the fact that the effectiveness of said test to measure ones ability in a graduate program drops off significantly at anything over 70th percentile. I would venture to guess that the difference between someone who scores 70th percentile on verbal and 90th percentile on verbal is negligible at best. I say that as someone who scored over 90th percentile on verbal. 

 

Oh no, please let it be true--I scored in the 93rd percentile for verbal; I guess that means grad school will be a breeze for me! And how can I not get in? After all, my GRE verbal score is pretty damn good.

 

What's that you say? The GRE doesn't guarantee admission, and if I matriculate, I still have to show up, make an effort, study, read, contribute, participate, and do in-depth work both inside and outside of the classroom? But...but...I did well on one section of one very specific test.

 

Dream killer!

Share this comment


Link to comment

You seem to be ignoring the fact that the effectiveness of said test to measure ones ability in a graduate program drops off significantly at anything over 70th percentile. I would venture to guess that the difference between someone who scores 70th percentile on verbal and 90th percentile on verbal is negligible at best. I say that as someone who scored over 90th percentile on verbal. 

 

I would agree with this.  I am also a 90%+ in V&Q.

 

Unfortunately, I think the GRE is a necessary evil.  Because grad programs spend hard-sought federal and state dollars on their students, they want to, quite rationally, minimize risks. Top programs want to also maximize returns.  One could object to the methods of the studies cited above, however if you accept their conclusions then the GRE is a reliable metric, but not a perfect one. 

 

This is an important distinction. There are definitely people who have extraordinary talents but do not perform on the test for a various reasons, and there are some valid and legitimate excuses for an underwhelming score.  In that case it must be up to the applicant to go to extraordinary measures to make a case for his/herself.  The bigger problem I think is whether AdComs, in the context of increased competition for funds and larger applicant pools, are tending towards a more narrow definition of what they see as student potential; a "fit-to-the-mold" test rather than a consideration of what originality they might bring.  At least in science & eng. it seems to have some elements of a job-hiring process.

 

I was very worried about my grades and GRE (fortunately turned out alright), so I went to extra lengths to "sell myself" to several depts I applied to.  For example, targeted and thorough reading of Prof's current/past research, devising intelligent questions for them, and communicating directly.  Making a simple website with your research experience that can be sent to POIs.  It's not the most flattering language, but I do think you had to think about this process as "selling" -- make a clear and compelling case why you would be a great researcher.  

Share this comment


Link to comment

All of your sources are older than 5 years old. I want newer proof please. 

I would endorse this call for more sources, if it is made in the right spirit: are there any studies that have demonstrated the effectiveness of the Revised GRE?

 

However, if you want newer proof because you believe that evidence has a shelf-life, and thus anything over five years old can no longer be considered applicable, then I cannot support the motion; I think, even after thousands of years, the validity of Euclid's proof of Pythagorean theorem has not decreased in the slightest; and while it may be foolish to claim that TheFez has stumbled upon a truth we might consider "eternal," we should perhaps open ourselves up to this inconvenient wisdom.

 

Do we have any data suggesting a correlation between the time a student spends excoriating the GRE for its being worthless and unfair (independent variable) and his/her GRE scores (dependent)? There's a graph I'd like to see!

Share this comment


Link to comment

Oh no, please let it be true--I scored in the 93rd percentile for verbal; I guess that means grad school will be a breeze for me! And how can I not get in? After all, my GRE verbal score is pretty damn good.

 

What's that you say? The GRE doesn't guarantee admission, and if I matriculate, I still have to show up, make an effort, study, read, contribute, participate, and do in-depth work both inside and outside of the classroom? But...but...I did well on one section of one very specific test.

 

Dream killer!

 

Uh, guys, my post above was a joke.

 

While I don't think the GRE is completely without merit (I've even defended it before), I don't think that someone's performance on one test on one given day can say much about a person's potential academic or professional performance--often in an excruciatingly specialized field--for years to come.

 

I scored well on the GRE verbal because I have a memory for and genuine interest in vocabulary; I wouldn't say that my vocabulary knowledge makes me smarter, more capable, or more ambitious than my competition. Quite honestly my vocab skills won't help me do anything but learn material a bit faster and write better reports, and once everyone else in my professional program acquires the same field-specific lexicon, my "talent" will seem even less impressive.

 

Anyway, there are numerous factors that affect standardized test scores that truly have little to do with innate intelligence, intellectual curiosity, or hard work, and I think the predictive value of such a test is specious at best, harmful at worst. But it's part of the admissions game, and that's why I took the test.

 

Interesting links:

 

http://ubiquity.acm.org/article.cfm?id=1071921

 

http://www.dukechronicle.com/article/why-gre-scores-might-matter

 

http://www.news.cornell.edu/releases/Aug97/GRE.study.ssl.html

Share this comment


Link to comment

Am neither strongly for nor against the GRE.

 

But purely in response to Arianna...there are actually tests and means to measure such qualities...e.g. see link below on research on measures of motivation. I am sure similar work exists in areas of creativity.

 

http://www.unh.edu/personalitylab/Reprints/RP2007-MayerFaberXu.pdf

 

How effective these are, and the fact that these can measure such qualities only at a point in time makes them very subjective. And of course the GRE does not incorporate these. But then neither does GPA (explicitly at least - there is always an argument for implicit measurement) or any other aspect of the application packet. At the end of the day, the GRE is one more tool in an application package comprising of many variables - it is helpful when comparing students across different universities, countries, backgrounds etc. when no other common measure is available. It is unlikely to be effective as the sole predictor of graduate success, yet, it is designed quite scientifically and I personally would not dismiss it as completely irrelevant. One could make a similar argument for all data points that the admission committee sees, and short of spending a few months with a person, it is very difficult to get an accurate view - but I would rather depend on the GRE and a few other data points than let it be a completely random draw of luck !

Share this comment


Link to comment

Its Just a test, an requirement in the application then why complain about it? well GRE is no measure about your intellect but then its pointless to discard a simple test. The thing i dont understand esp for quants sections is, 1 month decidication and you will be in early to mid 140s. That should be good for the humanities and lit dept isnt it. Plus i am sure regardless of the dept you want to apply, everyone have studies basic algebra and math in school. In a way they can check how much can you retain. The same goes for Verbal. Its more difficult for internationals whose mode of conversatoin in every facet of life in non-english. Even they have been able to get some decent score.

 

The only stupidish part if essay. They have be evaluated case by case not in format (% paras with point A B C D)

Share this comment


Link to comment

I am hoping that you actually read these papers that you cited.  Even if you did (which I hope you did!), you need to read critically and you can't take the results as "truth."  Let's examine what the authors provided as evidence.  

 

The first article in Science has presented fairly weak correlations (mostly r <= 0.4) between the GRE and any of the career outcomes.  Degree completion would be a huge one that graduate programs care about and it is correlated with the GRE at r = 0.20.  Yikes-- that is low! The qualifying exams has one of the higher correlations (r = 0.40) but that is not too surprising since both are high stakes tests.  You can imagine people who do well in one high stakes testing environment would also do well in another high stakes testing environment-- but the correlation is only modest at best.  I wouldn't want to be making predictions based on a 0.40 correlation.  Graduate school GPA (overall and in the first year) is also only correlated with GRE at r = 0.40 according to this article.

 

The second piece of evidence that you link to is a master's student thesis.  The sample that this author uses is problematic to assert that GRE is highly correlated with IQ for all graduate students.  She is examining 30 clinical psychology graduate students at one institution. Her findings are not generalizable to all graduate students because she does not have enough students in her sample (30 is extremely low for a quantitative study in general!), but also she only examines one discipline in one institution.  The correlations that she finds are stronger (0.5 to 0.7) between GRE and IQ but unfortunately her evidence is not convincing since her sample is not adequate to make inferences to all graduate students.

 

 

If your hypothesis is there is some merit in using the GRE for graduate admissions because it predicts Y (Y being IQ or GPA or degree completion), I would read and review the literature more carefully and present better evidence.

Share this comment


Link to comment

I am currently taking Psychometrics with a professor who specializes in measurement theory and testing, and I have to say that he is a staunch advocate of the GRE. He has assigned us a meta-analysis by Kuncel, Hezlett, & Ones (2001) that provides a compelling case using an enormous sample size for how the GRE is related to graduate and career outcomes. (in my field, usually "big-impact" meta-analyses get published once a decade or so, so there aren't any more recent meta-analyses that I'm aware of).

 

All that being said, however... I think there are some serious flaws to the GRE. No one can deny that the GRE is highly related to our "traditional" conception of general intelligence (e.g., verbal/ quantitative/ spatial), but I personally believe that there are some serious confounding factors, such as socioeconomic status, marital status, having children, etc.that interfere with the GRE's ability to correctly predict a student's potential. I was lucky enough to be able to take 3 months off work to study for the damn thing (my husband is awesome and supported me through the time), and I raised my score from the 50th percentile in quant and 70th percentile in verbal to 82% quant and 98% verbal. If I hadn't had the time and resources to study, I could have never improved my score. My experience has firmly led me to believe that intense studying and practice can substantially improve your GRE score. People who don't have the time, money, or resources to study are at a serious disadvantage. 

Share this comment


Link to comment

Remember, with a correlation of .63/.71 it is still possible for someone with a score of 1250 (outlier) to score the max/same on an IQ test as someone with a 1600 (higher than average). Regression simply performs a line of best fit, although most lie close to the center there are exceptions.

Share this comment


Link to comment

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!


Register a new account

Sign in

Already have an account? Sign in here.


Sign In Now