Fundamental flaw in GRE reading comprehension test

[canberra]

There is nothing wrong with that passage. In my opinion, it reads very well. Do the commas confuse you? I suppose the last few words could have said something to the effect of "in many cases, the only women employers would previously hire." What is your criticism here? How would you have written the passage?

It is not a passage but a single sentence! But it is so long that you mistook it for a passage, which kind of proves the point.

I think the author's already done a good enough job of re-working the sentence:

In the 20th century, more married women had jobs because they needed the money – not just because domestic appliances freed up their time. Also, employers used to only hire single women, but their supply was cut by high marriage rates.

[spctle342]

It is not a passage but a single sentence! But it is so long that you mistook it for a passage, which kind of proves the point.

I think the author's already done a good enough job of re-working the sentence:

In the 20th century, more married women had jobs because they needed the money – not just because domestic appliances freed up their time. Also, employers used to only hire single women, but their supply was cut by high marriage rates.

I didn't "mistake" it for a passage; I suppose I should have said, "There's nothing wrong with that excerpt from that passage." I know what a sentence is. And I don't care for the "re-worked" sentence. It sounds like something you might read in middle school. It also doesn't really make sense. "Blah blah -- not just because blah blah." isn't a sentence, it's a fragment. To begin the next sentence with "Also" when no argument was made about why women had jobs in the first sentence is grammatically incorrect. I agree that the sentence could be broken into several shorter statements, but the example you provide doesn't do that effectively.

Edited May 16, 2011 by spctle342

[canberra]

I didn't "mistake" it for a passage; I suppose I should have said, "There's nothing wrong with that excerpt from that passage." I know what a sentence is. And I don't care for the "re-worked" sentence. It sounds like something you might read in middle school. It also doesn't really make sense. "Blah blah -- not just because blah blah." isn't a sentence, it's a fragment. To begin the next sentence with "Also" when no argument was made about why women had jobs in the first sentence is grammatically incorrect. I agree that the sentence could be broken into several shorter statements, but the example you provide doesn't do that effectively.

Even if the author is wrong in how he's re-worked the sentence, he's broken it down into shorter chunks, something you've agreed with. I'm sure you would do a better job of reworking the sentence and writing it more clearly. If the GRE passages are written more clearly, they'll be easier to comprehend, which in turn means that the test-takers' scores currently cannot entirely be a reflection of ONLY their reading comprehension ability. Comprehension naturally depends on how clear the writing is too.

[Ennue]

Everybody can understand 'Miffy went to the zoo'. There have to be things that are difficult to understand (i.e. which require good reading comprehension), because otherwise everybody will get the same score.

[Eigen]

Even if the author is wrong in how he's re-worked the sentence, he's broken it down into shorter chunks, something you've agreed with. I'm sure you would do a better job of reworking the sentence and writing it more clearly. If the GRE passages are written more clearly, they'll be easier to comprehend, which in turn means that the test-takers' scores currently cannot entirely be a reflection of ONLY their reading comprehension ability. Comprehension naturally depends on how clear the writing is too.

And the better able you are to comprehend difficult passages, the higher your reading comprehension is. To some people, the passage you keep citing makes perfect sense with a single read through. To others, they have to re-read it a few times to make sense of it. This will likely differentiate their scores in the verbal section, as the former group has better reading comprehension than the latter.

Your argument is kind of like saying "If all word problems set up the equations you need to use, people would get them more easily"... Which is true, but having to think your way through a complex passage separates people according to their abilities. Some people read a problem through and know the answer almost right away. Some people have to struggle over how to frame it. One group should get a higher score than the other.

If the problems on the GRE were easy enough that everyone could make sense of them right away, it wouldn't be a very good test.

[spctle342]

Even if the author is wrong in how he's re-worked the sentence, he's broken it down into shorter chunks, something you've agreed with. I'm sure you would do a better job of reworking the sentence and writing it more clearly. If the GRE passages are written more clearly, they'll be easier to comprehend, which in turn means that the test-takers' scores currently cannot entirely be a reflection of ONLY their reading comprehension ability. Comprehension naturally depends on how clear the writing is too.

I believe you misunderstood me. While I agree that it is possible to break the sentence down into shorter statements, I don't agree that it is necessary or even appropriate for the purpose of the GRE. To me, reading comprehension doesn't mean regurgitating the statements in front of your face, it means interpreting them correctly. If the sentence clearly said "More married women started working because x and not y," followed by a question about why women started working, you don't have to comprehend anything.

I agree with you only to the extent that I think it makes sense to provide passages of varying difficulties (though, as someone who has not taken the GRE, I assume that's already the case). If all the passages are above everyone's head, then the test wouldn't be an accurate measure of reading comprehension. The fact that some people do answer the questions corresponding to "difficult" passages correctly leads me to believe that the test isn't flawed/biased/inappropriate. Even the author of the original article is making the case that these passages can be understood with a little thought. My view is that the person taking the test should be expected to decipher what is being said; it shouldn't be spoonfed to them.

[bigant]

Most of the comments seem to agree that the GRE passages aim should be to filter out candidates through the perecentile system, and therefore that the passages should be difficult. But I agree with the author of the article that the aim of a writer must be to communicate well so that the readers easily understand the points being made. The points being made can be complex, but the language must not be. Otherwise we spend our time arguing about and interpreting what the author is trying to say, when instead we should be debating the points being made (which we are clear about, because they have been communicated clearly)..

[bigant]

Just came across an article in Forbes magazine, underlining the point the author of the original article is making.

Why Trying to Learn Clear Writing in College is Like Trying to Learn Sobriety in a Bar

http://blogs.forbes....riety-in-a-bar/

The focus on clear writing has to start early, and the GRE testing process should re-enforce the point, rather than add to the problem.

Edited August 2, 2011 by bigant

[finknottle]

Imagine if someone were to write a passage like that for the AWA section. I bet he would get an abysmally low score. So, in essence, what the GRE is telling you it broadly this: We expect you to write precise, objective and crisp sentences which when read as a passage constitute one well reasoned theory or argument. That is what you are supposed to do in "academia". However, at the same time, we also believe that nobody really follows that in academia, and we'll give you a taste of that in the next section.

Edited August 22, 2011 by prasun

[Eigen]

I take it like this:

We expect you to be able to both write concisely and clearly and be able to comprehend complex and twisted sentences.

The two aren't mutually exclusive- and in fact, I find one of the things academics spend a lot of time doing is taking complex and twisted writing, processing it, and putting it in a clearer and more concise form- ie, teaching.

[finknottle]

I take it like this:

We expect you to be able to both write concisely and clearly and be able to comprehend complex and twisted sentences.

The two aren't mutually exclusive- and in fact, I find one of the things academics spend a lot of time doing is taking complex and twisted writing, processing it, and putting it in a clearer and more concise form- ie, teaching.

The question is not about mutual exclusion of skill or abilities. The issue is regarding how well reasoned and realistic the test is. Most of the passages that the test gives are dry, drab and convoluted. It is clearly giving out the message that that is what you should expect to find in academia. There are two problems with that. One, that most of the passages that GRE gives are intentionally or artificially convoluted. I have never read a paper that is so badly written in my areas of interest at least. The passages are not fair representatives. You might say that GRE tests and prepares you for the worst case scenario, but that is just absurd. That's like testing someone's ability to understand the functioning of a device with a poorly written manual at hand. Why would you want to test people's ability to do that? The other point is that if GRE truly believes that the passages are fair representatives, then that would imply that that is the norm in academia. If that is the case, the AWA makes no sense. Why would you test people's ability to write precisely against the norm?

[Eigen]

Actually, it's about spreading out test scores for a good resultant range.

You have the "average", which is a reasonable understanding of reasonably complex writing. Then as you tend to the higher scores, the writing is more artificially convoluted to see if the reader can keep up.

For most disciplines, a nice 500ish V score would be considered "solid", and the writing really isn't that twisted at that level.

But any good test should spread out the scores such that you can see definite striation at the high ends of the test- separate out the "average" from the "good" from the "very good", so to speak. Using artificially convoluted passages to do that may come across dry or artificial, but it serves it's purpose.

I don't know why you feel that the passages should be "realistic" in relation to a selection of academic writing- whether they're dry or drab doesn't matter, really, for testing purposes- just the accuracy of understanding relative to the degree of convolution in the writing. Nor do they need to be "passages representative of academic writing".... The GRE doesn't claim to measure someones ability to do well in graduate school, but rather an objective measure of reading comprehension, writing ability, and basic mathematical knowledge that can be evenly applied across the board, and it does that rather well.

[finknottle]

I don't know why you feel that the passages should be "realistic" in relation to a selection of academic writing- whether they're dry or drab doesn't matter, really, for testing purposes- just the accuracy of understanding relative to the degree of convolution in the writing. Nor do they need to be "passages representative of academic writing".... The GRE doesn't claim to measure someones ability to do well in graduate school, but rather an objective measure of reading comprehension, writing ability, and basic mathematical knowledge that can be evenly applied across the board, and it does that rather well.

Therein lies the problem. At least I thought it did, and as far as I know, that's how most people, including admission boards of universities view it. Why else would it be mandatory for most graduate programs? It needs to be realistic and representative of the real academic world. If it's just about doing well in the test for the sake of doing well and proving that you are smarter than the others, you might as well write the MENSA test. If everyone agrees that it doesn't measure the ability to do well in graduate school, it would just make the test redundant. You might as well do away with it.

[Eigen]

No single thing measures your ability to do well in graduate school.

Your application, then, is a combination of parts that support your ability to potentially do well in graduate school.

The *main* parts of your application are your transcripts (detailed accounts of your academic abilities) your letters of recommendation (detailed accounts of your personal strengths/weaknesses) and your statements (detailed accounts of your research experience, goals, ideas, etc).

The problem with those portions are that they cannot be compared between schools with any degree of objectivity.

Enter the GRE general test, a test that measures very basic abilities in two arenas- reading comprehension/writing ability, and mathematics. It doesn't measure either at very high levels, but it is meant to provide an objective major of skills that are useful to graduate students- how well can you decipher and analyze complex writing, how well can you concisely and logically present a written argument, and how well you can do basic mathematical manipulations. These things are useful to all fields, but in no way are the major determinant in whether or not you're successful in graduate school, just like the ACT and SAT are nowhere near accurate measures of undergraduate performance.

What these standardized tests *do* accomplish is to provide an objective measure of basic skills- hence why they are more often used as "cutoffs" for admissions rather than a significant weight in comparing two applicants.

No committee looks at one applicant with a 550 V and one with a 620 V and thinks that the latter applicant will be more successful in graduate school- they apply a sorting criteria to weed out all applications below what they consider to be a minimum acceptable score, and then compare the more pertinent parts of the remaining applications- how well they write in their writing sample, how they've done in they're courses, and most importantly, what kind of a researcher they seem to be.

I think most applicants make the GRE out to be "more" than it is- but I have yet to hear an admissions committee think that the GRE score is an important and accurate measure of much, other than an objective minimum standard for admittance.

[finknottle]

Yes that's how it should ideally be. Yet, people continue to fuss over it. In fact, lot of universities clearly mention the minimum cutoff, but at the same time report out previous years' average scores and trends. The average scores are obviously much higher than the cutoffs. The cutoffs themselves are pretty high in some cases. That just adds to the confusion. Nobody wants to regret missing out on an admit later due to a mediocre GRE score, and hence they would try their best to do as well as they can, or in some cases write it again as well to improve the score. My point is that either it shouldn't be there at all, or there should be standardized cutoffs and criteria for judging the GRE score, or it should be more meaningful.

[Eigen]

Please support your point.

Applicants fussing over published averages isn't a valid reason for re-tuning the GRE, or forcing schools to use it differently.

No schools have "standard" criteria and cutoffs for anything else in their application, why should the GRE be different? In other words, why should MIT use the same cutoff criteria as Ohio State? Each school sets their own standards for the interpretation of the GRE, and that's perfectly fine. They ask for the data they want to use to make what they feel is an informed decision about admits... And use it how they see fit.

The schools that do use a "cutoff" criteria usually have it well posted.

Just because admitted applicants have a high average GRE score doesn't mean that high GRE scores are required by the schools- you might be confusing correlation with causation, here.

For the record, none of the domestic students in my program studied for the GRE at all, to my knowledge- nor did they stress about their scores. They took it, submitted the scores, and went on with the rest of their application.

People worry about needing to have high scores, yet there are plenty of cases you can find on this very board where students with GRE scores below the cutoff have been admitted to quite prestigious programs, because the rest of their application warranted it.

TL;DR: The confusion comes not from the schools use of the GRE, but rather from applicants overthinking it. Why should the schools change what materials they use to make their decisions based on the applicants confusion?

Edited August 23, 2011 by Eigen

[finknottle]

Ok, we are digressing here from the original post and our posts are getting longer. I'l try to be brief.

If the GRE is clearly not a metric of technical aptitude or success in grad school, it should largely be ignored by most schools with just the requirement of clearing a cutoff with minimal variance across schools. Something like the TOEFL which is not nearly as important as the GRE. However, that is clearly not the case. Different schools attach varying amount of importance to the GRE score which just reinstates the fact that it is used as a metric, no matter how little or how much its importance is.

Given that it is used as a metric, it makes little sense to have a test with little correlation to what the test is used for. i.e. Grad admissions. Hence, the argument for it being relevant, realistic and representative and the original post in this thread.

[Eigen]

You're basing this off of the assumption that some schools put a lot of emphasis on the GRE...Do you have any data to support this?

That said, I don't think we agree on the usefulness of the GRE as a measure of graduate school aptitude. Just because it isn't perfectly filled with real world examples doesn't mean it doesn't properly measure a specific set of skills that are essential for graduate school:

1. The ability to understand convoluted writing.

2. The ability to write concise and logically consistent arguments.

3. Basic quantitative critical thinking.

I would argue that within the confines of the test, the GRE provides a very nice, objective measure of the above 3 skills- hence its usefulness in graduate admissions. Hence, I see no "fundamental flaw" in the GRE reading comp test- I think it's a reasonable measure of the ability to understand passages of increasing complexity.

Edited August 23, 2011 by Eigen

[wtncffts]

You're basing this off of the assumption that some schools put a lot of emphasis on the GRE...Do you have any data to support this?

That said, I don't think we agree on the usefulness of the GRE as a measure of graduate school aptitude. Just because it isn't perfectly filled with real world examples doesn't mean it doesn't properly measure a specific set of skills that are essential for graduate school:

1. The ability to understand convoluted writing.

2. The ability to write concise and logically consistent arguments.

3. Basic quantitative critical thinking.

I would argue that within the confines of the test, the GRE provides a very nice, objective measure of the above 3 skills- hence its usefulness in graduate admissions. Hence, I see no "fundamental flaw" in the GRE reading comp test- I think it's a reasonable measure of the ability to understand passages of increasing complexity.

I agree with this in terms of the caveat you express: "within the confines of the test". This is really a problem with tests in general, and not just the GRE, but actual academic work and research isn't done in test-like conditions. You're not prohibited from consulting sources, talking with others, heck, drinking and eating. You're not under artifical time constraints. In real life, you're able to, and frequently should, go back and reread things, highlight, make notes, etc. Good academic writing isn't a matter of what you can dash off in an hour without much forethought or much revision, but a constant process of writing, rewriting, editing, thinking, and so on.

To your first point, it's much easier, in real life, to read and understand "convoluted writing" when one is familiar with the jargon and ways of expressing things in a particular discipline, and how what you're reading fits into a larger picture. That's where the 'examples' of writing on the GRE fail to test the ability you're suggesting, since, as in the first example of the link in the OP, they're often far outside most of the test-takers' areas of knowledge. It's unfair, in my opinion, to give me a dense text in, say, biology, and make my comprehension of it the measure of my intellectual capacities.

In essence, I'd suggest that the points you mention really can't be truly measured by any standardized test such as the GRE, except perhaps the quantitative component. Even then, as I've alluded to, the whole test environment is artificial, and as this forum attests, some people who are otherwise eminently capable just don't do well in such conditions. I see no reason to believe that the GRE is useful as a true barometer of abilities, and given the costs, mental and material, that it has for many aspiring graduate students, I would rather it not be used. This is not some personal gripe, either; I did very well on the GRE (though it didn't seem to help much), but I have always thought it an unnecessary, detrimental in many cases, part of this whole process.

[Eigen]

I think the point that you bring up (that no test can truly provide a measure) is quite pertinent- and is at the crux of this debate. There's no real way to completely accurately measure any such skills by testing, and the GRE is no exception.

That said, when comparing students from all different schools and systems all over the world, having some baseline measure can be quite helpful- and that, in my mind, is what the GRE provides. It's not a central factor to admissions (in most cases), but it allows Adcoms to compare the range of students with some metric, and can be especially helpful when it comes to schools (or countries) with which they have no direct knowledge.

Personally, I have yet to talk to any prof doing admissions that thinks the GREs are very important- I think it's primarily applicants that place such a large amount of importance on them.

To your example of texts from outside the readers fields- none of them are outside of what someone could be reasonably expected to understand. The section you refer to was not "dense in biological terminology", but was rather a passage with some biological trappings, that should be interpretable to almost anyone that would be taking the GRE. By your logic, what passages could we use? Nothing could reference literature, history, sociology, politics, or current events.

I understand your point of view, but in my point it's not that the test shouldn't be used, or that the test is flawed, but rather that it should be understood to have specific constraints- within the constraints of the test, it's a good measure of those skills. But that's true for any test or measure. One of the important things in research in general is to understand that *any* data you collect from *any* source is only valid within the context of the source! The GRE is no exception to that rule, and other than people posting on these fora, I have yet to hear anyone suggest that the test scores are generalizable.

[finknottle]

You're basing this off of the assumption that some schools put a lot of emphasis on the GRE...Do you have any data to support this?

The lack of data to prove things one way or the other is exactly what I was talking about. Schools would publish cutoffs and averages separately. What that insinuates is that unless the rest of your profile can outweigh your somewhat low score, the average score that the schools publish should serve as the score that you should strive to achieve. In any case, the relative importance of GRE in relation to other things is not the point of contention. Everyone agrees that it is/should be used only as a preliminary filter. However, a lower than average score might put you at a slight disadvantage.

I think the point that you bring up (that no test can truly provide a measure) is quite pertinent- and is at the crux of this debate. There's no real way to completely accurately measure any such skills by testing, and the GRE is no exception.

Personally, I feel that the AGRE (subject GRE) makes more sense than the GRE. It at least tests your abilities in areas that you are applying to and not the ability to read arbitrary passages. As for basic reading, writing and speaking, tests like the TOEFL are far more practical. A combination of the AGRE and something like the TOEFL would be a more balanced approach.

[chaospaladin]

The math section is too easy and the verbal is too difficult.

[Kitkat]

Is there anything wrong with a lower score putting one at a slight disadvantage?? Shouldn't that be expected?

Isn't it normal to assume someone who cant think quantitatively on the spot is not quite as good a candidate for grad school, as is the student who can't figure out convoluted writing?

A slight disadvantage, sure, but I hope not too much of one, if everything else in the applicants package says that the student would be perfect for the program, I hope not too much. If everything else in their application says that they can read and think quantitatively, without the time constraints, then why shouldn't they be good for the program?

If any part of the GRE should be changed, I think it should be less focused on vocab memorization. Why should they care how well I can momorize words that no one ever uses? Yet someone else might ask why they care what math skills one has, if they are studying English. I argue that anyone going into grad school is expected to have reasonable logic skills, as well as reasonable reading comprehension skills. Engineers and English majors are going to differ on what they are best at, but a basic understanding of both is a necessity.

I'm not so sure the ability to memorize vocab is a necesary ability though...

They have changed it so that it is less vocab dependent(although it still is somewhat dependent), by taking out the analogies and antonyms. So that is a step in the right direction. But now that leads us with more reading comp, which was the complaint in the first place, because people don't think that it is very good....

[Lox26]

I think we all agree that the GRE, as all tests, fails as a comprehensive assessment of the full range of an applicant's abilities. The score should still matter to adcoms/POIs, however. Different undergraduate programs vary with respect to rigor of the coursework. Perhaps no one should be given an automatic boost for attending a prestigious undergrad or have his/her abilities questioned for attending a lesser-known school. There are duds and standouts at both types of college. The GRE is (so far) the fairest way to determine desirable qualities, such as analytical aptitude, logical reasoning, and motivation (across a range of colleges, majors, research experiences, etc.).

Moreover, standardized tests are extremely learnable. There is a wealth of free information on how to master the GRE. A qualified, industrious person is more likely to actively seek that information out and more likely to achieve better-than-average scores. If anything, I would view the scores as indicative of the level of preparation one puts into a task, no matter how "meaningless" the hoop to entry (e.g., into a solid grad program). That does not mean that I think a 1600/340 applicant is necessarily stronger than a 1300/320 applicant. Still, the ability to deduce the concept tested on a math problem or identify the logical structure of a dry passage seem like baseline skills required for graduate study. The two are really just opposite sides of the same coin: both require identifying patterns and synthesizing information to answer a smaller intellectual question.

I would also caution against field-specific tests (AGREs) becoming the new standard. The current system allows for cross-disciplinary enrollment. By that I mean that non-biology majors, for example, may apply to biology grad programs. Perhaps the applicant majored in English and took a bio seminar in his/her senior year. Perhaps that was the catalyst to switch gears and pursue biology. That person may have strong analytical skills (Humanities majors read lots of dense texts and boil those texts down to their essence in order to construct a novel, cogent argument on a regular basis), glowing letters of recommendation, research, and publications in English/lit journals. What that person may lack is the depth of knowledge assumed of a bio major who has taken 70+ credits in the subject. Thus, the new system of AGREs may bias the application process against individuals who did not major in their chosen field of study (assuming adcoms would use this score as a screening criterion). This is problematic because an applicant who has yet to demonstrate proficiency in biology (e.g, honors coursework, high AGRE score) does not necessarily lack the capacity to learn and excel in the field. Ultimately, the system would re-construct the unfair assessment to which some posters have alluded.

There is no perfect test, but a standard is needed; the GRE serves this purpose.

[wtncffts]

I understand your point of view, but in my point it's not that the test shouldn't be used, or that the test is flawed, but rather that it should be understood to have specific constraints- within the constraints of the test, it's a good measure of those skills. But that's true for any test or measure. One of the important things in research in general is to understand that *any* data you collect from *any* source is only valid within the context of the source! The GRE is no exception to that rule, and other than people posting on these fora, I have yet to hear anyone suggest that the test scores are generalizable.

I guess I'm not quite sure what you mean here, because even to use the GRE as a "baseline measure", you inherently have to view the results as generalizable, as speaking to the candidate's capacities for graduate-level work, and not just a measure of how she did on a particular day with no implications for her abilities outside that specific test environment. It's certainly not true, as a general rule, that any data is only valid within the context of its source. External validity, i.e., how well a causal relationship using data gathered from a specific sample can be reasonably seen as indicative of an actual relationship in the population as a whole, is one of the foundations of the whole enterprise of 'science', is it not? Again, I'm not sure what you meant, so please correct me if I misread.

On Lox26's last point, first, I think you're overestimating how often such things actually happen, especially in non-cognate fields such as English and Biology. I can certainly see in my own field how students with backgrounds in economics, history, sociology, etc., can easily adapt to political science. It's much more difficult to see how someone, an English major with little to no background in mathematics, is going to be accepted into a math program on the basis of 'potential'. I somehow can't picture a member of a math adcomm looking at a writing sample on Flaubert's use of foreshadowing in Madame Bovary as neo-Marxian ubercritique (completely made up) and saying, "Well, this person hasn't done math since high school and wouldn't know how to differentiate a function if his life depended on it, but let's throw him into advanced mathematics courses. He has so much potential".

I think in certain fields, such as mathematics or physics where a certain level of technical knowledge is necessary, it makes eminent sense to use field-specific GREs. Perhaps less so in other fields, but I still think, in my field, for instance, a GRE testing basic knowledge of political philosophy, certain basic concepts of politics, etc., would be much more useful than the general GRE.