Fundamental flaw in GRE reading comprehension test

[bigant]

So, in essence, what the GRE is telling you it broadly this: We expect you to write precise, objective and crisp sentences which when read as a passage constitute one well reasoned theory or argument. That is what you are supposed to do in "academia". However, at the same time, we also believe that nobody really follows that in academia, and we'll give you a taste of that in the next section.

Well said!

Any fool can write convoluted sentences. Clear and crisp writing requires clarity of thought and a willingness to work hard to make the writing clear and engaging.

Several commenters have said they haven't fully read the article because it is too long. If you take the time to read the whole article, the author has given the example of the financial crisis, caused by "complex" language that created ` FUD (Fear, Uncertainty, Doubt). Nobody understood anything and the economy went over a cliff. It sounds like the ETS wants to create the same FUD in GRE candidates.

As the author said, the real question is what the aim of any piece of writing is, ie is it to communicate, or create FUD?

[Lox26]

I guess I'm not quite sure what you mean here, because even to use the GRE as a "baseline measure", you inherently have to view the results as generalizable, as speaking to the candidate's capacities for graduate-level work, and not just a measure of how she did on a particular day with no implications for her abilities outside that specific test environment. It's certainly not true, as a general rule, that any data is only valid within the context of its source. External validity, i.e., how well a causal relationship using data gathered from a specific sample can be reasonably seen as indicative of an actual relationship in the population as a whole, is one of the foundations of the whole enterprise of 'science', is it not? Again, I'm not sure what you meant, so please correct me if I misread.

On Lox26's last point, first, I think you're overestimating how often such things actually happen, especially in non-cognate fields such as English and Biology. I can certainly see in my own field how students with backgrounds in economics, history, sociology, etc., can easily adapt to political science. It's much more difficult to see how someone, an English major with little to no background in mathematics, is going to be accepted into a math program on the basis of 'potential'. I somehow can't picture a member of a math adcomm looking at a writing sample on Flaubert's use of foreshadowing in Madame Bovary as neo-Marxian ubercritique (completely made up) and saying, "Well, this person hasn't done math since high school and wouldn't know how to differentiate a function if his life depended on it, but let's throw him into advanced mathematics courses. He has so much potential".

I think in certain fields, such as mathematics or physics where a certain level of technical knowledge is necessary, it makes eminent sense to use field-specific GREs. Perhaps less so in other fields, but I still think, in my field, for instance, a GRE testing basic knowledge of political philosophy, certain basic concepts of politics, etc., would be much more useful than the general GRE.

You're assuming that an English major would avoid math classes in college or be bad at them. Some people are proficient at reading, writing, and math. Some of these people major in fields that emphasize only two of the three. An English major who is bad at math would not apply to a quant-heavy program. An English major who took math classes for personal enrichment (these people exist) and could see herself/himself doing an advanced degree might.

Econ, poli sci, and sociology need not be similar. That would largely depend on the coursework taken. And I could see an English major smoothly transitioning to poli sci, even in a more quantitative-research-heavy program. In every program, there are intro prereqs that students may or may not have been exposed to. That's why these courses are part of the core curriculum and why new MA/Ph.D. students begin the program with coursework and don't just dive into their niche research.

Ultimately, you appear to reject the notion that people outside of the hard or social sciences can excel in these fields. On what evidence are you basing that claim?

[Eigen]

Ok, let me go back and re-work that paragraph- I'd been up about 30 hours at the point where I wrote it, and that may have had an effect.

Science doesn't work by doing an experiment on a specific population, and generalizing those results to other related populations- it works by doing an experiment on a specific population, and theorizing that there may be similar effects in other populations. Those other populations must then have the same (or a very similar) experiment done on them to verify. Writing scientific papers for publication these days, it's extremely important to not oversell your work- whatever you show, you have to realize you showed in specific contexts.

For example: If I show that a drug delivery system that I've just designed works perfectly in a simulation of biological conditions (temperature, pH, concentrations of salts, etc), that's all that it showed. Then it has to go into cell cultures, and from cell cultures to rats, and from rats on up the food chain until it makes its way to humans. From that first paper, I can theorize that the delivery system will work in humans based on my experiments, but the results I'm presenting are only valid in the context of the experimental conditions I used. There's no way around it. The theorizing must be proven.

Similarly, you can use GRE results and correlate them with people who have done will in graduate school (and there are plenty of studies out there), but it shows nothing of causation. Really, the GRE shows aptitude for a specific skill set, within the context of the test- ie, one day, timed conditions, non-real world (and non-field specific) examples. Not to mention that there's a randomization factor that can really screw with it. As Lox26 points out, however, it's the best baseline for comparison between different schools/majors/countries that we have, and it's quite good at applying a baseline. I think the confusion from my post was in how you choose to generalize the data: You seem to be working from the idea that the test (taken on a specific day) is generalized to be a representative example of that individuals abilities as a whole (time-wise). And you're right, to use it as a baseline, you have to. I'm making the point, however, that it only measures an individuals abilities under a certain set of conditions- and that within those conditions, you can compare the abilities of different individuals. It's not a perfectly holistic measure of their abilities, but it does allow for comparison. The fact that its an imperfect test that only measures those abilities within a narrow range of conditions is why it's normally used as a cutoff instead of a direct comparison of aptitude for graduate studies as a whole.

Make more sense?

And to the other point Lox made: I know someone with an undergrad in PolSci who's starting a Neuroscience doctoral program. Field specific knowledge really isn't as important as showing you have the ability to learn and apply that knowledge. All the subject GREs do is show what you've been exposed to and can memorize- they don't show your critical reasoning, how well you can analyze and make arguments, etc- all things that are best shown in your SoP and your LoRs. And these are the more important facets in making a successful graduate student, imo. I can teach someone my field in 6-8 mos, if they're really interested and want to make it their life's work.

[bigant]

Rule from a book on writing ("What not to write") that I recently read:

"A reader should never have to read a sentence twice to understand its meaning".

GRE passages flout this rule all the time.

[bigant]

That's why it's a test on reading, not writing. What kind of reading test would it be if every reader was guaranteed to understand the first time through?

You seem to be agreeing that the GRE passages are written badly. This means that however badly the GRE passages are written, the reader should be blamed for not comprehending. We might as well get rid of all the English writing teachers in school.

Besides, what's wrong if everybody understands first time?

[InquilineKea]

Hey - I have a question: how common is it for people to actually *disagree* over what the correct answer in a GRE reading passage is?

And do you get this impression more on the review prep books than on the official exams? I've seen some badly written science answers on the Barron's science reading comprehension passage.

==

I do agree, though, that the reality is such that it actually *is* a valuable skill to be able to comprehend poor writing (and poor thinking along with that).

Edited September 29, 2011 by InquilineKea

[bigant]

It's a test. What good is a test if everyone gets 100%? I also don't think they were written grammatically poorly, nor do I think they left out any required information,but I am saying that a passage that is easily understood defeats the whole purpose of the test in the first place.

The issue is not poor grammar or completeness of information. The issue is clarity. You’ll agree that the use of punctuation is a part of good grammar. However if we look at a sentence from the example passage, punctuation is hardly in evidence:

The increase in the numbers of married women employed outside the home in the twentieth century had less to do with the mechanization of housework and an increase in leisure time for these women than it did with their own economic necessity and with high marriage rates that shrank the available pool of single women workers, previously, in many cases, the only women employers would hire.

As the author of the article says, would this sentence be of acceptable standard on the written portion of the GRE?

Besides, the mark of a real test is that if all the elements of good writing were present and a candidate still got poor scores, only then could the candidate’s reading comprehension skills be considered truly poor.

[Kitkat]

As a response to those people who say that if you have to read a passage more then one to really understand it, I would like to make a point on this as well for another interpretation. There are several books I have reread because I have enjoyed them. And every time I seem to bring out something new. Its not to say that they are written badly, or that I can't really understand things the first run through, but that at different points in my life I can read it differently. This might be more applicable to fictional works, but a different level of understanding in the reader can affect how they read a passage. This is why I think that so many of the prep books and courses say that if you are familiar with the subject matter in the reading passage might be more difficult for the person, because they are thinking about what else they know.

That is not the issue in a way that a person might think about the GRE passages, in the sense that many people think that they are just poorly written. But at the same time, I think that I have seen enough people on this website alone who have seemed to have done perfectly well on the GRE to think that it is not a matter of them being poorly written. It might be more of them being poorly edited for the test, because they do that. And I think that other posters are perfectly right to say that some of the stuff you will be writing in your future career will not necessarily be the best writing you will ever come across. It's not so much to ask that you can find the sense in that material as well as the better written material.

[habanero]

There are many poorly-written papers in academia. It does seem important to be able to understand confusing passages as quickly as possible.

The increase in the numbers of married women employed outside the home in the twentieth century had less to do with the mechanization of housework and an increase in leisure time for these women than it did with their own economic necessity and with high marriage rates that shrank the available pool of single women workers, previously, in many cases, the only women employers would hire.

This is ridiculous. Was it seriously on a GRE?!

Edited September 29, 2011 by habanero

[bigant]

There are several books I have reread because I have enjoyed them. Its not to say that they are written badly, or that I can't really understand things the first run through,

If these books were badly written, you would presumably not want to re-read them. We re-read books (out of choice) only because we think they're well written, and therefore enjoy them.

[Kitkat]

If these books were badly written, you would presumably not want to re-read them. We re-read books (out of choice) only because we think they're well written, and therefore enjoy them.

I question that point on the fact that a person can have a really good story, and have done that really well, but not have the best way of delivering said story. I could imagine that if they write badly enough, then yes it can seriously get in the way of a good story. Good writing makes a story better, bad writing can make it worse, but there can be a case for the middle ground where a good story makes up for so so writing style.

[bigant]

This is ridiculous. Was it seriously on a GRE?!

Official GRE Sample Questions:

http://www.ets.org/b...c3set1&onitem=1

See lines 34 through 42.

[Eigen]

I still don't really see what the problem is with that sentence. It's a bit awkward, but it's not like the meaning isn't clear on the first read through. There's nothing grammatically incorrect either- and punctuation *is* correctly used.

Would it be easy to clean it up a bit while still retaining the meaning? Sure. But that doesn't mean it's "poorly written", just that it's not an exemplary example of writing. The main thing it could use is an "oxford comma" in the middle of the sentence, but there's still a lot of debate on whether that would be proper usage or not.

But since the test proposed earlier was that you shouldn't have to read it more than once to get its meaning, I think it passes that fine.

You keep pointing it out as an obvious example, but I'd really like to hear you dissect what *exactly* is wrong with it. Not what you think you could improve, but things that are actually wrong.

Edited September 29, 2011 by Eigen

[route66]

At the very least, why can't the passages have headlines?

[Kitkat]

At the very least, why can't the passages have headlines?

I haven't come across it as a case for this new version of the test. But with other standardized tests, where one of the questions is what title you would give the passage. It might be that in the future that ETS might want to use that as a type of question in the future. I think that they are testing your ability to read and understand what the passage without the help of a title. Yes, you can make an argument that you would never come across something with a title giving you an idea of what it is about. But ETS has never said that they were going to try to make things easy. They go out of their way to try to make it as hard as possible, and this is one of those ways.

[route66]

But ETS has never said that they were going to try to make things easy. They go out of their way to try to make it as hard as possible, and this is one of those ways.

ETS's attitude is daft and as the author says, it is a crystal-clear case of the emperor's new clothes. Any non-fiction writing, and that includes the test passages, should communicate clearly (including headlines etc), so that the reader can concentrate on understanding the actual content. Imagine getting a gift, and spending all your time fighting the packaging. You lose time and energy just fighting the packaging, when the real aim should be to get you to the gift (ie the passage's main ideas/thoughts) as quickly as possible.

The point may have been made before, but we all know that any fool can make something appear difficult by writing without headlines, long sentences, little or no punctuation etc. We've all had teachers like this, who make things needlessly complicated and difficult because they are terrible at communication.  

I think ETS is just making a big fool out of everybody taking the test.,

Edited October 4, 2011 by route66

[Lox26]

Wow, why can't people seem to understand that the purpose of a test is not to make everything as easy as possible ensuring 98% passing rate? A test is not a gift, nor is it a model of anything. It is a test, and that's all. If most people understood a portion of it, it would be worthless - if it was written well enough to be easily understood, that is. Why don't we all argue that our finals are all too hard, and that we should be given easy stuff so everyone can pass?

I completely agree. The gift analogy is flawed. The test is not a gift from your buddies; it's an (admittedly imperfect) assessment of your abilities. The reading passages tend to bore me, unless there is a passage with content about which I'm actually knowledgeable or curious. Part of the challenge is to be able to focus and persevere despite the discomfort. When you are tired, distracted, uninterested--as we all are on some days--can you still process and synthesize new information efficiently (i.e., do your job, whether that's working, researching, "student-ing")? Not everything in life can or should be easy. Ultimately, the test rewards those who accept this and muddle through anyway.

Granted, there are other factors to consider, such as language barriers or differences in personal aptitude (e.g., a flair for rote memorization vs. for critically reasoning through a novel problem) or asymmetric access to information. For example, just because there is a wealth of information available about the test does not mean that everyone knows the proper channels through which to obtain the information. Maybe his/her parents don't know the system because the parents don't have the same formal education as their children aspire to or because the parents were educated abroad, with a different higher education system; maybe the person's advisers did not care, etc. I am sympathetic to these plights because I live with them and I know many people who have the same circumstances. Still, I don't think there is any excuse for a native English speaker with complete knowledge of the test and no learning disabilities to be upset that there are no headlines to spoonfeed information on a test for graduate school!

[Kitkat]

Still, I don't think there is any excuse for a native English speaker with complete knowledge of the test and no learning disabilities to be upset that there are no headlines to spoonfeed information on a test for graduate school!

I completely I agree with this statement, as well as a good portion of the rest of your statement. But this one line seems to crystallize the point. You are looking at graduate school here. If you really need a headline to help you understand a passage, maybe graduate school isn't the thing for you. I have had plenty of handouts given to me in literature and science classes with out them, and been expected to be able to read and understand them. If you can't do this by the end of college, you might have missed something. Yes, they help, but aren't necessary.

The point may have been made before, but we all know that any fool can make something appear difficult by writing without headlines, long sentences, little or no punctuation etc. We've all had teachers like this, who make things needlessly complicated and difficult because they are terrible at communication.

As for long sentences, you come across those all the time, and not all of them are necessarily hard or difficult to understand. I am sure that many can be written in several smaller sentences, but that isn't always an improvement, just a change. Little or no punctuation, sure that's a bad thing, but I haven't seen that many signs of it with the GRE yet.

[Lox26]

I want to stress that the example about the parents was more my experience with applying to college. I do believe that, at 21+, a motivated individual should be able to poke around the GRE website, browse Amazon for a prep book, and seek out resources better than a wide-eyed teenager.

I also accept that a prep course provides the added value of a knowledgeable instructor and that some people benefit more from the structure a course provides than from self-study. Another barrier to doing well on the GRE for people with this learning style may be the lack of disposable income to pay for a class. I realize that such courses range from several hundred to 1000+ dollars. I don't know how someone would work around that, other than giving it his/her best shot. This speaks to a socioeconomic bias of the exam and is perhaps another dimension of unfairness.

[route66]

Imagine if someone were to write a passage like that for the AWA section. I bet he would get an abysmally low score. So, in essence, what the GRE is telling you it broadly this: We expect you to write precise, objective and crisp sentences which when read as a passage constitute one well reasoned theory or argument. That is what you are supposed to do in "academia". However, at the same time, we also believe that nobody really follows that in academia, and we'll give you a taste of that in the next section.

I completely agree with Prasun. Here’s a line from the paper, “An Introduction to the Analytical Writing Section of the GRE revised General Test”

“What matters is……….the cogency of your ideas about the issue and the clarity and skill with which you communicate those ideas.” (my emphasis) http://www.ets.org/M...pdf/awintro.pdf (Page 8)

It's obviously not just about correct language but communication of ideas, though apparently ONLY for the test-takers and not for the GRE passages. A clear case of double standards.

Edited October 5, 2011 by route66

[Eigen]

I really don't understand the critical thinking involved in the conclusions that are being drawn here. A portion of a test that is examining reading *comprehension* will be written differently than the test expects you to write for a different portion of that same test.

One part tests your ability to understand complex passages, the other tests your ability to write clearly and concisely... Why exactly are you comparing the two? They're meant to test two completely different things.

If the reading portion was all written as clearly and concisely as possible, it wouldn't be much of a test.... Heck, even the word choices are not those that you would use if you're trying to communicate as clearly as possible.

[bigant]

One part tests your ability to understand complex passages, the other tests your ability to write clearly and concisely...

Just because it’s difficult to understand a passage, that doesn’t mean the ideas in the passage are complex. Would you agree that there is a big difference between complex ideas communicated clearly, and complex ideas communicated poorly?

Heck, even the word choices are not those that you would use if you're trying to communicate as clearly as possible.

Why should faculty not have to communicate clearly? What absolves them of that responsibility? Presumably they wrote clearly as part of their GRE AWA to be able to gain admission to a graduate program. Why do they suddenly lose that ability once they get into academia?

Edited October 7, 2011 by bigant

[Eigen]

Why are we suddenly switching to talking about faculty from the GRE?

And as to it being written poorly, you still haven't answered my previous request for a breakdown of exactly what is wrong with the sentence. All you've been able to say is that it's too long and too complicated, but I'd like to hear some distinct reasoning on why it's poorly written.

[habanero]

In the sciences and analytical writing: something that is well-written will present the idea to the reader in a completely understandable way. The idea will not be hindered by the prose.

I can't speak for the humanities.

I'll give a quick rewrite of the paragraph. It certainly isn't perfect (and it is probably less grammatically correct than the former), but I feel as though it's much easier to understand. I tried to use the same words, which really made me grit my teeth! I want to sub dwindled for shrank, ...'high marriage rates' --> ''shrank as marriage rates grew', etc. The paragraph I wrote shouldn't be difficult to understand because it gives you all of the main points. You don't have to think about them. It's almost like a bulleted list.

The increase in the numbers of married women employed outside the home in the twentieth century had less to do with the mechanization of housework and an increase in leisure time for these women than it did with their own economic necessity and with high marriage rates that shrank the available pool of single women workers, previously, in many cases, the only women employers would hire.

In the twentieth century, there was an increase in the numbers of married women employed outside the home. This change had less to do with the mechanization of housework and increasing leisure time and more to do with economic necessity. Additionally, the pool of single women that employers had shown exclusive hiring preference for shrank with high marriage rates.

Edited October 7, 2011 by habanero

[Eigen]

As I said on the last page, I think the sentence is awkward and could be cleaned up- but it isn't "wrong", simply not optimal.