Fundamental flaw in GRE reading comprehension test

[bigant]

As I said on the last page, I think the sentence is awkward and could be cleaned up- but it isn't "wrong", simply not optimal.

Nobody is arguing that the sentence is wrong.

The idea will not be hindered by the prose.

This is precisely the point. habanero's re-write, achieved with minimal "interference" to the text, is far better than the original.

I can't see why allowances should be made for writing that is opaque or awkwardly written (to put it mildly) at the graduate level. I would think that at especially at graduate level, you should be able to write clearly and concisely while still expressing your ideas. The GRE RC writers show little evidence of that.

Edited October 10, 2011 by bigant

[Eigen]

You guys do realize that the GRE passages aren't written by some anonymous "GRE writers", right?

They're all sections pulled from journal articles. Hence, they're supposed to be a wide smattering of what you might encounter in your graduate career... ETS just doesn't select out those they think are written "poorly" from those that are written "well". They're all peer-reviewed examples of academic writing, generally from reputable journals.

Edited October 10, 2011 by Eigen

[habanero]

Eigen, I said something similar a few pages back: "There are many poorly-written papers in academia. It does seem important to be able to understand confusing passages as quickly as possible." I still feel that the sentence isn't particularly easy to understand. It took me a few reads. I got a top percentile on verbal, so I'm not complaining in a bitter grapes manner*. Peer-reviewed doesn't always mean that the writing is clear and cohesive. I do agree that, as researchers, we will need to wade through loads of confusing papers. Not everybody has the same style and good research publications aren't limited solely to those with good writing abilities!

*not to suggest that the only valid critique comes from people who did well!!!

Edited October 10, 2011 by habanero

[Eigen]

Eigen, I said something similar a few pages back: "There are many poorly-written papers in academia. It does seem important to be able to understand confusing passages as quickly as possible." I still feel that the sentence isn't particularly easy to understand. It took me a few reads. I got a top percentile on verbal, so I'm not complaining in a bitter grapes manner*. Peer-reviewed doesn't always mean that the writing is clear and cohesive. I do agree that, as researchers, we will need to wade through loads of confusing papers. Not everybody has the same style and good research publications aren't limited solely to those with good writing abilities!

You did indeed say that. I was more responding to Bigant's comment that the GRE writers should be at the "graduate" level. I just thought it worthwhile to point out that these aren't articles that are mass produced just for the GRE, but rather a large pool of articles cut from journals.There seems to be a large sentiment that it's what "ETS" is doing to "US", or put another way, that some things are acceptable for ETS in their writing section but not for those taking the test in the AW section, and I thought the reminder that ETS isn't writing the selections they're using might be helpful to offer some perspective.

This is also an argument that goes back for 6 months, it's hard to keep the different parts of it straight in my head!

Edited October 10, 2011 by Eigen

[bigant]

I'm aware that the GRE passages are taken from journals, I was using "GRE writers" as a shorthand. I've spoken to several researchers who've all told me that the quality of writing in journals is terrible. However, since these articles are always reviewed before publication, these articles go through several rounds of iteration with the editors, so that they understand what is being said. This could also mean asking the author for clarifications via email/phone. Even once the article is published, readers always have the option of contacting the author with questions or clarifications.

Obviously, this process of asking for clarifications is not available to us taking the GRE. That's why these journal articles need to be professionally edited for the GRE.

Edited October 13, 2011 by bigant

[Kitkat]

Obviously, this process of asking for clarifications is not available to us taking the GRE. That's why these journal articles need to be professionally edited for the GRE.

At that point you should just say that professional journals should be professionally edited. If professors say that the journals are terrible then that's where it should be fixed. While I am not a huge fan of ETS, by arguement, they don't seem to be the cause of the problem. They are showing us what is out there and saying "Here read this, try to understand it. This is what you are going to have to read in grad school". If they are giving us terrible writing that is accurate to what we have to read in grad school then what is the problem with the passages that they give us? That is, after all, what they are trying to test us on .... No it's not fun for those of us who have to take the test, but I can't say that it is wrong of them.

[habanero]

For me, the issue lies with the wording of the multiple choice questions. The passages are generally fine--some are confusing, but they're still fine. I had to work on understanding how to choose a multiple choice answer when none of them clicked with me.

[Eigen]

For me, the issue lies with the wording of the multiple choice questions. The passages are generally fine--some are confusing, but they're still fine. I had to work on understanding how to choose a multiple choice answer when none of them clicked with me.

This. I think it's the inherent issue with a multiple choice test. Ideally, this type of exam would be short answer asking for summaries of the articles... But that would be so much more difficult to subjectively grade, leaving us with a poor but objectively fair substitute (it punishes everyone equally).

[route66]

If they are giving us terrible writing that is accurate to what we have to read in grad school then what is the problem with the passages that they give us? That is, after all, what they are trying to test us on

Even terribly written journal articles have headlines. Why can't they at least give us headlines?

[Kitkat]

Even terribly written journal articles have headlines. Why can't they at least give us headlines?

Because I think that would defeat the point of having hard questions in the first place. Headlines are there to tell you basically what the passage/ article is about. In the real world, we use that to distinguish what we want/need to read from what we don't want/need to read. On the test, we have to sit there and read the passage no matter what. They want to see if we can figure it out without the easy method of looking at the headline. What if they want to give us the question of what type of headline would we give a passage? I'm sure that this is something they want to keep open, as I remember them asking on the old version. I feel that if they gave us headline, that they would then feel the need to give us harder questions.

[bigant]

They want to see if we can figure it out without the easy method of looking at the headline.

It's a bit like the driving license authorities saying, let's see you drive a car without a steering wheel, because the steering wheel will make the test too easy

[Kitkat]

It's a bit like the driving license authorities saying, let's see you drive a car without a steering wheel, because the steering wheel will make the test too easy

I think that is a bad analogy for this. You need a steering wheel to drive. You don't need to have a headline to understand a passage. The headline helps you and makes reading the passage easier, but not having one doesn't make it impossible. Lacking a steering wheel would make driving completely impossible, until we engineer a car that doesn't need one.

I am wondering why you think that you need a crutch of a headline to help you read though a passage of a couple of paragraphs, at most? They are not going to be testing you on anything that a headline can give you. At most, them giving you a headline, they will give you questions that make the headline pointless, so I am wondering what it is you think the headline will help you with? After all, as a form of a reading test, they want to see what you get out of the whole reading.

[bigant]

I am wondering why you think that you need a crutch of a headline to help you read though a passage of a couple of paragraphs, at most?

A headline is not a crutch; it is an important part of the overall text.

Besides, all writing in the real world has headlines - text books, journal articles, opinion pieces, even text-based adverts.

[Kitkat]

A headline is not a crutch; it is an important part of the overall text.

Besides, all writing in the real world has headlines - text books, journal articles, opinion pieces, even text-based adverts.

I don't think that headlines that important to the text. In my mind they give a very vague idea of what it is that it is trying to tell me. Try to explain to me why you think that they are important, how exactly you think that the headline will help you. I say all of this in context of what the test is trying to test us on, not what is applicable to the real world.

As to their function in the real world. Yes, all sorts of things in the real world have headlines, titles and that sort of thing. But the function of headlines and titles in this case is to help us figure out whether it is something that we want to read in the first place. It is to help us figure out what is relevant to us. Back to the test, we have to read the passage that is in front of us. It is not like they are giving us two passages, and ask us a question, and we have to figure out which passage to look at. Which goes back to my point. What is it about the headline that will help you on the GRE? You would still have to go into the passage to find the answer. I can't think of a situation where they have asked a question, where I thought, "Oh I wish I had a headline".

[Lox26]

I'm with Kitkat on this. Headlines are redundant for those proficient at extracting salient information from a novel text. Yes, titles would make the reading comprehension questions easier (I sometimes had to re-read paragraphs about obscure theories in order to understand the point). But, an easy exam is pointless when you are trying to quantify a type of intelligence. You need to show a difference between those who can and those who can't do X or Y well.

[bigant]

There are two issues here. The first is written communication. The purpose of a headline is not necessarily to help you decide what to read (eg in newspapers), but to tell you what the basic message of the text is. And it is the writer's responsibility to do give you that message. If the aim for the writer is to communicate to an audience, then it follows that headlines and so on are important and necessary. Many times the question on the GRE is along the lines of, "What is the author trying to say". It is up to the author to tell us that through the headline.

The second issue is the test. If you want to make reading comprehension as difficult as possible, there are many ways to do so, starting with removing headlines. You can make the text hard to read by with-holding punctuation, not explaining terms, jumping from one topic to another without any breaks and so on. If the idea is to make reading as tough as possible, I think we can all figure out ways to make the test so difficult that only 10 people can understand.

What the ETS folks need to decide is whether they want to test for testing sake, or to test whether someone can comprehend a piece of text through which a writer is trying to communicate.

[Eigen]

Newspapers are not technical or academic communications- they're meant for the layperson with very little background. Academic writing of the sort you encounter in graduate school is not. You keep fixating on headlines, but I have yet to run across a single peer-reviewed paper that uses headlines- section breaks for "Introduction" "Results" and "Discussion" notwithstanding- but I don't think that's what you were referring to.

As to your second point- there's a difference between making something difficult within the bounds of acceptable and correct writing, and jumping the shark so-to-speak. Even the piece (and I'll note that you're only harping on a single example) that you decry in this thread is difficult while using correct grammar, punctuation, and maintaining a progressive logical flow. It's not an easy piece to comprehend, but it's within the bounds of correctness.

But lets be honest, if you need a headline to tell you what a section is talking about, your reading comprehension skills are pretty low. The point of reading comprehension is to be able to figure out what's being said from the actual content!

[Kitkat]

A headline is not a crutch; it is an important part of the overall text.

Besides, all writing in the real world has headlines - text books, journal articles, opinion pieces, even text-based adverts.

I am not going to argue that the real world had headlines, It does. Then there are the section headlines, but they are generally just things like Intro, conclusion, that sort of thing. So in that sense, headlines are useless, and that is if you are lucky. If you look at major science journals like nature and science, there are not even those sections really. And as I have said before, headlines are really there to tell you a very simple description of what is in the article. In a case like this, having a headline will not help me if I am trying to write a grant proposal based on an article, the headline won't be useful. Being able to read, understand and extract the information is. The headline is a tool for me to find the articles worth reading. The headline does not give me the details that I would need.

Bigant, how much important information do you think you can really get out of the headline, when the question asks for details from the passage.

Newspapers are not technical or academic communications- they're meant for the layperson with very little background. Academic writing of the sort you encounter in graduate school is not. You keep fixating on headlines, but I have yet to run across a single peer-reviewed paper that uses headlines- section breaks for "Introduction" "Results" and "Discussion" notwithstanding- but I don't think that's what you were referring to.

But lets be honest, if you need a headline to tell you what a section is talking about, your reading comprehension skills are pretty low. The point of reading comprehension is to be able to figure out what's being said from the actual content!

I am totally with Eigen on this. This is the point that I have been trying to make this whole time.

I feel that headlines are a very simplified view of what the article/journal/passage. They don't give you detail, which the test is looking for. Other issues with the test notwithstanding, I still don't see how headlines will help you find the details in a passage.

[habanero]

This conversation has gotten a little ridiculous. ETS needs some way to test us. If every article read like Newsweek, we would all get 100% of the questions correct. The test is hard because we're all college graduates/almost graduates with excellent reading comprehension skills. ETS can't just ask us easy questions! They are meant to be hard!!! A title would change nothing because the new questions would reflect the change. I COMPLETELY agree that most of the articles aren't amazing and that the questions are phrased in very obnoxious ways, but you must understand that the exam is designed to give a relatively normal distribution of scores. If we all did very well, the exam's curve would have failed.

[bigant]

I have yet to run across a single peer-reviewed paper that uses headlines - section breaks for "Introduction" "Results" and "Discussion" notwithstanding- but I don't think that's what you were referring to.

But lets be honest, if you need a headline to tell you what a section is talking about, your reading comprehension skills are pretty low. The point of reading comprehension is to be able to figure out what's being said from the actual content!

We're certainly not reading the same papers. I've never come across a paper without a headline or title (I use the words interchangeably), eg "Nocturnal habits of lions in captivity."

Just because someone requires a headline, that doesn't mean their reading skills are poor. Imagine presenting a headline-less and title-less paper to the President of the US without any introduction, and when he asks, "What's this about", saying to him, "Are you reading skills so poor that you want a headline?"

If you look at major science journals like nature and science, there are not even those sections really.

Bigant, how much important information do you think you can really get out of the headline, when the question asks for details from the passage.

I went across to the Oxford journals website, and clicked on a random paper, http://her.oxfordjournals.org/content/25/1/83.full

‘Smoking’: use of cigarettes, cigars and blunts among Southeast Asian American youth and young adults"

Obviously, this is a headline/title. Of course, the sub-headlines such as Abstract, Introduction and so on follow a format for papers. These are all sub-headlines that help, rather than having one long piece of text. Also there is nothing preventing researchers from adding other useful sub-headlines if they think they're useful. Moreover, there's use of bold and italics text of different font sizes.

Agreed, you cannot get all the information from a headline, and that is not the job of a headline. But a headline matters.

If every article read like Newsweek, we would all get 100% of the questions correct.

Given its content, I'm guessing most readers of Newsweek are graduates. Besides, I view the reading comprehension test more like driving license tests. If everyone is proficient to a certain standard, that's all that counts. The problem with the percentile system is that it makes the test artificially difficult by sub-standard writing so that relatively few people "pass" the test.

Edited November 4, 2011 by bigant

[Eigen]

Yes, papers have titles. It would have saved a good bit of debate if you'd made clear that you were using "headline" and "title" (two non-synonymous words) interchangeably.

That said, writing samples used for the GRE are small segments of larger papers- as such, they are rarely titled, since they are excerpts. Titles are typically used for standalone works.

The GRE is not a drivers license test. Its point is to striate the pool taking the test such that there can be a rough means of comparison. Your metaphor between college admissions and the DMV makes little sense- the DMV has an infinite number of licenses to give out, assuming proficiency to a minimum standard. Graduate schools, on the other hand, have a limited number of spots to give out to the best applicants. The driving test is not a competitive process, such that percentiles are interesting to anyone- the GRE is.

[bigant]

The point I am making is, papers have titles, and titles help in comprehension (otherwise, why would titles be used?). So it is not fair to deprive test candidates of titles, even for excerpts.

[bigant]

With due respect to KitKat, the purpose of a headline is not ONLY to help you choose which article to read. Even if you are reading a single article, a headline is still useful and necessary.

Edited December 5, 2011 by bigant

[Helpplease123]

I think the point made by Eigen sums it up quite nicely, they need to differentiate between candidates somehow and clearly having no headlines/titles is one of the ways to do that.

The GRE is not a drivers license test. Its point is to striate the pool taking the test such that there can be a rough means of comparison. Your metaphor between college admissions and the DMV makes little sense- the DMV has an infinite number of licenses to give out, assuming proficiency to a minimum standard. Graduate schools, on the other hand, have a limited number of spots to give out to the best applicants. The driving test is not a competitive process, such that percentiles are interesting to anyone- the GRE is.

[bigant]

I think the point made by Eigen sums it up quite nicely, they need to differentiate between candidates somehow and clearly having no headlines/titles is one of the ways to do that.

Ok, so what I'm hearing is that the passages should NOT clearly communicate, but deliberately obfuscate (eg by removal of headlines, usage of long-winded sentences etc)?

Edited December 19, 2011 by bigant