Manhattan Grade Inflation

[alf10087]

I can't seem to understand how Manhattan calculates the scores for their Online Practice Tests, as they are giving me results that make no sense (a lot higher than they should be on Quantitative Reasoning).

Check this relation between the number of wrong questions and the score:

Test 1: 13 wrong, score 159.

Test 2: 9 wrong, score 161.

Test 3: 9 wrong, score 164

Test 4: 9 wrong, score 164.

Test 5: 12 wrong, score 166.

Test 6: 9 wrong, score 166.

The difference cannot be attributed to the relation between the first and second set, as on some I've had the same amount of wrong in the first related to the second one, and obtained different scores. Also, the difficulty of the questions seems to be distributed in the same way among all tests.

I'm worried about score inflation on practice tests from Manhattan, as getting 9 or 12 wrong questions cannot give you a score of 164 or 166 by any chance. On Kaplan, which I did all of their 5 exams, missing 7 or 9 questions would yield a score between 160-162.

Too add to the confusion, on the first Powerprep test I missed 3 questions and got a 166, which seems to be accurate with the scores that Manhattan is giving me, but with the wrong/final score relation that Kaplan has.

Has anyone had the same problem? Is anyone from Manhattan around here who can shed a light on the matter?

Edited September 8, 2012 by alf10087

[Instigate]

That is extremely weird for sure. If it were me, I would mail them and ask for an explanation. They could well be rigging scores so that people would feel that there is some kind of progress throughout the course (which I think they want you to take with the practice tests).

[alf10087]

Another explanation could be that they measure the score based on the fact that their tests are harder than the actual GRE, but it doesn't make sense that basically on the same tests, the same number of wrong questions would yield a different result.

I did a second Powerprep test and got 166, which coincides with what Manhattan is giving me, but only having 3 wrong questions...

[Instigate]

My advice is believe whatever Powerprep tells you.

Actually I was thinking of buying the 5 online tests, but now I am a bit discouraged. Would you recommend them as a good exercise? (besides the obvious raw score conversion problem)

[alf10087]

Oh no, definitely buy them. The fact that the scores may be a little messed up doesn't obscure the fact that Manhattan has the best tests available (with Kaplan being a close second).

[Usmivka]

The scores are based on the relative scores and spread of the original testing group for those tests. The number wrong has little or no relation to the score on the real GRE, so I don't see why it should matter on emulated tests. I took this awhile ago (old rubric, out of 800), but I got a 770 or 780 with only one or two wrong answers, and later in the test at that (the "rough" scores are bracketed early in the test--do the first one wrong, you will do poorly even if you do everything else right, since it shifts you to easier questions)--clearly the test group as a whole did quite well with these questions.

Edited September 11, 2012 by Usmivka

[pemdas]

- - - - - -

Edited September 11, 2012 by pemdas

[pemdas]

manhattan, magoosh, kaplan (in the order of difficulty) - these are materials to help us if we aim around 160s (low, middle, close to 165/6) but not the perfect score. Nova's GRE math bible is the most extensive guide I have seen so far for the quant. section of test. I believe the bible will help me score higher in my next attempt.

[aberrant]

I thought a perfect exam (no incorrect answers) is the only way to get you a 170. Therefore " 9 wrong, score 166" doesn't make much sense to me.

On the other hand, do they have the "adaptive" function in the test? I remember I took Manhattan once and they were pretty accurate (not as accurate as powerprep, however).

[chaetzli]

Manhattan gave me a 163Q and I had 160Q on the actual test. However, other prep test also overestimated my score (Kaplan 164, Princeton 164). I'm very sad that I am just not able to do as well on the actual test.

[pemdas]

Manhattan gave me a 163Q and I had 160Q on the actual test. However, other prep test also overestimated my score (Kaplan 164, Princeton 164). I'm very sad that I am just not able to do as well on the actual test.

don't be sad, as I said above I have tried several of prep books and all in vein - they don't show you right strategy for plugging in, substitution and short-cuts. These are however very well addressed in GRE math bible by nova.

[chaetzli]

don't be sad, as I said above I have tried several of prep books and all in vein - they don't show you right strategy for plugging in, substitution and short-cuts. These are however very well addressed in GRE math bible by nova.

I just wish I had an idea where I stand with V161 (86%) and Q160 (81%). I am not going to start a "what are my chances..." thread as there is much more that counts (besides the GRE scores). Nonetheless, I have no idea where one could apply with these scores (for Pol. Sci). Is it good enough for top programs? Just a few more points and I would not have to worry about that as I could apply everywhere. It is frustrating that I was not able to improve my scores.

[pemdas]

I just wish I had an idea where I stand with V161 (86%) and Q160 (81%). I am not going to start a "what are my chances..." thread as there is much more that counts (besides the GRE scores). Nonetheless, I have no idea where one could apply with these scores (for Pol. Sci). Is it good enough for top programs? Just a few more points and I would not have to worry about that as I could apply everywhere. It is frustrating that I was not able to improve my scores.

if you allow me to give you friendly advice, I would offer you to retake the test and apply to all programs you intended initially. Think about this like, it's some nuisance and inconvenience to take the same test multiple times, but considering the marginal cost of retake - that is with each retake and score increase your standing in the list of selected candidates before acads and LORs are reviewed is higher. With every 10-15 percentile increase in scores we get farther in the GRE screening process by AdComs. So prep with the book I mentioned and retake the test. You can practice with hard section problems to save time and retake the test in October.

[chaetzli]

if you allow me to give you friendly advice, I would offer you to retake the test and apply to all programs you intended initially. Think about this like, it's some nuisance and inconvenience to take the same test multiple times, but considering the marginal cost of retake - that is with each retake and score increase your standing in the list of selected candidates before acads and LORs are reviewed is higher. With every 10-15 percentile increase in scores we get farther in the GRE screening process by AdComs. So prep with the book I mentioned and retake the test. You can practice with hard section problems to save time and retake the test in October.

I really appreciate your advice but it don't think that there is any room for improvement. This was my second retake (yes... I almost feel embarrassed writing this). My first try (a year ago) was an abysmal 150V and 155Q. My second try (two months later) 151V and 159Q and the last one mentioned above. I spent months practicing (I bought 10 books + Magoosh). I guess these scores reflect who I am (not a genius but a hard working clever student). As said, I always did much better on the prep tests and I have no idea why I cannot get the same scores on the actual test. I guess I have to live with it and hope that other aspects of my application will be more convincing. The only thing that really bothers me is that I do not want to spend a fortune on application fees if there is no chance that the admission committees will even look at my application. I wish that the universities would clearly indicate their cutoffs.

[pemdas]

swiss, if you never had to deal with math before GRE or the only math classes you took were high school of college math, then please understand that I, being a math minor, and having spent some time with magoosh, kaplan and manhattan still landed near 160 on math. These books are pure marketing ideas - they don't get you there. Only you working with various questions will be able to improve and score higher on quant. sectino. Again try barron's gre and then see what happens. However, be as much diligent as possible, since only you can get there and not books will take to your desired score.

[Usmivka]

I found the Barrons book to be particularly useful (it came with some adaptive tests that were harder but of very similar format and pacing to the real test). I didn't shop around though, I just used the text available at the used bookstore.

In terms of material review, the free Quant section topic review on the ETS website was great.

[chaetzli]

Thank you pemdas and usmivka. I very much appreciate your advice. I just think I can't do it. I am probably just not smart enough. And I will stop posting irrelevant things in a thread that's about manhattan grade inflation .

[pemdas]

hey swiss, in intended to speak for nova's gre math bible and the barron's slipped away in my post

[jmbrown88]

It's strange, even though I encountered a similar grade inflation, my practice test scores were nearly dead on with how I scored on the actual test.

[Tdearr]

Hi Everyone,

I work for Manhattan Prep and I was hoping to weigh in on this thread. We do work to try and make our tests accurate predictors of student performance on the real GRE. We do this using an adaptive scoring system and a weighted algorithm similar to that of the real GRE. This algorithm can cause two different exams of ours, each with 9 missed questions, to be scored differently.

However, as a few students speculated above, our tests can skew above the average difficulty of the real exam (meaning students will miss more questions on our exam than on the real exam). We like to include higher difficulty questions because our materials and courses are designed to help students achieve up to 99th percentile scores. The adaptive nature of the test allows us to create very difficult tests for high performing students without making our exams inaccessible to students scoring at lower levels. That said, we also value our exams as a diagnostic tool, so we don't want students to score lower on our exam because it is slightly harder. As such, our algorithm accounts for the extra difficulty (when high difficulty questions are missed). So while on the real GRE missing 9 questions may push you a bit below 166, a student who scores a 166 on our exam is truly capable of scoring a 166 on the real GRE.

As some students mentioned above, they found the score on our exams similar to their score on real exams (although the number of questions missed was different). We periodically calibrate our exams by matching students performance on our exams to their performance on real exams. As such, we are confident that our exams provide accurate indications of a students true ability level on the actual GRE.

I hope this information is helpful. If you have any other questions or concerns about our tests, please let us know!

Best Regards,

Taylor Dearr

[chaetzli]

@Tdearr: do you have an explanation why people like me cannot get the same scores on the actual test? I had 163 on your prep test and 160 on the actual one. I just cannot find a sound explanation for this discrepancy. Something must be different but I cannot figure out what it is.

[Tdearr]

@Tdearr: do you have an explanation why people like me cannot get the same scores on the actual test? I had 163 on your prep test and 160 on the actual one. I just cannot find a sound explanation for this discrepancy. Something must be different but I cannot figure out what it is.

Hi SwissChocolate,

Unfortunately, there is some score variance with all practice tests. Every test is slightly different and students tend to score slightly differently from test to test. A three point difference is within a reasonably small margin of error. I noticed that you scored higher on other practice tests as well; some students do see a drop in scores from test day jitters and don't do quite as well on their actual test as they do on practice tests. I don't know if this was a problem for you, or if it just happened to be that the test you got on test day didn't go as well as your practice test.

That said, 160 is a good score placing you above the 80th percentile of quant scores and will be seen a acceptable for many political science programs.

I hope this information is helpful. Please let me know if you have further questions.

Best Regards,

Taylor Dearr

Edited September 17, 2012 by Tdearr

[alf10087]

Well, Tdearr's explanation seems legit, as I took the real GRE last friday and my unofficial scores are 164Q, 160V.

Manhattan ended up being even more accurate than Powerprep.

[chaetzli]

@Taylor Dearr: thank you very much for your reply. I'll probably never find out if it is test day jitters or sth else but it doesn't matter anymore (I am definitely not going to do it again) .

@alf10087: congrats! this is a great score!!

[Dimbulb]

Hate to post something inane but here is my dumb question. I am applying to a MACC program that requires a minimum 1000 GRE score. I have taken 4 Manhattan practice tests and after receiving a 1200 on the 3rd, pulled the trigger and registered for the test next week.

After shoring up some quant deficiencies and practicing RC's, I took a fourth and scored a 299. The disparity was on my verbal. This is the exact score I received my first practice test months ago and I am now worried that I am going to bomb.

Has anyone experienced a similar disparity? I am unable to take the PowerPrep as I have Mac's and the public and school libraries do not have C drives.

I realize that 1000 isn't tough but hey, I am just clearing 1000 on Manhattan's after 2 months of 4 hour daily study.