ETS vs. Magoosh divergence?

[InvisibleHand]

Some weeks ago, I took the first of the two PowerPrep exams after going over the GRE's quantitative content in their quantitative prep book and doing the practice questions. I got a 170V and a 164Q. I had done no real verbal prep besides doing a few vocab apps on the bus en route to work over the past month or so.  

These were good scores, but I wanted to improve my math performance (and touch up on the Verbal section: I missed one critical reading question but still got 170 with the curve), so I subscribed to Magoosh. They seemed to have an extensive bank of questions they wrote. With only one exception, the questions I missed on the quantitative section of the first PowerPrep exam were careless mistakes. So my goal with Magoosh was really to have an extensive bank of test-prep questions to drill down on the discipline and test-taking technique necessary to avoid careless errors. And their large number of questions seemed like a good fit for that particular task.

While I don't score as consistently well on the "Hard" or "Very Hard"  Magoosh questions as I would have tended to anticipate, I'm somewhat skeptical that many of the tactics that separate those who answer the Magoosh questions successfully from those who don't are necessarily tactics or strategies that will prove valuable on the GRE exam itself (though some certainly seem like they could be). It is, of course, possible that my PowerPrep I score was simply an aberration and that Magoosh is giving me a more accurate forecast of future test scores. 

Has anyone else been in a similar situation of scoring (relatively) highly on the first PowerPrep exam? The situation, here, being that I scored fairly well on the first PowerPrep exam and now want to make the minor improvements that seem like they could get my score up to the ceiling, but am now feeling as if third-party content is introducing to at least some extent a different set of "signals" into their questions than does the GRE.

I'm leaning towards using Magoosh to brush-up on certain content areas of the quantitative section, but turning to extremely close scrutiny of ETS-produced material for general test prep and for critical reading questions. This seems like a marked departure from what the consensus on GRE prep boards like these would have you believe, but it does not necessarily seem like the consensus wisdom is well-suited for help in raising a 164 Q / 170V to a straight 340. While I'm still somewhat hesitant to base a test-prep strategy on the PowerPrep II first exam alone, I've done the "practice problem sets" in the GRE quant book and have been doing fairly well, so I don't think there is a TON of noise in that score.

Anyhow, advice appreciated. Thanks for reading!

Edited September 19, 2016 by InvisibleHand

[coffeecupanalogy]

I believe that Magoosh questions are more difficult in order to prepare you for the adaptive mode of the real exam. As far as I know, if you do good on the first math section, the second will be harder, that's why Magoosh offers a wide range of practice problems, ranging from easy to very hard levels, in order to prepare us for that.  I'm not sure if you are in the Magoosh facebook group:  Magoosh Superfans, a lot of the posts there are related to the accuracy of the real scores and the scores from Magoosh practice tests, which are almost the same. 

[InvisibleHand]

Yes, but even Magoosh's blog says that the GRE's materials, however limited they are in quantity, are better preparation for the GRE than anything else...and the PowerPrep II exam is adaptive.

Magoosh Superfans will not be a great source of evaluation for Magoosh, since it would only include Magoosh users who have chosen to "super fans". 

 

Edited September 21, 2016 by InvisibleHand

[Yanaka]

I found the actual GRE much more difficult than all the Magoosh test prep I had done. However, I do not understand why you would want to boost your scores. They're already perfect...

[InvisibleHand]

Yanaka: how did you think the actual GRE stacked up to the PowerPrep exams from the ETS? 

It seems the ETS concept is more "conceptual" in nature and tries to dig at mathematical intuitions, whereas Magoosh is more computationally-intensive. So I don't think necessarily think the one is harder than the other, just that they are different.

[TK2]

I'm starting to think* that a lot of this is statistical noise. It's hard to get a complete picture from the semi-anecdotal data we have anyway (unless someone wants to go through the predicted/real scores thread and fill out an excel and try to disentangle the variables .) Substantial differences in scores are a function of a 2-4 question wrong/right difference. Exogenous effects like level of concentration and distraction, stress-induced (or just random) calculation errors, etc, make it look to me like it's very unlikely to find a real patterns between the different practice sets and the GRE itself. Some people may do better or worse on Magoosh/Manhattan/Princeton/The Real Thing, but it might be because the layout of one or the other is slightly more comfortable for them, or a particular set of questions is heavy a concept they're very good with, or they had a really good nights sleep or they do particularly badly under stress or, hell, particularly well under stress, and a lot less of the difference is down to a universally-applicable difference between the tests.  

*I'm also starting to procrastinate studying for the GRE by analyzing studying for the GRE. I would make such a fine academic, if I could get through this thing. 

[InvisibleHand]

On 9/26/2016 at 1:26 AM, TK2 said:

I'm starting to think* that a lot of this is statistical noise. It's hard to get a complete picture from the semi-anecdotal data we have anyway (unless someone wants to go through the predicted/real scores thread and fill out an excel and try to disentangle the variables .) Substantial differences in scores are a function of a 2-4 question wrong/right difference. Exogenous effects like level of concentration and distraction, stress-induced (or just random) calculation errors, etc, make it look to me like it's very unlikely to find a real patterns between the different practice sets and the GRE itself. Some people may do better or worse on Magoosh/Manhattan/Princeton/The Real Thing, but it might be because the layout of one or the other is slightly more comfortable for them, or a particular set of questions is heavy a concept they're very good with, or they had a really good nights sleep or they do particularly badly under stress or, hell, particularly well under stress, and a lot less of the difference is down to a universally-applicable difference between the tests.  

*I'm also starting to procrastinate studying for the GRE by analyzing studying for the GRE. I would make such a fine academic, if I could get through this thing. 

Ah, but one cannot permit one's level of concentration or stress or level of sleep to be "exogenous" for something as (potentially) consequential as the GRE  Admissions committees do not run OLS regressions to adjust your score for "exogenous" factors that have influenced it on that particular date and time...

To the master, even the slightest detail must become what an economist would term "endogenous"...no one carers if the sniper *could* have hit the target, if only he figured out how to calm his nerves. And so the marksman cannot dismiss the banalities of execution as any less elemental to the successful shot than the grandest of theories. 

Sampling variance, I would wager, is endogenous at the individual-level. The only way to (with 95% confidence) get your target score is to get your target score to within two standardized deviations of your mean score. This requires the right mean and a sufficiently small level of variation around that mean. I think it's easiest to converge on the desired mean first, and then narrow the variance second.

Edited September 28, 2016 by InvisibleHand