Metric of "Overall GRE performance"

[lizardclan]

Hi!  I'm sharing the following because I want to find out if it makes sense to other people.

Motivation:  I want to compute a metric for overall GRE performance.

Assumption:  I assume that there is no correlation between Verbal, Quantitative, and Analytical Writing scores.  If there were, that would be a serious flaw of the GRE General Test.

Definitions:

Let V = percentage below on the Verbal section expressed as a decimal, that is:  0.0 for 0%, up to 1.0 for 100%
Let Q = percentage below on the Quantitative section expressed as a decimal, that is:   0.0 for 0%, up to 1.0 for 100%
Let AW = percentage below on the Analytical section expressed as a decimal, that is:  0.0 for 0%, up to 1.0 for 100%

Proposition:  Your overall performance on the GRE General Test is given by:

(1.0 - V) * (1.0 - Q) * (1.0 - AW)

For example, I scored V:155 (69%), Q:155 (58%), AW:4.0 (59%) on my most recent attempt on the GRE.

So, my overall performance on the GRE is:

(1.0 - 0.69) * (1.0 - 0.58) * (1.0 - 0.59) = 0.053382, or 5.3382%

In other words, overall, I finished in the top 5.3382%.

Criticism?
Thanks!
/James/

[Warelin]

I don't think subtracting your percentile score makes sense.

I think a clearer metric would be:
(Percentile score) * (Percentile Score) * (Percentile Score)

GRE scores already assign you a percentile score on each section individually. As such, the test already seperates your scores when assigning them. I think multiplying them would give you a percentile score on the overall exam. 

Using your example:
(.69)*(.58)*(.59)=0.236118 or top 23.61%.

[lizardclan]

.69, .58, and .59 are *percentage below*

(1 - .69), (1 - .58), and (1 - .59) are "top percentage" or "percentage above"

My metric computes overall percentage above, not overall percentage below.

Assume there were only 2 sections on the GRE General Test.  Let's say you got 90% percentage below on both.

Then by your reasoning, the overall score would be .9 * .9 = .81, or 81% overall which can't be true.

Using my formula, you get (1 - .9) * (1 - .9) = .01, or 1%.  You finished in the top 1%.

/James/

Edited October 28, 2018 by lizardclan

[Warelin]

Using your original formula, someone scoring in the 50th percentile on all 3 sections:

(1 - .5) * (1 - .5) * (1-.5) = 0.125 or top 12.5 percentile. 

or in 30th percentile
(1 - .3) * (1 - .3) * (1-.3)=0.343 or top 34.3 percentile.

That doesn't make sense either. I think the biggest thing you're forgetting to account for is that the percentiles in each section change depending on how well people in the previous 3 year cycle have done. As such, it already accounts for how well you've compared to other top score getters in each section.

I'm not sure if there is a formula that can determine your 'overall' percentile. I'm not sure the GRE was designed to be that way. I think averaging your percentile scores might be a better indication of overall "rank".
(69+58+59)/3 = 62 percentile = top 38 percent of test takers.

[lizardclan]

Hey I'm just using probability theory.

In probability theory you *multiply* probabilities together, not add them and divide by 3 like you did.

P(getting a 155 or higher on the Verbal section) = 0.31
P(getting a 155 or higher on the Quantitative section) = 0.42
P(getting a 4.0 or higher on the Analytical Writing section) = 0.41

Multiply these 3 probabilities together (because that's what you do in probability theory) to get a top 5.3382% result.

I'm pretty certain in my results.

/James/

[Warelin]

"Probability theory, a branch of mathematics concerned with the analysis of random phenomena. The outcome of a random event cannot be determined before it occurs, but it may be any one of several possible outcomes. The actual outcome is considered to be determined by chance."

Is it fair to use probability theory when the distributions aren't random though? It's a great formula to use when trying to determine things like flipping heads 5x in a row. (1-.5) * (1-.5)* (1-.5)* (1-.5)* (1-.5)=0.03125 or .3.125 percent chance. Each roll is seperate in this case and each has a 50 percent chance of happening. There is no roll in the GRE case though. Numbers are assigned a percentile which already showcases how well you did. This is why I think averages are a better indication of how you did overall if you wanted that number. It weighs things equally and already takes into account how well you did compared to other people in each section.

[edward130603]

4 hours ago, lizardclan said:

Hey I'm just using probability theory.

In probability theory you *multiply* probabilities together, not add them and divide by 3 like you did.

P(getting a 155 or higher on the Verbal section) = 0.31
P(getting a 155 or higher on the Quantitative section) = 0.42
P(getting a 4.0 or higher on the Analytical Writing section) = 0.41

Multiply these 3 probabilities together (because that's what you do in probability theory) to get a top 5.3382% result.

I'm pretty certain in my results.

/James/

This is the probability someone scores better than you on all three sections. I think you can use that as a performance metric, but it is not a percentile (i.e. the top 5.3382% interpretation is not valid).

We can take another overall metric of performance to be the sum of the percentiles. Given your assumption, the three percentiles are i.i.d. Uniform (0,1) random variables, so their sum has the Irwin-Hall distribution with n=3. Using your example, the sum of the percentiles is 186/100, which is the 75.4 percentile for Irwin-Hall distribution with n=3.

In reality, the three scores are quite correlated due to other factors such as time spent studying, test-taking ability, etc.

Edited October 29, 2018 by edward130603

[parukia911]

6 hours ago, lizardclan said:

Hey I'm just using probability theory.

In probability theory you *multiply* probabilities together, not add them and divide by 3 like you did.

P(getting a 155 or higher on the Verbal section) = 0.31
P(getting a 155 or higher on the Quantitative section) = 0.42
P(getting a 4.0 or higher on the Analytical Writing section) = 0.41

Multiply these 3 probabilities together (because that's what you do in probability theory) to get a top 5.3382% result.

I'm pretty certain in my results.

/James/

There are a few reasons why thinking like this is relatively impractical/your theory is slightly incorrect.

First of all, it's not practical to think about where your overall rank is without a proper ordering on the sections (i.e., a way to weight each of the sections). Without this, it's impossible to say whether a 80 V, 90 Q is better or a 90 V, 80 Q. If you weigh all of the sections equally, then you get the Irwin-Hall distribution, that edward130603 mentioned (intuition behind this is that if your true expectation is 50th percentile, then getting 60-70 multiple times is more telling of your true rank than getting it once). 

Secondly, you've found an upper bound to what percentile your score is. The probability that you calculated is the probability that someone beats you on all 3 categories (which would be ranked higher than you regardless of how the sections are weighted). You can go the other way and find the lower bound on your rank (0.69*0.58*0.59 = 24th percentile). Thus, what we show is that your true overall percentile is somewhere between 24th and 95th - seemingly not very useful. 

Feels like the safest way to sort of "unbiased"-ly determine what your rank is is an average of your percentiles. Once again, thinking about this kind of stuff isn't really that important/not what the admissions committee would likely be thinking about, but taking an average is how I would do it.

[randata]

This may be helpful to you: GRE General Test Interpretive Data Guide from ETS

The scores are correlated, and the correlations are published:

"The correlation between Verbal Reasoning and Quantitative Reasoning scores is.35, the

correlation between Verbal Reasoning and Analytical Writing scores is.68, and the correlation between

Quantitative Reasoning and Analytical Writing scores is.16."

 

[cyberwulf]

GRE scores aren’t independent. If you assume scores are normally distributed, and you know the means and correlations (posted above), then you could work out (or simulate) the probability of someone scoring higher on all three sections than you. In your case, I would bet it gives a number higher than 0.05.