That Official GMAT Question Might Cost $3000
UPDATED FOR THE NEW GMAT IN 2024
I spend a lot of time telling GMAT students that no GMAT practice problems can ever be as good as real, retired questions from official GMAT tests and publications. And if you read a bunch of GMAT blog posts written by other GMAT test-prep “experts”, you’ll find that many of them seem to disagree with me – since they're trying to sell you a nice bundle of non-official GMAT questions and exams.
Sometimes, even my own GMAT students object when I tell them to avoid using knockoff GMAT tests from, say, Manhattan GMAT or Veritas: “How terrible could those materials be? There are smart people working at those GMAT test-prep companies, right?”
Yes, of course there are smart people working at GMAT test-prep companies – though some test-prep firms clearly employ smarter GMAT question-writers than others. But even at their best, no GMAT test-prep company can possibly compete with the precision of real, official GMAT questions. Part of the reason is that copying the style of an intricate standardized test is inherently challenging, but the bigger reason is basic economics.
Developing an Official GMAT Question Costs Money… a Lot of Money
If you’re reading a GMAT blog, you’re probably a business-savvy future MBA student, and you probably have some intuition for the business models of companies like Kaplan or Manhattan GMAT. So take a guess: how much do you think those test-prep companies spend developing each individual question on their GMAT practice tests or in their GMAT books? Go ahead and think of a number.
OK, got something in mind? Great. Personally, I would imagine that GMAT test-prep companies spend something in the range of $5-30 per “knockoff” GMAT question if they actually want to make a profit, but I could be wrong.
Now let’s ask Dr. Lawrence Rudner, former GMAT Chief Psychometrician, how much the GMAT’s developers spend on each practice question:
Test items are costly to develop, often in the range of US$1,500 to US$2,500 per item.
This comes straight from a paper that Rudner presented at the GMAC Conference on Computer Adaptive Testing way back in 2007. If question development costs have kept pace with inflation, then each GMAT question would cost somewhere between $2,000 to $3,500 per question in 2022, though that figure is likely to be even higher, since shortages of trained psychometricians continue to inflate GMAT development costs.
(Incidentally, I got halfway through a PhD in education policy and psychometrics – the statistical science behind standardized testing – before I came to my senses and realized that I’m much happier as a GMAT tutor than as an academic or psychometrician. If you’re interested in a career in psychometrics, send me an email, and I’ll do my best to either help out, or scare you away – whichever you’d prefer.)
So yeah: that GMAT question you saw on the latest version of the exam? It might have cost $3000 or more to develop – perhaps 100 times more than a profitable GMAT test-prep company could reasonably spend on each question.
GMAT Perfection Takes Time
And if that sounds batsh*t crazy – or at least wildly inefficient – it probably isn’t. Actual GMAT test items go through a painstaking cycle of editing, testing (often as “experimental questions” on actual GMAT tests), re-editing, and re-testing before they actually become part of a real GMAT exam. For every question that actually makes its way onto the GMAT, countless others are discarded for any number of reasons.
For example, there might be evidence that the discarded GMAT questions were phrased badly, that they don’t seem to capture the skills the GMAT promises to measure, or that they (accidentally) discriminate against GMAT test-takers from certain demographic groups. Much of this testing requires substantial skill and training – and employees with these skills certainly aren’t inexpensive.
There’s a good amount of evidence out there that this process of repeated testing and editing takes a long time – even years. Here’s a quote from LSAC about the development process for LSAT® questions:
In all, every question goes through such a thorough vetting process that it takes at least three years — and often even longer — for a question to get from the initial writing phase to being used as a scored question on the LSAT®.
Needless to say, three years is a long time to spend finalizing one question. Since the GMAT employs a broadly similar process, it actually makes a lot of sense that each question costs thousands of dollars to create.
You probably know that I'm not particularly impressed by the GMAT’s ability to identify the most talented business leaders, but the GMAT’s creators deserve credit for producing subtle and meticulously phrased questions. How could any GMAT test-prep company possibly produce anything of comparable quality or precision, given test-prep firms’ economic limitations?
So whenever you work through practice tests from major GMAT test-prep companies – or GMAT practice worksheets from a small GMAT tutoring company based in Colorado – remember to take your results with a huge grain of salt. Sure, our “knockoff” GMAT questions can probably help you build skills, but none of us can ever give you a truly accurate GMAT practice test – simply because none of us will ever be able to afford to do so if we actually want to keep our doors open.