Questions about Manhattan's CAT quant section

[B_Naber]

I just finished another CAT exam and I'm starting to question the accuracy of the quant score in these tests.
Overall 32 out of the 37 questions were 700-800 level questions and the ironic thing is that I only scored a 45 in the quant section. I've already taken the real GMAT exam once and scored a 48 in the quant section, I've also taken the GMAT prep mock tests and from my experience the type of questions that are asked in these tests are of a completely different caliber than the ones you find in the quant section of the Manhattan CATs. If you're scoring in the range of >48 you might get 10-15 700-800 level questions, but definitely not 32!!!!
So my question is why did you construct your mock tests this way and dont just simply try to make it as close to the actual exam as possible?

[StaceyKoprince]

We do make our exams as close to the real thing as possible - the data that you're seeing can be misinterpreted / misleading in some small ways.

First, we put the questions into broad "buckets" to give you a sense of how difficult a question is - 700-800 is harder than 600-700 - but that isn't how an individual question is actually rated. In fact, anything on the 200-800 scale doesn't have to do with quant (alone) at all, as that scoring range is used only once Q and V are combined. But we use that rating scale because students are familiar with it - and it really is intended just to give you a general idea of relative difficulty.

Next, 100% of the CAT questions could be 700-800 and yet someone could still score below the 90th percentile (which is the percentile ranking for a 700). The test is not scored based on some kind of an average of the difficulty levels. The algorithm is very complicated. The mix of question difficulties doesn't directly translate into your score. (I could spend a long time right now describing how the algorithm works - but I'm guessing you probably don't really want to know. :) Suffice to say that it's quite complicated and what you describe is not unusual or unexpected for this kind of test.)

Third, as you may have found when you took the real test, most people feel that the quant portion of GMATPrep tends to be easier than the real thing (and than our tests), particularly at the higher scoring levels.

Now, there is one real limitation to our tests that doesn't mimic the real test - we don't have any experimental questions. The real test has some number of questions that don't actually count towards your score. Those questions are not tied to your performance or to any particular difficulty level, so the better you are, the more likely it is that you'll see some experimentals (on the real test) that are easier than your scoring level. It's like getting little 'mini-breaks' every so often on the real test.

Of course, it's also possible that your experimentals will also (randomly) be hard and you won't get those mini-breaks.

Our test doesn't have experimentals. We don't want to give you mini-breaks when that's only the *best* case scenario - but you might not have the best case scenario. We want to prepare you for the hardest possible scenario - ideally, we want you to feel that the real thing is the same as or easier than ours. The last thing we want is for you to get into the real test and think it's a lot harder! :)

For the above reason, the time pressure also tends to be harder on our tests - again, because we want you to be fully prepared for the worst-case scenario. If you're struggling on our tests more so than on the real thing / GMATPrep, part of the problem might be timing issues - in which case, you want to have those pointed out so that you can fix them before you get back into the real thing again.

If you'd like to get some more specific advice about your latest results and where to go from here, please post in the General Strategy folder of the Ask An Instructor section. Use the below to analyze your most recent MGMAT CAT(s):
http://www.manhattangmat.com/blog/index ... ice-tests/

Then post (in that older folder) and tell us the results of your analysis and what you think you should do based on that analysis. (Note: do share an analysis with us, not just the raw data. Part of getting better is developing your ability to analyze your results - figure out what they mean and what you think you should do about them!)

Hope to see you over in the General Strategy folder!

[joey_jamal]

Hi Stacey,

Thank you for explaining the logic behind your 200-800 difficulty scale for questions and how that doesn't reflect the actual score. However, I still fail to see how the CATs mimic the real GMAT even if it doesnt have the occasional experimental questions.

My questions is if the CATs give me 32 out of 37 questions from the difficult bucket of questions ( provided that I didnt answer all the difficult questions right) is that the same difficulty level I should be expecting on the real GMAT? if No then will it be slightly easier in the real GMAT or it will be much easier on the real GMAT and you just try to give us extra preparation?

[samflem]

Hi there,

Please post all GMAT Strategy questions (including scoring inquiries) in the "General GMAT Strategy Questions" thread under the "Ask an Instructor" section of our forums. Thank you!

Best,

Sam
Student Services
Manhattan GMAT

[irwin.kim]

I have a slightly-related observation. I just finished my 6th CAT. For the earlier CATs, my questions were almost all in the 700-800 range, for both quantitative and verbal sections. By the 5th CAT, half of my verbal questions were 600-700, including a string of 7 questions at the end of the section (which is when I would assume I would be seeing 700-800 level questions). By the 6th CAT, less than half of my verbal and half of my quant questions were 700-800 level.

Here's the breakdown of 700-800 level vs. total questions:
CAT3: Q31/37 V30/41
CAT4: Q30/37 V32/41
CAT5: Q30/37 V21/41
CAT6: Q14/37 V13/41

Is there a cap on 700-800 level questions, which results in the lack of such questions in my last CAT? My score was higher for CAT5 and CAT6, but the difficulty level of questions makes me think that the scores are artificially high. Is this true?

Thanks!
Irwin

edit: caught a rather embarrassing grammatical error! I guess I need to spend more time on sentence corrections...

[Eric_in_SS]

Hello Irwin,

There is no cap on the number of questions at any given difficulty level. There are enough questions in the question bank for 6 fully distinct practice tests with the appropriate level of difficulty for each question.

The scoring of the practice test mimics the scoring on the official exam. We work to make our algorithm as close as possible to the official GMAT scoring algorithm. It should not be the case that your practice test score is artificially high.

I hope this answers your question. Please let us know if there is anything else we can help you with.

Best,
Eric