Behind the Scenes: Your Role in Validity

Oct 25, 2012
Tags: Official GMAT, Validity

By: Lawrence M. Rudner 
 

The Graduate Management Admission Council recently introduced the Integrated Reasoning section to the GMAT exam. It is an incremental change, and both test takers and schools can still rely on the GMAT Quantitative, Verbal, Total, and Analytical Writing Assessment scales that they know and that have well-documented validity and reliability. Most of the material will be familiar, meaning proven methods for doing your best on the exam continue to hold. By allowing you to continue to demonstrate your skills in ways schools understand and already use in the admissions process, this incremental approach minimizes the risks for schools and you. Recent advice in online discussions implying that you should not take IR seriously because, for now, business schools are not is simply wrong. That bad advice could cost you a valuable advantage in admissions.

New tests, whether they are a complete overhaul or a new section, are typically developed after years of research. Various item structures and measurement models are explored in order to identify what does a good job of measurement and what is scalable. Questions are piloted to gather data to assure that the items are measuring as intended and are free of bias. New forms are developed and equated to assure that results are always comparable. These efforts help assure that the test has content and construct validity. The development process helps assure that the questions do a good job covering desired content. But, until a new test or section is live in the field with motivated test takers, there is no way to accurately assess how well it actually predicts performance. It could turn out to be a major disservice to schools and students. Given the high stakes associated with admission testing, we view this as an unacceptable risk. 

The incremental approach taken by GMAC assures that the “new” GMAT is at least as valid as before. Quantitative, Verbal, Total and AWA are unchanged. To the extent that IR measures a related but different skill, its introduction will improve the test’s predictive validity. Why do you care? The greater the predictive validity, the better the test is able to show everyone’s strengths and the fairer it is. Our research with unmotivated test takers has led us to expect a significant improvement beyond our already well-documented, highly valid test. 

Until it has been demonstrated to be relevant and to work well for them, admissions personnel should not, and typically do not, give much weight to any new section. But this is not the same as ignoring the new section. It is hard to imagine that anyone will disregard readily available, objective data, no matter how unproven. If two applicants look almost identical and one has a better score on a new section, who do you think will be preferred? The test taker who blew off the section may be more likely to blow off a course. 

It is in everyone’s interest for you to make a serious effort. Admissions personnel get a realistic view of how well the new test or section works in their program. GMAC obtains quality data to document the improved validity of the GMAT exam. You have the opportunity to showcase your skill set in an area that business schools agree is critical to success in the classroom. Making no effort when even one competitor might is a clearly a mistake. 

The reality is that studying for IR will probably help you with the other sections. The GMAT is about reasoning skills. As a test taker, you will have nothing to lose and possibly much to be gained by becoming comfortable with IR questions.

Lawrence M. Rudner, PhD, MBA, is vice president of research and development and chief psychometrician for the Graduate Management Admission Council. This blog post is reworked from a Demystifying the GMAT column that originally ran in Graduate Management News in October 2012.

OK