The Word on Unofficial SAT Questions

Some of The Impostor comments are worthy of highlighting.  This is long, but I think worth reading every word:

PC Keller left this comment on Kitchen Table Math:

I recommend staying with College Board questions only. There are so many of them available that if you use them efficiently, they should be enough. You have the blue book, the on line course, and whatever QAS tests that fall into your hands. If you need more than that, I'd recommend old psat or even outdated SATS.

Here's why I don't like "fake" tests:

1. Timing issues: when you practice, you are not just practicing with the concepts. You are also working on your time management. But fake tests often take too long (or less often, too short). They are not real so you are not getting real information about how your time strategy is working.

2. Level of difficulty issues: when you practice with real tests, you are in a sense calibrating yourself. You are learning how hard you have to think to solve a problem #5 vs a problem #17. Fake tests do not have the right level of difficulty throughout so they completely mess up your calibration.

3. Quality control: some of the problems are awful -- misworded, amgiguous, whatever -- and you don't know if you are having trouble because it's you or the problem. This comes up on the forums at college confidential all the time.  People occasionally argue that doing harder problems will make the SAT seem easier. But this is not like lifting weights. What will make the SAT seem easier is mastering the SAT-level problems that you have available to you.

Akil Bello from Bell Curves:

The problem with bad fake questions is they give a clouded message about what is valid SAT logic approach and what is not. One or two bad questions here or there are not terrible but if the flavor and theme are not correct you are likely to be severely disadvantaged. I think this holds true of questions that are too hard, ones that are too easy, and ones that are just plain off kilter.

Erica Meltzer from Ultimate SAT Verbal:

In response to your semi-rhetorical (but not really) question "so what if the answers of most test-prep books are off?" I would say: on one hand, yes, in some cases dealing with "off" questions can build some mental flexibility (if you know it, you know it, and it doesn't matter how the question is asked), but for the vast majority of people who are just encountering certain concepts (e.g. dangling modifiers) for the first time -- especially on Reading and Writing -- it's a big help to know exactly how the material will be presented.

In addition, the real problem with the questions that don't quite hit the mark is that there's often no way to determine the answer through any logical process. The answer is only the answer because the makers of Kaplan or Barron's or PR say it is, not because it's actually the answer. You don't get to refine your reasoning skills working like that. And if you don't work on that *process*, it doesn't actually matter how much prep you do because, in the end, your ability to engage in that process is a big part of what's being tested.

JK Duffy:

The College Board is not telling us where their questions come from. Their "very careful review process" explanation is intentionally vague to put it generously.

These tests are *Standardized*. They must give consistent results across millions of test takers year after year after year (Barrons, Kaplan, etc have no such constraints or goals). The goal of the test is as follows... a score of 2000 by a male student from New Jersey in June 2011 MUST represent the EXACT same thing as a score of 2000 by a female student in California in October of 2006. Colleges will only rely on a test like the SAT so long as they believe the test is meeting that goal (or as close as measurably possible). You cannot do this by throwing a few Math and English teachers in a room and whipping up some questions that are "based on what kids are learning in school".  Frankly I think it is disingenuous of the College Board to even claim such a thing but that is a different argument 'altogether' (not 'all together', which is the type of thing you will have drilled into you at Gruber's and Princeton Review).

So how does the College Board construct the test questions then? Well, I don't know, but I will give you my opinion. They use 'formulas'. They have a library of concepts that they want to test, a library of methods to test each concept, and a library of incorrect answer choices that go along with each concept/method pair. This is why you see patterns repeat so frequently on the tests. This is why an experienced test taker is able to pick actual CB questions out of a list of impostors. This is why a student who has studied the wrong answer choices on passage questions can get many of the questions right without even reading the passage (or after reading the passage get all the questions right simply through process of elimination).

The big test-prep companies do not seem to make any great effort to figure out these formulas. In many cases, they don't even make much of an effort to identify the concepts that are being tested in the first place. At least this is the case with the ones I am familiar with. As you've noted, there are some quality tutors out there who have good material. I think in every one of these cases you are looking at someone who has made the effort to accurately replicate both the concepts and the formulas that the CB uses on actual tests. And in most cases I think even they would recommend you spend most of you time with actual CB questions.

To conclude (finally, sorry so long), I think that once you grasp all the underlying concepts (what is a comma splice, what are the exponent rules, etc) the rest of your time should be spent mastering the test itself. The highest scorers are not the best writers, readers, and mathematicians because that is not what the CB is actually testing (even if they claim otherwise). The highest scorers are the best at what the test *is* actually testing, namely, SAT test taking skill.

And PWNtheSAT:

Astutely said, JD. It's true that much of the material begins to look cookie-cutter after a while, but I have to give the writers of the test credit for managing to throw one or two questions onto almost every test that makes me say "Hmm...haven't seen this before." The "weekend challenge" I posted today is a great example. That's based on a question from January 2006, and although I changed the number of sides of the figure, added some color and removed the multiple choice aspect, the question is otherwise identical. And I haven't seen a similar question before or since. That's what makes it a good "weekend challenge," but nothing more.

Also, I'm flattered to be mentioned in this thread as an example of one of the "good" question forgers, but you're correct to posit that I encourage students to spend most of their time in the Blue Book. I write questions to expose and remedy gaps in student knowledge, but I'm never satisfied with mastery of my questions until I see it on corresponding questions from real tests as well.

Illustrations by Jennifer Orkin Lewis