Six months into this Perfect Score Project, and I can spot a College Board knock off question a mile away.  They feel like impostors and make me suspicious.

They feel like handbags from Canal Street: almost real, but the lining starts to feel funny, and then the zipper sticks, and soon even the decorative stitching looks “a little off.”

The question is, what’s the big deal?

The answer is, I’m not sure.  Maybe impostors are good for test prep?  The 800 math kids swear by Dr. Chung’s book.  My friend Catherine loves Chung’s book and the reviews are amazing:

I have been using the College Board SAT book and other SAT books for many years. I was really desperate to learn and aim high on the SAT score which I couldn’t get. But when I got Dr. John Chung’s SAT MATH book my SAT score has increased greatly in over few months.

Could working with impostor material make your knowledge more flexible?

It’s no accident that the College Board questions feel like a genre unto themselves.  They’ve been voted on!

Every SAT question goes through a very careful review process before making it into your exam booklet. Each question that you see has been:

  • Reviewed by a team of experts, including math and English teachers, to make sure that it reflects what most college-bound students are learning in school.
  • Thoroughly tested to make sure that it is fair for students of all backgrounds and ethnicities.

Questions that don’t make it through these steps will never show up on an actual exam.

 

They have been heavily vetted, and not just by one writer.

 

Illustrations by Jennifer Orkin Lewis

  • Akil

    Nothing wrong with GOOD simulated SAT questions but the problem is many of the simulate ones suck. Lets see how good you are… which are real and which are fake (all could be real or fake)

    1. Sophie was the perfect hostess, displaying —- all her guests.

    (A) cordiality towards (B) duplicity towards (C) astonishment about (D) disdain for

    (E) sympathy towards

    2. The ——- of her company was largely the result of her ——- approach to business, an approach that included and accounted for every aspect of a market economy.

    (A) resilience . . myopic (B) callousness . . engaging (C) complacency . . visionary (D) rebellion . . pacifist (E) profitability . . comprehensive

    3. Many private universities depend heavily on ——-, the wealthy individuals who support them with gifts and bequests.

    (A) instructors (C) monitors (B) administrators (D) accountants (E) benefactors

    The problem with bad fake questions is they give a clouded message about what is valid SAT logic approach and what is not. One or two bad questions here or there are not terrible but if the flavor and theme are not correct you are likely to be severely disadvantaged. I think this holds true of questions that are too hard, ones that are too easy, and ones that are just plain off kilter.

    Think of it this way can you prepare for the Nathans hotdog eating contest eating kilbasa? you can but will your performance in the actual contest suffer? Probably.

    • I should have added that PWNtheSAT and UltimateSATVerbal questions are are good as they get. And as I said above, people swear by Chung (last time I tried I got 5 out of 20….so I’m waiting to go back).

      #3 is the real deal ;)

      • Anonymous

        In response to your semi-rhetorical (but not really) question “so what if the answers of most test-prep books are off?” I would say: on one hand, yes, in some cases dealing with “off” questions can build some mental flexibility (if you know it, you know it, and it doesn’t matter how the question is asked), but for the vast majority of people who are just encountering certain concepts (e.g. dangling modifiers) for the first time — especially on Reading and Writing — it’s a big help to know exactly how the material will be presented. 

        In addition, the real problem with the questions that don’t quite hit the mark is that there’s often no way to determine the answer through any logical process. The answer is only the answer because the makers of Kaplan or Barron’s or PR say it is, not because it’s actually the answer. You don’t get to refine your reasoning skills working like that. And if you don’t work on that *process*, it doesn’t actually matter how much prep you do because, in the end, your ability to engage in that process is a big part of what’s being tested.

        • You just described what I’m feeling in my bones (i.e. “there’s often no way to determine the answer through any logical process”)  — and why some impostor q’s work for me, but most don’t.  

          • JD

            The College Board is not telling us where their questions come from. Their “very careful review process” explanation is intentionally vague to put it generously.

            These tests are *Standardized*. They must give consistent results across millions of test takers year after year after year (Barrons, Kaplan, etc have no such constraints or goals). The goal of the test is as follows… a score of 2000 by a male student from New Jersey in June 2011 MUST represent the EXACT same thing as a score of 2000 by a female student in California in October of 2006. Colleges will only rely on a test like the SAT so long as they believe the test is meeting that goal (or as close as measurably possible). You cannot do this by throwing a few Math and English teachers in a room and whipping up some questions that are “based on what kids are learning in school”. Frankly I think it is disingenuous of the College Board to even claim such a thing but that is a different argument ‘altogether’ (not ‘all together’, which is the type of thing you will have drilled into you at Gruber’s and Princeton Review).

            So how does the College Board construct the test questions then? Well, I don’t know, but I will give you my opinion. They use ‘formulas’. They have a library of concepts that they want to test, a library of methods to test each concept, and a library of incorrect answer choices that go along with each concept/method pair. This is why you see patterns repeat so frequently on the tests. This is why an experienced test taker is able to pick actual CB questions out of a list of impostors. This is why a student who has studied the wrong answer choices on passage questions can get many of the questions right without even reading the passage (or after reading the passage get all the questions right simply through process of elimination)

            The big test-prep companies do not seem to make any great effort to figure out these formulas. In many cases, they don’t even make much of an effort to identify the concepts that are being tested in the first place. At least this is the case with the ones I am familiar with. As you’ve noted, there are some quality tutors out there who have good material. I think in every one of these cases you are looking at someone who has made the effort to accurately replicate both the concepts and the formulas that the CB uses on actual tests. And in most cases I think even they would recommend you spend most of you time with actual CB questions.

            To conclude (finally, sorry so long), I think that once you grasp all the underlying concepts (what is a comma splice, what are the exponent rules, etc) the rest of your time should be spent mastering the test itself. The highest scorers are not the best writers, readers, and mathematicians because that is not what the CB is actually testing (even if they claim otherwise). The highest scorers are the best at what the test *is* actually testing, namely, SAT test taking skill.

          • Thanks for the thoughtful comment.  Great advice and I agree with much of it.  

            You don’t think “the best writers, readers, and mathematicians” would score well on those sections with a little practice?

            What about the reverse? Can a great “test taker” who’s not one of “the best writers, readers, and mathematicians” become one of the highest scorers?

            I guess it comes down to how you define “grasp of the underlying concepts.”  My experience is that you’ve got to grasp those concepts with vigor and strength, until mastery, to really do well on this test. Am I wrong?

          • JD

            “You don’t think “the best writers, readers, and mathematicians” would score well on those sections with a little practice?”

            They may score well, but not necessarily, and not for those reasons. The best student are often the best test takers, independently. I can’t write without grammatical errors to save my life but the college Board will never get a dangling modifier question past me, EVER! Not because I am a great writer but because it tests them in similar ways over and over again.

            “Can a great “test taker” who’s not one of “the best writers, readers, and mathematicians” become one of the highest scorers?”

            Yes, by definition. The SAT tests how good you are at taking the SAT, that’s my position. I didn’t start out that way but I am becoming more convinced the more research I do. I think they have their concepts (“what college bound students are learning in school”), but what they are really trying to test is your ability to quickly decipher their methods of testing those concepts. They are not interested it asking you what the Pythagorean theorem is, because it is NOT a math test. They are interested in testing how quickly you can determine when the Pythagorean theorem needs to be applied when presented with some abstract question. It is their form of a quasi-intelligence test. But the test is standardized, which means their testing methods have to stay within strictly defined bounds. Sure there are infinite ways to write a question, but there are strict boundaries that encapsulate those ‘infinite ways’, if that makes any sense at all.

            “My experience is that you’ve got to grasp those concepts with vigor and
            strength, until mastery, to really do well on this test. Am I wrong?

            I think this is open for argument. You definitely need to know the Pythagorean theorem and what subject-verb agreement is, etc. But I have come to believe that those things are just not what the SAT is really testing. The concepts aren’t the test. The methods are the test. But obviously the more solid you understanding of the concepts the more time you will have to crack the methods.

          • I’d love to hear more about your research.  I’ve been reading all of these brain books lately, and from what read, I was thinking that the SAT tests how flexible your knowledge is.  

            Have you read anything by Daniel Willingham?  If not, try this one about Flexible and Inflexible knowledge, because it describes the type of knowledge I believe the SAT tests.  I.e. DOWN COLD so that you can move and grove on SAT day.  You don’t want to be taxing your working memory with things like how to find the slope during the test, right?  Not a good use of working memory!  That’s got to be second nature so you can puzzle out the problem. http://www.aft.org/newspubs/periodicals/ae/winter2002/willingham.cfmPersonally, I believe we want to raise kids to know more than just the Pythagorean theorem; we want to raise kids who can problem solve, right?  That’s what I’m thinking!I always say to my son, “if UPS is close, GO TO FED EX.”  That’s code for “MAKE IT HAPPEN” in our house (which, I *think* is what the SAT is testing for).Thoughts?  

          • Akil

            I have a different question.. who cares? whats the value of getting a perfect SAT score..

            I’d posit that all scores above like 680 are all the same with a bit of luck tossed in. I can work every question on the SAT on ANY give day and know about 99% of the words yet when i last took it i missed a #15 in math, what does that mean?

            I think people oversell what the scores mean.. the college board itself says the test has a “standard error of measurement” of +/- 30 points ( http://ow.ly/5g7lL ) so for all intents and purposes there is no difference between a 600 and a 630 statistically speaking. That’s why the test gives a range for ever score. I think that margin is bigger. I think that its more like a range of +/-70 points.

            The SAT i think tests skills that people need, i think its valuable that you know vocabulary, its valuable that you can read with specificity and and understanding, its valuable that students can put a sentence together properly. I think taken at a 10000 foot level the SAT is a good tool however when you look at it in context of school systems and how students are taught these days its contrary to their experiences and thus you have to ask what does it reflect and how is it “fair.”  If your entire experience in English class is someone telling you that your interpretation of meaning of a written piece is correct then you think its unfair that the SAT requires you to interpret exactly whats said in the way its said based on dictionary meaning rather than social meaning. I once struggled to convince a group of kids that “Do you have the time?” is a yes or no question requiring only a yes or no answer. The SAT sort of forces that type of thinking on kids, in a high pressure, timed setting, with a weird scoring system that does not reward “effort” (which they are probably used to in school).

            rant done.

          • Akil

            and oh yeah …colleges do think 30 points is a difference that matters, which is sad

          • Elise

            I have enjoyed reading all your comments and probably could get into a very long discussion about all this…I do tend to agree with what you are saying.  I remember the first time my son took the SAT he got a 730 in math and the second time an 800.  We also got the detailed report the first time and I remember being amazed at how few problems he had gotten wrong to bring his score down to 730…and all silly mistakes.  It is sad that the colleges are so obsessed with these scores…I don’t think silly mistakes should keep you from getting into a top notch college.  But then again, I have an opinion on “top notch” colleges as well…ha ha.

          • Agreed!  Call me crazy, but I’d *think* a top notch college would understand what a small difference that is when you’re up in the tippy top like that.

          • ha!  Well, I could answer the question by saying Colleges care, but you said that below.

            I would argue though there’s a big enough difference between a 680 and an 800 that I was able to score that (and a little higher)  in both the Reading and Writing sections without knowing at all what I was doing.  I don’t think I could have pointed to ONE “rule” that I used to answer any of those questions.   I’d like to believe that if I know what I’m doing, I could score higher.  But I’ll confirm that after I get there.  I could be completely wrong ;)

            Math, well, that’s another story.  I have a higher mountain to climb there, which weirdly, is more fun for me.

            I remember when I ran the NYC marathon, I thought I was going to achieve this incredible sense of accomplishment that would make me feel like superwoman.  Instead, I crossed the finish line and thought to myself that it was the biggest waste of time.

            I never ran again after that day.  Ever.

            I hope I finish this project with more of a sense of accomplishment than I did that marathon!

            Missing a #15 in math means you’re human.

            The real purpose of this project is to have fun, connect with my kids, and make up for a lot of stuff I never learned in the first place and have always regretted.  Especially as my kids have become teenagers.  I’ve felt really said that I never learned all of the material I need to help them study.  

            If I get the perfect score, AMAZING.  And if not, c’est la vie.  I can tell you for sure that I had fun (and I don’t even need to get to the end to confirm that). I connected deeply with my kids over the SATs, I understood in a visceral way what they go through when they are learning, and I gained a sense of empathy that I don’t think I could have ever achieved by not having climbed in the SAT trenches.

            So, I can say from the bottom of my heart that I care.  Not about the perfect score, but about the journey.

          • Akil

            I didn’t mean you. I figured you take the test for the intellectual exploration. I’m going to take it in oct (if i can convince myself to wake up that early).. I might even come up and take it with you..

            I think in writing and reading the skill set that expressed by a difference in a 680 and 800 is almost irrelevant. If you dont know 5 arbitrarily selected vocab words what does that mean? If you dont know/notice some random grammar rules does it really mean anything. I mean the different between an 800 and a 680 in reading is getting 8 of 67 questions wrong .. what does that really say about your knowledge of how to read? I probably says more about how well you read in the timeframe given. We all will read sloppily if we read fast enough.. 

            while i value the theoretical skills the SAT tests i think that the scores and usage of the test are mostly irrelevant and overblown. Its a weird conundrum..

            anyway trash/recycling wont put itself out on the curb.

          • Elise

            I find that so interesting and surprising about the NYC marathon being a let down for you.  I climbed Mt. Washington (NH white mountains) last year and did have a great feeling of satisfaction.  I wonder what the difference was for you…hmmm.

          • JD

            I think you are doing different and deeper research than I am. I have read a few articles by Willingham after seeing them mentioned on your blog. I think you are on to something re. flexible vs non-flexible knowledge.

            I am more looking at the history and structure of the actual test. What its purpose is and how it is constructed to achieve that purpose. I am looking for patterns in the questions, answers, an incorrect answer choices. The College Board seems to go to great lengths to make the test look a lot more varied than it is. I think they are heavily constrained by the fact that the test must be standardized. I also think they are not completely honest it their public comments re. the construction and purpose of the test. I find this all very interesting independent of the goal of getting higher scores.

          • “I find this all very interesting independent of the goal of getting higher scores.”
            Same.  

            I never expected to dig so deeply into “how we learn” — but it’s actually one of the most interesting aspects of this project, for me.  

            I just put my May SAT scores on a graph and am distressed at how UNimproved they are.  Will add them to the Charts & Graphs page for all to see.  Recalcitrant is the word I’d used to describe my brain.  I feel like I have some seriously resistant mental blocks. 

            Have you taken the SAT officially?  Are the reading and writing sections as easy for you as the math?

          • JD

            I have not taken the SAT officially in over 20 years :)

            Reading is getting there. Writing is even closer to getting there. Based on my progress so far (1st stab at writing was a disaster), my expectation is that they will both eventually become easy. We’ll see how that prediction works out. I am more confident about writing. The patterns are a lot easier to spot.

          • Come on, admit that you think it’s fun like I do ;)

          • Astutely said, JD. It’s true that much of the material begins to look cookie-cutter after a while, but I have to give the writers of the test credit for managing to throw one or two questions onto almost every test that makes me say “Hmm…haven’t seen this before.” The “weekend challenge” I posted today is a great example. That’s based on a question from January 2006, and although I changed the number of sides of the figure and added some color, the question is otherwise identical. And I haven’t seen a similar question before or since. That’s what makes it a good “weekend challenge,” but nothing more. 

            Also, I’m flattered to be mentioned in this thread as an example of one of the “good” question forgers, but you’re correct to posit that I encourage students to spend most of their time in the Blue Book. I write questions to expose and remedy gaps in student knowledge, but I’m never satisfied with mastery of my questions until I see it on corresponding questions from real tests as well.

          • JD

            I like those “Hmm…” questions. The more elaborate the set-up, the less advanced the underlying concept that is being tested (usually). You did not present a polygon with an unknown number of sides. You presented a polygon with four sides. Not only that, you gave two angles and told us the other two were equal to each other. Anyone who knows the basic formula (and every student should, it is tested repeatedly) for the total interior angles of a polygon should get this problem pretty quickly if they aren’t slowed down by the abstract method in which it is being presented.

            A real math test would have a student explain the concept, perhaps even prove the formula. When you add an angle, you are really adding a triangle, or 180 degrees. So if it has 3 sides (3 angles), it has one triangle, or 180 degrees. If it has an extra angle/side, it can be split into 2 triangles, or 360 degrees. If it has 5 sides/angles, you can split it into 3 triangles, or 540 degrees (have the student draw the polygons to prove it). If it has 10 sides/angles, it can be split into 8 triangle, or 1440 degrees!

            But the SAT is not a real math test. They don’t really care whether you know the formula or grasp the ‘math’ concept. What they are really testing is whether or not the student can be deceived by the method used to frame the question. Not whether they have a solid grasp of the underlying math. Although a solid grasp of the underlying math is an asset, it is not what the college board is testing.

            I seem to be getting more cynical by the minute :)

            PS   I think math is the most forgiving when it comes to impostor questions. You might write a question that misses the mark as far as emulating the SAT goes, but it may still give a student good practice at ‘plugging in’ for example. That experience can be useful when the student runs into actual SAT question whereas a poor writing question is almost completely useless for a student (who vs. whom; waste of time).

          • There actually IS an SAT math question that tests that polygon concept. Strangely, I happened upon it in the last few weeks while doing a blue book math section with my son. I got it wrong; he got it right. When he explained the solution to me, he taught me the polygon rule you describe below.

            I’ll see if I can track it down. It may have been an online College Board test.

            I have 2 questions:

            1) How do you feel about the ACT?

            2) Do you think that the puzzle of an SAT question has any value? I have no idea, but I do think they are fun because of that aspect.

          • JD

            1) I think the ACT is just another version of the SAT. College admissions offices seem to agree. Some grading, format and base-concept differences, but nothing material as far as what is really being tested. Is there any science knowledge required at all on the Science section? There doesn’t appear to be.

            2) Are you asking if I think the ability to solve these puzzles has any value? I think in the context of the SAT the value is in how colleges use the test scores (IOW, the skill is highly valuable). In the context of the real world, it depends on the situation. There are times where a brilliant piece of problem solving is indispensable. OTOH, I’ve worked with some very successful people in the business world who I think would have a difficult time working to a draw in a game of tic-tac-toe. It just depends.

          • The more I think about it, the more I think puzzling/problem solving IS an extremely important skill — whether you’re a line worker in a factory or a middle manager.

            In fact, I see examples of the necessity to critically think everywhere I look.  

            Take, for example, today: My last 2 water bills have increased over 100% per month (this sounds like an SAT problem, right?!).  No one can find a leak; it’s not my meter.  The man in charge at the water company thinks that asking if I took a lot of showers this month, then throwing up his hands with an “I don’t know then” look, is “problem solving.”  

            After spending about 10 hours dealing last month, I gave up, paid the bill, put my head in the sand, and hoped it would resolve itself.  

            Of course it didn’t.  And then the next bill arrived and reminded me that I had to face it again.  

            But now the guy in charge is in the hospital, so I got his #2 man on the phone, who started “puzzling” with what could possibly be the problem (omg, what a novelty.  Someone who cares.).  He came up with a whole list of ideas of things for me to try (which unfortunately did not result in finding the problem…..but I appreciated that he went beyond “too many showers” as a response.)

            This is all just to say that whether you’re a factory worker or a CEO, I think the value of being able to puzzle out the angles of an obstructed polygon, is being underplayed by a lot of people who would prefer to disparage the test than make sure kids learn massive amounts of core knowledge to mastery. 

          • Anonymous

            I agree.  “Puzzling through” skills are vastly under rated. Life isn’t like school where you get a clearly defined question, only the relevant info, and it leads to one best answer. I wish the schools taught, and that the SAT reflected, more real-world scenarios – where you are presented with a problem, given a bunch of relevant and irrelevant info and you need to sort out what you know, what you need to know and what info you are going to use to bridge the gap.

            My best students can clearly articulate what they know and what they need to find out in order to get to the answer.  My worst students can’t articulate what they know or don’t know much less what they need to know in order to solve  a problem. It’s like they don’t have any sort of methodical evaluative process. And not having that in place sinks them on the test and everywhere else.

            The one thing I hope to teach my daughter is *how* to learn.

          • It all sounds so much easier than I’ve found it to be.  

            At the moment, sh**y final exam grades just arrived in the mail, and I’m left wondering whether it’s even possible to teach these skills — or if some people are just born with it, and others are not.  

            Off to read Willingham article about whether or not it’s possible to teach Critical Thinking. http://www.readingfirst.virginia.edu/elibrary_pdfs/Crit_Thinking.pdf

            Also, can’t stop thinking about Battle Hymn of the Tiger Mom.  I just read it in two days and loved every word  for it’s brutal honesty and self-depricating humor. I can’t believe everyone had such a savage reaction to her.  She was being funny……and HONEST.  You don’t see that kind of honesty too often.  For that alone, I think it’s a great book.

            I won’t give away the ending, but it’s not what you think (or at least not what I thought).  Personally, I think it’s a must read for every parent.

          • Anonymous

            I’ve got in on hold at the library.  Yes, I think these skills can be taught – over a long period of time. And at the same time – you can lead a horse to water but you cannot make him drink.  And parenting, as you know, is for the long haul. You are planting seeds now that haven’t yet come to fruition. What you are doing matters and will pay off :-)

          • I sure hope so.  My brother is here tonight, and he identifies strongly with my son, and he says I need to send him away to some special boarding school where he went, and they teach you how to study properly (he ended up there after 3-4 other high schools).  I said “really?  There’s no other option short of that?”  Apparently, not that he knows of.  Oy.

      • Anonymous

        I also wanted to say that in terms of Writing, most of the writers for commercial books tend to have trouble telling the difference between a sentence that is relatively complex and that has precisely one well-disguised error and a sentence that is just plain awkward and confusing and awful. 

  • Akil

    You are spotnon! What made you discount the others

    • The answers to question # 1 don’t feel right.  
      #2 was close, but my gut just said #3.  Sometimes I can’t even tell what it is.

  • Akil

    Let’s try once more and make it harder:

    As shown by her reclusive behavior, Janet ——- interacting and conversing with people at social gatherings.

    (A) rewarded (B) condoned  (C) embraced (D) relished (E) abhorred

    As one would expect, the reclusive poet ——- public appearances, and invasions of privacy.

    (A) rewarded (B) condoned  (C) embraced (D) relished (E) abhorred

    • @c808a286fd32fe29893ccc0228f2721a:disqus #2 (this is fun)

    • although that comma after appearances is suspicious now that I’m off the phone and looking at it.  But I’m still going with 2.

    • although that comma after appearances is suspicious now that I’m off the phone and looking at it.  But I’m still going with 2.

  • Akil

    you are awesome.. the comma was my typo!

     

    1. Once the plan to ——- offices is complete, the managers
    of the company expect that the combined departments will require less money to
    operate.
    (A) comprise   (B) disconnect (C) consolidate (D) continue (E) sever

    2. In the classroom, Carol was unusually ——-; on the playground, however, she became intractable as the other children.

    (A) optimistic (B) mercurial (C) magnanimous (D) taciturn (E) docile

    3. Whether Nathan Lane is performing on Broadway, acting in a
    film, or discussing the techniques of acting, the actor’s animated disposition ——- his passion for his profession.

    (A) misrepresents (B) exaggerates (C) satisfies (D) reflects (E) disguises

    4. It is not surprising that the new book tracing the history and cultural impact of comic books has a ——- tone since comic books are rarely discussed seriously.

    (A) cathartic    (B) didactic    (C) reserved (D) congenial    (E) flippant

    • Oh, i’m getting good! #2 is the real one, though I think there could be a typo in there.  A missing “as.”  

      1 and 3 or not right.

      #4, is very close to a real question.

  • Pingback: The Word on Unofficial SAT Questions | Perfect Score Project()

  • Pingback: Pick the Impostor | Perfect Score Project()

  • Pingback: What Makes a Good SAT Question? | Perfect Score Project()

  • Pingback: Good Poker Players Make Good SAT-Takers (And Other Quotable Moments) | Perfect Score Project()

  • Pingback: SAT Prep On A Budget | Perfect Score Project()