Assigning multiple choice exercises as homework

I'm in the process of writing nightly problem sets for my new Honors Physics I course.  These are essentially the same problems that I used for AP Physics B; however, I'm rewriting the problems to include more verbal explanations, and so that they're typeset on the front and back of a single page. 

Most of my homework assignments are simply rewrites of textbook end-of-chapter problems to make them AP style, and to explicitly demand verbal responses.  Occasionally, though, I want students to work through a set of multiple choice questions.  I *could* give these as a quiz in class; but to make the quiz worthwhile, I'd have to go over the quiz in detail.  I usually want to use class time for other purposes.  How can I usefully assign multiple choice problems for homework?

The issues are probably obvious... It's too easy for students merely to copy the (presumably) correct answer from friends without thinking through the answer thoroughly.  The simplest response is to require students to justify every answer with verbal reasoning.  I do this occasionally... but I don't want to assign more than three or four multiple choice per night in this manner.  How can I get folks to work through a longer set?

Once in a while, I'll pass out a 20-question-or-so multiple choice exercise and a scantron form.  I require each student to answer each question on the scantron by himself, without collaboration.  (You can enforce this either with an honor pledge, if you can trust it, or by using 20 minutes of class time.)

Next, I require each student to check his answers with classmates.  Everyone's final answers go on the reverse side of my two-sided scantrons.  I only grade the final answers.  The trick is, if a student changes his answer based on collaboration, he must write a verbal justification.

Grading is easy, since I can scan the scantrons, and spot-check the justifications.  The assignment is not excessively long, because students only have to write justifications for the ones they missed initially, and since the discussions with classmates when they are finding out which ones they missed will make justifications quick.

Sometimes I'll ratchet up the grade incentive for useful collaboration.  I'll take off one point for the first wrong answer, but two MORE points for the second wrong answer, and three more for the third... someone who gets 16 of 20 right would thus earn a 50%.  This grading system has led to wonderful physics arguments within the class, which of course is the whole point of any homework assignment. 

Basic astronomy unit in general physics -- can you help me?

For most of a decade I've been teaching a three-week "astronomy" unit to my general physics class.  The students are generally excited by the idea of learning astronomy, which is a boon to the teacher of a class of seniors in May.

My inspiration for the unit was a one-week summer course I took at the University of Montana called "The Celestial Navigation of Lewis and Clark."  Burrito Girl (my wife and sidekick) and I spent a week learning how to use a sextant, and how to determine latititue and longitude from the sextant reading.  We also visited some sites where L&C actually made camp, there finding National Park Service employees giving demonstrations of period instruments. 

*It was a bit disappointing to learn that the celestial observations that L&C diligently recorded throughout their trip proved to be essentially worthless.  By far the more accurate map of their journey was reconstructed from Clark's "dead reckoning" intuitive estimates of travel distances as recorded in his log.

The first day of class consisted of a crash course on local astronomy -- nightly and yearly motions of the sun, moon, and stars.  Though the terminology ("celestial equator," "ecliptic") was new to me, the general concepts were not new -- for example, stars, including the sun, apparently rotate counterclockwise around the north star.    To the majority of the class, though, who were not physicists, these ideas of motions of the stars and planets WERE new and difficult.  Even in their final presentation, a group of classmates indicated that the sun would be directly overhead in Montana at noon on June 21.  Oops.  I thought it might be useful for me to teach my classes some of these basics. 

Between the material from the U Montana course and some exercises I picked up from a CAPER* workshop, I took three weeks to cover:

*CAPER stands for Conceptual Astronomy and Physics Education Research. "Teams" of astro professors at several university work under this umbrella. At an AAPT meeting I picked up some of their "lecture tutorials," which have been most useful in this unit. Much of this sort of work is online -- search under "CAPER."


* motion of the earth, moon, and sun
* apparent motion of the stars
* measuring stellar distances with parallax
* use of a sextant to determine longitude and latitude

My fundamental goal is to bust the standard misconceptions.  If nothing else, I want students to know that seasons don't depend on the earth's distance from the sun, and that moon phases are NOT a result of earth's shadow.  Just getting this much retention is more difficult than you might imagine.

We culminate the unit with a nighttime observing session.  That's easy to arrange at a boarding school -- we just gather behind the dorms after lights out.  We use Starry Night software on a laptop, or Star Walk on an iPad, to help deterime what we're looking at. 

What about homework assignments?  Well, I do assign homework.  Many of the problems are based on end-of-chapter questions in the two astro textbooks I have.  These are based on what we did in class, but sometimes thought provoking -- "How high will the sun be at noon on June 21 at our latitude?"  "Star A is measured to move at most 2 arcseconds in relation to the fixed stars over the course of a year.  How many light years away is this star?"  "Using a map but not online search tools, choose an American city in which the sun sets as late as possible."

Your help?  I want to find multiple choice quesitons for quizzing and testing purposes.  For standard physics topics, I have AP and Regents and SATII and Physics Bowl test banks.  These provide a virtually limitless source of multiple choice ideas.  But for astronomy, I've only found a few online sources -- and these have been pretty danged lousy.

Can anyone point me to a well written bank of multiple choice questions based on basic astronomy concepts?  I can and have written some of my own questions, but I always like to have some external validation for what I'm doing.  Post a comment, or email.  And a future post will discuss the observing session -- assuming it's not clouded out for the third year in a row.

GCJ

Mail Time: Electric field and potential inside a conductor

Georgian Michael Gray writes:

metal lattice representation from nanotechweb.org

I'm having a hard time with a rationale for number 19 on the 2009 MC exam. Can you help me?

Michael got a copy of the 2009 exam from last year's AP Summer Instititute.  It is also available for purchase from the College Board.  (That's why I can't post the text of the question here.)  Come to a workshop or institute, and I'll give you a copy of EVERY released exam question since 1979.

 

The question in question asks about the electric potential on two sides of a metal block.  The potential is V on the left side of the metal block; what is the potential on the right side?

So can I help?  Sure... but the answer depends on how deep you want to go.

At the truly introductory level, it's a fact of physics: metal objects have no electric field inside them, and are at the same electric potential everywhere. Charges can only reside on the surface of a metal object.

At the physics C level, you only need the last fact. Then you can use Gauss's law to show that a gaussian surface drawn inside the metal encloses no charge, such that there can be no electric field inside a conductor. Then potential is the spatial derivitive of electric field (i.e. voltage tells us how electric field changes). Since electric field is unchanging throughout metal, the potential must be the same everywhere but not necessarily zero.

And there's an even deeper answer, the one that discusses the periodic potential of the lattice of atoms in a metal and why electrons are free to move about the metal in the first place.  That's a graduate material science answer.

This is a HARD physics B question. I wouldn't worry if your students miss it.

GCJ



An AP physics problem-of-the-day

AP Physics reader Craig Fletcher made me aware of learnapphysics.com, a site run by his colleague Richard White at the Polytechnic School in Pasadena.  [No, not Cal Tech, an independent high school near Cal Tech where many professors' offspring attend.]  Richard provides -- for free -- an new AP physics-style multiple choice problem each day.  He'll even deliver his problem automatically to your email inbox.  Richard bills his service as a study aid, but I think of it just as much as a teacher's aid.  Need a multiple choice question quickly for today's quiz?  Look at his site.

I browsed a bit, and while I can't vouch for every question, everything looked quite strong -- at the correct level, doable without a calculator, with reasonable language.  The only caveat I'd attach right now is that physics B and C questions are mixed together.  Nothing wrong with that at all, just be aware.  Generally multiple choice questions from unedited sources are buyer-beware.  I don't recommend using questions from the ubiquitously available College Physics tests you can find online, for example.  But Richard's site seems like one I can recommend.

It's not like there's any real shortage of good AP-style problems out in the online universe.  But, any further suggestions, please send them along, and I'll point readers in that direction.

GCJ

More multiple choice: New York Regents exams

When I first started teaching, my college roommate and Yankees fan Dai Duong sent me a New York Regents exam.  Of course, in those days the exam was hard-copy only, not online.  I remember liking the exam, and using some questions from it on my own tests, but since I lived nowhere close to New York* the Regents exam strayed from my mind.

*I was in Boca Raton, Florida, which is close to New York only in terms of culture.

Frank Noschese's comments on my previous post about multiple choice appropriate for general physics inspired me to check out recent Regents exams.  Frank is right -- the National Science League tests are essentially a rich man's version** of the Regents.

** "Rich man's" version not because NSL questions are any better, but because it costs money to purchase each year's test.  The Regents tests are available free online.

The level of the Regents is dead-on to what I teach in my general physics course.  My choice of topics does not exactly mimic that of the Regents exam, but is pretty close.***  In any case, now that Regents tests are online, I've now found another solid souce of physics questions.  Thanks, Frank!

*** More on topic selection in General Physics soon.

Breaking news: No more penalty for guessing on AP multiple choice

Just released in a letter to College Board consultants:  Beginning next year (on 2011 exams), the multiple choice score will not include the "guessing penalty."  The multiple choice raw score will be the total number of correct answers, period.  Expect a formal announcement to all AP teachers in August; the course descriptions on AP central supposedly now reflect this change.

What does this change mean for you?  Well, it should mean almost nothing... students should always guess on multiple choice questions that they read.  However, it now becomes critical that students make random guesses if they are running out of time at the end.  This is certainly a substantive scoring change, but I believe that its effects will be way overblown on message boards and in conversations.  It's not a big deal.

GCJ

Multiple Choice appropriate for General Physics

For juniors and seniors, my school offers only two levels of physics that don't require a prerequisite:  AP physics B and General Physics.  I will have a number of students taking General Physics as a graduation requirement, so this course must be accessible to folks of all academic abilities.  My approach to General Physics has been to teach mechanics, waves, circuits, and optics to the AP-B standard.  The goal is that if my students could take the AP physics B exam, but strip out all of the topics that we haven't covered, they'd all pass.  That makes for a slow-moving but powerful course.  On my February exam, I do give a long set of AP free response questions, on which my students perform easily to passing standard.

I've already detailed the free response questions, most of which are at the AP level, that will be on this spring's final exam.  But I want to give a set of multiple choice questions as well.  Most of the multiple choice questions available in the physics universe are at the AP level.  Since I have not spent any time this year with the General class practicing the correct approach to multiple choice questions, and since they have not had multiple choice questions on any other test, and since I want the multiple choice to HELP their performance rather than to serve as an additional challenge... I need some straightforward multiple choice. 

Enter the National Science League.

The NSL is a shamelessly commercial organization which sells science (and math!) contests at all levels from elementary to high school.  Their business plan is sound -- create straightforward multiple choice tests annually, promote the tests as a national "contest," and get schools nationwide to open their wallets.  In return for a sum of money, the school gets the test itself, some certificates and medals, and recognition on the NSL website on which schools are ranked in each subject by the performance of their top 10 students.

I personally have no trouble with the commercialism.  The NSL is filling an open economic niche.  From a competitive standpoint, the "contest" is worthless:  the 40-question multiple choice physics test is written such that any AP student who misses even one should be flogged.  Commercially, though, that makes a lot of sense... more teachers will be willing to give the NSL contest, and thus pay money for it, if they know that their students will feel successful.  Nothing wrong with that.

I use the NSL physics contest not as a contest, but as a set of multiple choice questions uniquely suited to my general physics class.  By coincidence, the topics of the NSL questions align nicely with my own choices:  mainly mechanics; some waves, optics, and circuits.  There are always a few thermodynamics or electrostatics questions that I have to cut out, but usually at least 35 of the 40 questions are right in the General Physics wheelhouse.

As part of my final exam this year, I will give the 2010 NSL.  As an added benefit, I have a built-in review packet: for practice leading up to the final exam, I have given out a previous year's NSL.  My students are consistently missing no more than 3-6 questions out of nearly 40. 

GCJ

Multiple Choice questions may have more value than you think

It is common for teachers in other disciplines to view multiple choice questions as the lazy teacher's way of avoiding grading.  In physics, that could hardly be farther from the truth.

Even physics teachers often think of multiple choice questions merely as a useful way of evaluating student understanding broadly and quickly -- after all, it takes a student only about 1-2 minutes per question to respond, and a teacher 1-2 hundredths of a second to grade by machine.  A multiple choice question can be even more valuable.  Some ways to use multiple choice questions creatively:

* I've detailed many times the "test correction," in which students earn back half credit on a multiple choice item they miss by explaining the answer thoroughly.

* I've also explained my typical "clicker exercise," in which teams of two students each have a chance to respond to a multiple choice item on the classroom response system.  The ensuing discussions of each questions can be more valuable than the best-designed homework question.

* Multiple choice questions can be expanded into free response-style homework question with the addition of three words: "Justify Your Answer."  Just today I decided that my class had had enough AP free response review homework.  So I took three of the tougher magnetism questions off of the recently released 2009 AP multiple choice exam, printed them out on a page, and assigned the justifications for homework.

* Even after a question has been assigned and justified, you can develop a further quiz based on the situation presented.  For example, consider a typical multiple choice question in which two railroad carts bounce of each other.  Originally, students may have had to find the amount of mechanical energy dissipated in the collision.  For some reason, that calculation frequencly causes trouble.  So, after I've demanded a thorough justification, I give a quiz -- same question, only this time the carts stick together after collision.  If the student truly understood the concept and calculation on the original problem, the new one should be no trouble.

Condider the multiple choice question below:

A car collides with a mosquito.  Which experiences more acceleration in the collision?

(A) The car, by a factor of about 10⁶

(B) The mosquito, by a factor of about 10⁶

(C) The car, by a factor of about 10¹

(D) The mosquito, by a factor of about 10¹

(E) Both experience the same acceleration.

When correcting this problem, some students will obediently go through the Fnet=ma calculation, estimate the mass of the car to be a million or so times the mass of the mosquito, and correctly answer B.  But not everyone will truly recognize the underlying principle here: This reasoning depends on Newton's Third Law, which demands that the forces experienced by each object in the collision must be the same.

So I'll ask this follow up question on a quiz:

A car collides with a mosquito. The mosquito sticks to the car after the collision.

(a) Which experiences more acceleration during the collision, the mosquito or the car?

(b) Which experiences more impulse during the collision, the mosquito or the car?

(c) Which experiences more force during the collision, the mosquito or the car?

GCJ

In prase of Giambattista's multiple choice questions

People ask me about textbooks a lot.  I don't have a favorite.  I've tried several, but I've never been truly happy with any of them -- most are too detailed for the novice, even though they serve as reasonable references once students have experience.

Lately, it seems that all the textbooks are trying to advertise their usefulness for test preparation, which is silly -- if it's a good textbook, then it will be useful in preparing for good test.  Period.  But someone is insisting that texts add test prep gimmicks, sometimes via online resources, more often by merely adding a section of multiple choice questions.

Be careful with these multiple choice questions.  Most are LOUSY.  Serway, for example, has taken a bunch of plug-and-chug problems and made them into "multiple choice" items; no, that's not what a multiple choice question is about.  They and other texts also ask questions that are so involved that they are not answerable in a minute or two -- EVERY multiple choice available requires an average of, at most, a couple of minutes per item.

The best multiple choice questions that I personally have seen in a standard textbook are in the Giambattista, Richardson, and Richardson text.  This one was recommended to my by Martha Lietz, a colleague from the AP reading who teaches outside of Chicago.  Giambattista has a reasonable grasp of the purpose and scope of a multiple choice item.  Take a look at these, from the second edition, that I adapted for a recent quiz:

23. An organ pipe is closed at one end. Which sketch is NOT a possible standing wave pattern for this pipe?

24. Which sketch shows the lowest frequency standing wave for an organ pipe closed at one end?

Now, I know that these would be back-to-back problems on an extensive exam, and I know that many exams use five rather than four choices.  Who cares.  The thrust of these problems is good.  The first tests whether the student recognizes that a closed pipe includes an antinode at one end and a node at the other; the other simply tests visually the student's ability to apply v=λf based on a picture of the wave.

Giambattista has more good multiple choice questions, of course, and a few bad ones as well.  But if you're looking for a good source, get yourself a copy of this book.

GCJ

Mailbag: Should you guess on multiple choice problems?

From Joey Konieczny, from Georgia:

Another question I had was about how you grade your multiple choice questions. The AP exam takes off 1/4 points for every wrong answer, correct? Do you do the same? Do you encourage your students to not answer your exam questions if they are not confident in their answer?

Yes, the AP multiple choice exam is graded exactly like the SAT: with 5 choices per item, students get 1 raw point per correct answer, and lose ¼ raw point per incorrect answer as a correction for guessing. And yes, I do the same on my in-class tests, because they are given as authentic AP practice exams.

However, I encourage my students to answer EVERY SINGLE multiple choice item, regardless of how confident they may be. Why? Look at the mathematics, to begin with.

Imagine a 100 question multiple choice test answered completely randomly, with absolutely no hint as to the correct answer. A person who leaves everything blank gets zero raw score. The person who answers every question will, on average, get 20 right for 20 raw points; but this person will get 80 wrong, for -20 raw points, leading to an overall raw score of: zero. Exactly the same as if the test were blank.

My point here is that, contrary to the myth that some idiot test-prep corporations propagate, there’s no harm in random guessing. On average, random guessing scores exactly the same as leaving blank. Of course, there’s really no point to truly random guessing: if someone runs out of time with ten questions left, I do not advise randomly filling in bubbles.

However, this analysis shows that there is, in fact, benefit to guessing that is in any way not random. Now, some will give the advice that if you can’t surely eliminate a couple of answers, then you shouldn’t guess. I disagree. My students – and, since you’re reading this blog, yours too – develop pretty good instincts about physics problems by the end of our year-long course. Even if they can’t bet their life on the wrongness of an answer choice, my students are certainly more than 20% likely to pick out the correct answer on virtually every multiple choice problem I’ve ever assigned. So they should always guess! In the best-case scenario, these non-random guesses add a few raw points to the overall raw score. And in the worst case – THIS IS THE CRUX OF MY ARGUMENT – these random guesses do no harm.

My rule is, if you read a question, mark an answer.

Now, it’s actually a rather tough process to convince students to guess, because they’ve heard for so long about guessing penalties. The most convincing argument to encourage students to guess on multiple choice questions came from my former student, Bret Holbrook, who truly understood the methods to my madness. He reminded his classmates that I make students do a correction for every multiple choice question that isn’t right, whether or not an incorrect answer was marked. So, he reasoned, he might as well guess. That gave him at least a 20% chance of avoiding the hard work inherent in the correction that he’d have to do anyway if he left the question blank.