Tips for creating M/C midterm Qs

[Beals]

Hello everyone!

 

I am TAing for a first year women & gender studies class.  I ran the lecture one week and need to make come up with 25 multiple choice questions on the material I covered for the midterm.  My prof will select 20 of the 25 I come up with so they don't all have to be perfect.

 

I've never created exam questions before, so I want to ask for some advice.

 

Any tips? Things to avoid? 

 

All input is greatly appreciated! 

 

Cheers,

B

 

PS, I searched for previous related topics and couldn't find any.   Please link if this has been discussed before.

[queenleblanc]

From a teacher with a M.A.T. ... (headed back to school for Speech-Language Pathology, researching validity of standardized testing, etc.) 

 

 

1) What are the major concepts you've been teaching?  Those will determine at least 75% of your questions if not more. 

 

2) How "in depth" do you intend to test?  Does that match up with how "in depth" have you been teaching?

- If you have not been teaching above the level of definitions and connections of concepts (level 1 and 2 on Bloom's taxonomy), you should not expect your students to push to level 4 of Bloom's taxonomy.  FYI -- From most basic to most complex, Bloom's taxonomy includes: Remembering - Understanding - Applying - Analyzing - Evaluating - Creating.

 

3) You should include some questions that test the finer details of concepts that may be more important than others; include several questions pertaining to a particular topic or lecture if that materials is supposed to be more highly weighted for the content of the class. 

 

4) Regarding wording: MAKE SURE your question targets what you want it to target.  Don't leave room for mis-interpreting the question.  Many PhD professors (and not just doc students) forget this basic tenet of a valid and reliable assessment.  If the students are mis-interpreting the question or prompt, you will get a range of answers that do not coincide with the content you were trying to target, and many upset students who DID know the material you were trying to target when they see their test and hear your explanation of the question.  

- There are tons of resources online if you google "creating good questions for tests" or something along those lines.  Look for journal articles FROM THE EDUCATIONAL RESEARCHERS (not your content area) about standardized assessment.  I cannot emphasize that last part enough.  You go to your content folks for clarification on the material; you go to educators for clarification on effectively assessing the acquisition of knowledge by your students. 

 

5) Regarding choice selections: It is good to use 4 choices.  Avoid a test full of "all of the above" or "none of the above" as choice E.  Use when necessary, but overuse is confusing and inappropriate.  Additionally, you will want to use a "distractor" answer item -- one that if the student does not fully understand the concept, they might choose.  HOWEVER -- don't let the distractor item be too close or too likely.  You might think that you're giving the answer away, but that's because you know the content much better than those you are teaching.  Some will inevitably choose the distractor.  The other two choices might pertain to one or two parts of your question and would be there as choices for students who don't accurately read through the test question OR those who just haven't studied that concept.  They need to be semi-likely but could be the "opposite" answer (if there is a dichotomy of answers) OR a "not quite fully wrong" answer.  Above all -- be sure that the correct answer is truly correct and does not reference terminology that you haven't taught. 

 

- Be careful about knowing what you've taught --- what you mentioned in one sentence in a lecture 5 weeks ago is not what you taught.  What you scaffolded in instruction, built upon as a foundational concept, and was repeated at least twice is what you taught.  So many "professors" who haven't studied education need to be aware that picking some random sentence off the powerpoint is NOT the way to create a test.  If you're testing it, the question / prompt / item should be essential knowledge for your content area and should be built upon in future classes.  

 

6) If this applies to you or anyone else:  If English is NOT your first language (not making any assumptions here, just generalizing): Get a second pair of eyes on your questions from someone else in your content area to see if your use of the language is working to target the goal.  If in a TA position, your professor / mentor may be a good resource, but be careful that they're not just supporting you and not really thinking through the question.  I have had TA's with ESOL and they struggle to explain concepts and struggle even more after they get dozens of emails about the quiz they put up on blackboard -- the question written, in 90% of those cases, did not match up with what they were intending to ask.  

 

 

Honestly, the best tip for TEACHING (and assessing) is the following: "Begin with the end in mind."  If you don't know what you're going to assess, then how do you know what you're going to teach?  This is just a self-analytical idea -- not accusatory of you or anyone else.  Look back through and think about what the "take home" messages are from your classes.  Did you effectively teach them?  Is it worthy of assessing on a M/C test, or should it be assessed in some other way (such as a short essay, project, whatever).  If you didn't effectively teach them, (1) the answer choice distribution will be variable and not normalized; (2) you will have "smarties" who struggle unnecessarily.  

 

 

There is an art to creating a good M/C test.  Knowing what you intend to assess BEFORE you start teaching is a key place to start.  Hopefully you have been doing that all along.  

 

 

Good luck! 

[Beals]

CatSLP,

 

You are the greatest.  Thank you so much for all of the detail!!