Asking an exam question that requires a specific technique

This question Inadmissible theorems in research mentioned a scenario in which a student was not allowed to use a theorem that he knew because it wasn't in the course. I've administered exams in which the instructor said "using X technique, solve for Y", and grades of 0 were assigned when a different, admittedly simpler, technique was used. Students seemed to consider the question to be giving a hint, rather than a requirement.

Students often get angry about this, since they feel they got the right answer. The instructor's goal, though, was to get the students to prove that they could use the particular technique, and therefore the answer itself was not relevant.

To complicate matters, there are often times in which not using technique X results in a simpler method for the particular question. This is usually because X is a more general method and is only really powerful when the problem exceeds what is reasonable on an exam. A student could reasonably walk away from the exam thinking that X is just "more complicated" and not worth the trouble.

How can you ask a question like that that tests their ability to use a particular technique in a way that gets them to understand the purpose of the question? Especially, how can such a question be asked to avoid having students erroneously discount the importance of technique X because a simpler method exists for the particular problem?

10 Answers
10

You say, "Solve for Y. You must use X technique to demonstrate that you know how to use it, even though there may be simpler other ways to solve this particular problem."

Edit: One should cover in class why the students need to know X. The point in the exam question is to make it clear that using X is a requirement of the question, not a hint.

Example: "Use a Karnaugh map to simplify the following Boolean expression." Here, the question asks, "use a Karnaugh map," not "simplify the expression." The latter is the mechanism to demonstrate the former.

I second Bob Brown's suggestion of writing explicitly in the question that X technique must be used.

As to the concern that they will get a mistaken impression as to the usefulness of X: don't rely on the exam to convey this idea! My philosophy is that an exam is just to assess the student's knowledge and understanding. I don't find that it works well to assign an "interesting" exam problem from which the student is supposed to learn something new. An exam setting is too stressful to be a good time to acquire new knowledge.

Presumably there are examples that illustrate the full power of technique X, where simpler techniques don't work well. You can still have the students work those examples; just not on an exam. Maybe as a homework assignment, or a term project, or whatever. Hopefully by the end of the course, they've done enough of them that one "toy" example on the exam is not going to distort their impression of the technique's power.

(If your educational system uses exams as the sole graded element of the class, so that you can't realistically assign projects and such - my condolences. I don't have any brilliant ideas in that case.)

Especially, how can such a question be asked to avoid having students erroneously discount the importance of technique X because a simpler method exists for the particular problem?

I think you're looking at it the wrong way. If you want students to use a particular technique to solve a problem, then I think it should be your responsibility to design a problem where this is the best solution. Asking your students to use a (self-admitted) suboptimal technique to solve a problem is convoluted and goes against any problem-solving skill they might have developed.

Also, if you want your students to understand the importance of technique X, then you must show them problems where it really is the best solution.

There are good answers already, but here is a another solution. It may not be the best solution and it may be not practical in some situations, but it may still be a solution. Ask to solve the same problem with two different methods, i.e. write the problem as

(x+y points) Solve the problem A

a) with method X (x points)

b) with method Y (y points).

(Still somebody may come up with method Z, but my guess would be that this formulation will make it more clear that you require the use of specific methods. Also it should be simpler to argue that the problem statement was clear if a student with method Z comes complaining.)

While, as others have already pointed out, it is important to make the criteria of a question explicit (e.g. words like "must" or "required" are a good bet), I believe that there's a bigger underlying problem, namely a disconnect in expectations, that cannot be solved by wording alone.

The task "demonstrate knowledge of a specific technique", where the technique is not an optimal fit for the problem, is quite uncommon outside of exam-like situations. Many students will not have encountered this kind of question very often. It is also counterintuitive to anyone already used to working in the problem domain, as they are usually more aware of what method is appropriate under specific circumstances. *

In addition, outside of trivial problems, there's always more than one way to use a tool, and whether or not two approaches are considered equivalent may depend on the level of abstraction (and experience). Personally, as a student with extensive prior knowledge across different fields, I used to have a hard time judging whether an instructor was

More often than not, I ended up answering the question exactly as stated only to find there were subtle differences in interpretation, resulting in anything from zero to full (sometimes +bonus) points.

Add time constraints, ambiguous wording and the potential for errors or "trick" questions into the mix, and it's not surprising that a student might ignore what they perceive as a hint and solve the problem in the most efficient way they know.

While there may be good reasons to design an exam around a "suboptimal" approach, you should not expect a student to be automatically familiar with your course's context, philosophy and didactic goals.

More specifically, don't expect students to know what piece of knowledge you're trying to assess. A question defined in, say, two or three sentences is bound to infer a lot of context, and accurately limiting the scope is a difficult task.

If you give a list of techniques that are not allowed, expect at least one student to come up with something you didn't even consider. If you teach a particular variant of a method, or a particular notation, expect at least one student to have learned the method from a different source and do it that way.

Do make your question as clear as possible, but don't expect phrasing alone to solve the issue entirely.

Consider publishing a short guideline on question intent and scope that addresses frequent misunderstandings. Include pointers on notation and allowed axioms. Make it general or exam specific, as needed. The point is to move useful "boilerplate" info and caveats out of the text of each question, so questions can remain concise, but will be less ambiguous.

When possible for the question, give students some way to quickly assess whether they're "on the right track".
E.g. "A correct answer will have the following form/characteristics..."

*While restrictions like this can sometimes be encountered in research, as previously commented in the "Inadmissible theorems" question, these tend to be self-imposed and can be traced to a specific goal.

If the exam question states to solve a problem using a particular method, then a mark of 0 for not using that method is fair. The question is to assess knowledge. The marker cannot assess whether a student knows the stated method if they do not apply it. The student cannot (should not) arrive at the answer a different way and still expect full marks.

An exam asks to solve a problem using a particular method.

Use the stated method and get the right answer: full marks.

Use the stated method but make a mistake and get the wrong answer (by propagating the error through): partial marks.

Use a method other than the stated method: 0 marks.

Some students also think that if they solve a problem by:

Problem

--- a bit of working ---

--- some magic happens here ---

--- a bit of working ---
Right answer

that they should get full marks.

If you want to keep it simple, use the word "only"

"using only the X technique, solve for Y"

As others have said, you just need to be clear as to what they use. If you chose to spell out why then that is at your discretion.

If you say use only the x technique, and they don't, then there can be no argument that the answer did not address the question, whether or not the answer is correct.

Personally, I would put a little more explanation in there too to set context though, something like,

"due to the parameters of Z which limit ABC, using only the X technique, solve for Y"

Here is one way to scaffold the problem:

a. Describe technique X.

b. Use technique X to solve problem Y.

So students can earn credit knowing the technique in general, and they get a hint that they are expected to use it in this problem.

There are two options to address the issue here. One already mentioned is to instruct the students not to use other tools than required.

If the forbidden theorem applies only for a limited set of cases while the requested one applies for broader set you can build the test so the workaround gets tricky, hard or impossible to use.

You can also request them to prove the chosen technique can be used and why it is better than the requested one. Correct answer shows the student's knowledge of both techniques.

Sometimes, setting the stage can help.

If your concerned that students won't follow the directions correctly, you can always give the old classic quiz. The one that starts:

Read all directions before starting the quiz.

Then, in the directions, tell them:

Make no marks on the paper besides your name.

So, anyone who does anything beyond putting their name on the paper fails the quiz. Make the grade trivial (or, give a small amount of extra credit to those who pass and take no points from those who fail).

The point would be that the directions on your quizzes and tests are important.

It may be overkill, but it might actually get the point across.

Thank you for your interest in this question.
Because it has attracted low-quality or spam answers that had to be removed, posting an answer now requires 10 reputation on this site (the association bonus does not count).



Would you like to answer one of these unanswered questions instead?