Optimal Test Design for Knowledge-based Screening

A principal evaluates an agent based on his knowledge of a subject matter, modeled as a binary state, using pass-fail tests. I show that the optimal test takes a "pick the correct answer" form, with at most three options, regardless of how the principal values the agent's information. In the natural case where she prefers greater precision, such a test would reward only correctness, if the agent's knowledge were observable. In contrast, screening leads to under-rewarding of "obvious" answers and over-rewarding of "counterintuitive" ones, when there is such an ex-ante obvious answer. When the principal can pick the question, however, she picks one with no ex-ante obvious answer, so that the optimal test, once again, rewards only correctness. In particular, in this case, regardless of the specifics of the setting, the optimal test is the familiar True-False test which passes the agent if and only if he is correct