Administering IPIP Measures, with a 50-item Sample Questionnaire

These are suggestions, not requirements

The concept of "standardized testing" stresses the importance of administering psychological assessment measures exactly the same way each time they are used. Instructions are to be identical; the order of items is to be identical; options for responding are to be identical, and so forth. Potential IPIP users are therefore sometimes surprised to hear that there is no standardized procedure for administering IPIP items. There are three reasons for this.

First, sets of IPIP items have been administered to the Eugene-Springfield Community Sample (ESCS) over many years. The sample responded to an 858-item set in the spring of 1994, a 275-item set in the fall of 1995, a 179-item set in the fall of 1996, a 160-item set in the summer of 1998, and again on other administrations. It is unlikely that current IPIP users will similarly administer the same sets of IPIP items over many years. Furthermore, this means that the various IPIP scales and inventories that were constructed to represent existing scales and inventories do not have a specified item order.

Second, because the IPIP is in the public domain, users have complete freedom to use the IPIP in any way that suits their purposes. They can administer the items on the Internet or in paper-and-pencil format. They can use instructions that best helps them accomplish their research goals. They can administer the items in any order, mixing the items with non-IPIP items if they wish. They can edit the wording of items and translate them into other languages. They can shorten existing IPIP scales or combine similar IPIP scales into a longer scale. They can have respondents use a binary true/false or agree/disagree format or rating scales with as many anchor points as they wish, with anchor descriptions of their choosing. In short, users have the freedom to do what they like with IPIP items.

Finally, slight variations in administrative procedures do not have profound effects on substantive research results. That is, the order in which items are presented generally does not matter very much. Whether one provides respondents with three or five or seven response options does not matter very much. Therefore, one can choose just about any presentational format for IPIP items, safe in the knowledge that the format will have little impact on the reliability and validity of measurement.

If you are not sure where to start, however, you can refer to the sample questionnaire below. The formatting in this sample questionnaire, which was placed on the IPIP website early on in response to requests for formatting guidance, has been used by researchers in dozens of published studies. Using this format will therefore make your research findings comparable to existing studies. For example, if you use five response options scored 1-5, your means and variances can be compared directly with means and variances in published studies. Also, the formatting in the sample questionnaire illustrates several sound principles for administering questionnaires, described below.

If you browse to any IPIP scoring key, you will find that items that are scored on a scale are listed together and grouped into "+keyed" and "-keyed" items. (See the Scoring Instructions page for more information on +keyed and -keyed items.) For example, the top of the scoring key for the 50-item sample questionnaire lists five +keyed and five -keyed items for Factor I, Extraversion. If you scroll down you will find six +keyed and four -keyed items for Factor II, Agreeableness, and so forth. But if you look at the sample questionnaire below, you will find that the extraversion, agreeableness, and other items are not listed together but are distributed throughout the questionnaire. Also, the questionnaire alternates between +keyed and -keyed items. Why is this?

The theory behind distributing items from different scales throughout an inventory and alternating between +keyed and -keyed items is that this encourages respondents to pay close attention to the content of items and therefore increases the probability of valid responding. If items from the same scale are listed together, especially if they are all +keyed or -keyed items, respondents may be more likely to see that the items are measuring the same construct. If this happens, instead of giving more differentiated ratings (from Very Inaccurate through Very Accurate), they might tend to give the exact same response (e.g., Very Accurate) to all of the items. If many respondents in a sample answer this way, the overall variance of scores will be smaller than the variance if items from different scales are distributed throughout the questionnaire. And lower variance in a set of scores will underestimate the magnitude of relations with other variables in the study.

The principle of distributing items from different scales throughout an inventory and alternating between +keyed and -keyed items is not an absolute necessity. Some studies might employ only one short personality scale wherein most of the items are either +keyed or -keyed. In that case, it is not possible to alternate +keyed and -keyed items from different scales, yet it is still possible to get reliable and valid scores from such a scale. But when one does have an opportunity to alternate +keyed and -keyed items from different scales, one should probably employ this strategy.

Below is what the 50-item IPIP version of the Big Five Markers looks like when formatted according to the above suggestions. Note that the numbers in parentheses next to each item indicate whether it is +keyed or -keyed and which of the five scales it is scored on; this column of numbers would not appear on the Web page or paper questionnaire that is presented to respondents. So how does one get from a list of items that are grouped into +keyed and -keyed clusters in the IPIP scoring keys to a nicely formatted Web page or paper questionnaire such as the one below? One must copy the items from the IPIP website, paste them into a word processor or Web page editor, and arrange them with editing tools. For further information on how to present questionnaires on the Internet, an excellent resource is Chris Fraley's book.

How Accurately Can You Describe Yourself?

Return Home