The Federal Selection Interview: Unrealized Potential

The Federal Selection Interview:

Unrealized Potential

A report to the President and the Congress

of the United States by the

U.S. Merit Systems Protection Board

this page intentionally left blank

U.S. Merit Systems Protection Board

Susanne T. Marshall, Chairman

Beth S. Slavet, Member

Office of Policy and Evaluation

Director

Steve Nelson

Deputy Director

John Crum, Ph.D.

Project Manager

James J. Tsugawa

Project Analyst

Bruce Mayor

this page intentionally left blank

A Report by The U.S. Merit Systems Protection Board vii

Table of Contents

Executive Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1

Background . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5

The Business Case for Effective Interviews . . . . . . . . . . . . . . . . . . . . 9

Elements of a Structured Interview. . . . . . . . . . . . . . . . . . . . . . . . . 13

The Benefits of the Structured Interview . . . . . . . . . . . . . . . . . . . . 19

The Case Against the Unstructured Interview . . . . . . . . . . . . . . . . 21

The Structured Interview Process. . . . . . . . . . . . . . . . . . . . . . . . . . 23

The State of Federal Employment Interviewing . . . . . . . . . . . . . . . 27

Recommendations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33

Appendix A — Types of Employment Interview . . . . . . . . . . . . . . 37

Appendix B—A Brief Discussion of Selection Tool Validity . . . . . 39

Appendix C—Sample Structured Interview Question and Rating

Scale . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41

Appendix D—A Model Structured Interview Process . . . . . . . . . . 43

this page intentionally left blank

A Report by The U.S. Merit Systems Protection Board 1

Executive Summary

Interviews that are used to make personnel decisions—such as hiring and

promotion—are assessments, just like written tests and ratings of training

and experience.

Data from the Board’s Merit Principles Survey

indicate

that the use of interviews as an assessment tool is widespread and that

managers rely heavily on interviews when making selection decisions.

Therefore, it is important that managers conduct good interviews that pro-

duce reliable results. The report presents the concept of the structured inter-

view, describes the elements and benefits of structured interviews, discusses

the disadvantages of unstructured interviews, and outlines a process for

using structured interviews.

This report examines the selection interview,

an interview whose pri-

mary purpose is to serve as an assessment tool; that is, a tool for evalu-

ating a job candidate’s qualifications and abilities. Selection interviews

may be used to rank candidates (that is, place them in order or group

them based on interview performance), or simply to inform the select-

ing official’s decision when choosing from a group of available, quali-

fied candidates. Selection interviews are distinct from informational

interviews, which are generally used to educate potential candidates

about the organization and its employment opportunities.

The report has two primary objectives: (1) to discuss how agencies use

selection interviews, and (2) to identify and promote interviewing

practices that support merit-based selection. The U.S. Merit Systems

Protection Board (the Board) prepared this report pursuant to statu-

tory responsibility to evaluate the Federal civil service and other execu-

tive branch merit systems.

Rating of training and experience is most frequently done through review of a written application, but

other methods are possible. For example, Service Centers of the U.S. Office of Personnel Management

(OPM) frequently use an automated questionnaire to evaluate applicants’ qualifications and experience.

The Merit Principles Survey is a Governmentwide survey that MSPB administered in the spring of 2000

to over 17,000 Federal employees. Questions 77A and 77B of the survey asked supervisors about sources

of information that they use, including interviews, when selecting entry-level professional and administra-

tive employees.

The term “selection interview” is sometimes used to refer to unscored interviews that are used to choose

from among a group of candidates. In this report, the term “final selection interview” covers this type of

interview, and the term “selection interview” refers to

any

interview that influences the selection decision.

2 The Federal Selection Interview: Unrealized Potential

Background

Selection interviews cover a continuum of technique and content. At

one end is the unstructured interview—a casual conversation where

the questions asked may be unplanned and vary across interviews, and

the results are analyzed and applied subjectively. At the other end is

the highly structured interview, where trained interviewers ask ques-

tions based on job analysis, ask the same questions of each applicant,

and score answers using predeveloped rating scales.

Selection interviewing is a nearly universal practice in both the public

and private sectors. Over two-thirds of the Federal managers respond-

ing to the Board’s 2000 Merit Principles Survey stated that they con-

sider interviews to a great extent when selecting a new employee.

Despite its prevalence, interviewing in the Federal Government is not

highly regimented

—at least on the surface. Federal employment

laws, regulations, and guidelines emphasize assessments such as writ-

ten tests, level of education, and evaluation of training and experience,

but appear to regard the interview as an afterthought. Nevertheless,

interviews must meet the same broad standards as these other assess-

ments: they are covered by the Uniform Guidelines on Employee

Selection and Placement,

and should, as part of a merit-based system

of employment, support “selection and advancement…determined

solely on the basis of relative ability.”

Findings

Federal managers do not merely conduct employment interviews: they

rely heavily on them to identify the best candidate. This reliance is

appropriate, if managers use a properly developed structured inter-

view. Structured interviews are interviews that use multiple mecha-

nisms, such as questions based on job analysis, detailed rating scales,

and trained interviewers to make the interview more job-related and

systematic. Research shows that such interviews can increase the odds

of selecting good employees.

However, information from the Board’s previous studies and surveys

suggests Federal managers are not realizing the full potential of the

selection interview. Agency investment in assessment tools such as

interviews varies substantially, including varying levels of job analysis,

interviewer preparation, and rigor in the development, conduct, and

There are exceptions to this generalization. For example, many agencies have policies that specify the

use and content of interviews for selected occupations, and collective bargaining agreements may include

provisions describing how interviews will be conducted and scored.

The Uniform Guidelines on Employee Selection and Placement (Uniform Guidelines) are a set of prin-

ciples and standards for employment practices. The Uniform Guidelines were developed by the Equal

Employment Opportunity Commission, the Department of Labor, the Department of Justice, and the

Civil Service Commission (now the Office of Personnel Management). They provide a framework to help

employers make proper use of tests and other selection procedures, meet legal and regulatory require-

ments, and comply with antidiscrimination laws.

Merit Principle No. 1 [5 U.S.C. 2301(b)(1)].

A Report by The U.S. Merit Systems Protection Board 3

use of interviews. Consequently, structured interviewing appears to be

the exception rather than the norm, and Federal managers often resort

to (relatively) unstructured interviewing because their only other

option—which they are understandably unwilling to accept—is not

interviewing at all. Unfortunately, unstructured interviewing is prob-

lematic. Unstructured interviews are less valid than structured inter-

views, are subject to bias, and may expose employers to complaints

and challenges.

Recommendations

In light of Federal agencies’ widespread use of the interview—and its

potential to contribute to high-quality selections when properly devel-

oped and conducted—we recommend that agencies take steps to

optimize their use of the interview.

1. Agencies should decide which purpose(s) the interview will serve,

and design and conduct the interview accordingly. Interviews can

be purely informational, supporting recruitment efforts by educating

candidates about employment opportunities and conditions of

employment. Or, interviews can focus purely on assessing candidates’

qualifications. Or they can do both. Agencies must make a conscious

and informed choice, because that choice has significant implications

for the design and conduct of the interview. Interviews that function

only as a recruitment tool do not require a high level of structure.

the other hand, interviews that function as assessment tools—that are

used to screen candidates, rank or group candidates, or make a final

selection decision—call for careful design and considerable structure,

if the organization is to realize their full potential.

2. Agencies that use interviews to assess job candidates should use

structured interviews. In making this recommendation, we distin-

guish between interviews that function as a continuation of the evalu-

ation of training and experience—for example, an interview that

simply confirms or elaborates on information in the candidate’s

application—and interviews that are intended to develop new infor-

mation. For the former, a high level of structure is neither practical

nor necessary, provided that the interviewer asks job-related questions

and treats candidates consistently. But for the latter, the case for the

structured interview is compelling. Research shows that structured

interviews, as part of a systematic candidate assessment process, can

increase the likelihood of a good selection by helping managers

develop new information on candidates such as past behaviors, in the

context of the workplace. Structured interviews can also reduce the

costs associated with unsound employment practices, including turn-

Even here, some structure—such as recruiter training and “talking points”—will help the organization

project a clear, consistent image and maximize the effectiveness of the informational interview.

4 The Federal Selection Interview: Unrealized Potential

over, poor performance, and grievances and complaints. The

alternative—the unstructured interview—is much less desirable.

Research indicates that unstructured interviews are, on average, little

more than half as effective as structured interviews and unstructured

interviews may be subject to bias and challenges.

3. Agencies should invest the resources (time, training, funds, and

expertise) needed to add structure to selection interviews. Struc-

tured interviewing can be cost-effective, but it is not free. Structured

interviews require a coordinated application of thought and expertise.

An agency cannot realize the benefits of structured interviewing sim-

ply by reading about it. Many Federal agencies understand and have

acted on the “business case” for structured interviews. However, our

previous studies also indicate that many agencies and managers do

not, for varying reasons, use the best available tools when assessing

candidates. This failure is not merely inconsistent with the merit prin-

ciple of selecting from the best-qualified candidates; it is also inconsis-

tent with effective and efficient government. Therefore, we strongly

encourage agencies that use selection interviews to move toward struc-

tured interviews. This includes investing in both the assessment tool

(the interview itself) and the users of that tool (most likely supervisors

and managers).

Agencies that may lack the resources to make large, immediate invest-

ments in structured interviewing should pursue incremental improve-

ments in their interviewing practices. The body of the report discusses

this strategy in more detail, and provides some examples.

Agencies that wish to adopt structured interviewing may contact the

Office of Personnel Management, which offers guidance and training

on structured interviewing. Agencies may also be able to draw on the

expertise and experience of agencies that currently use structured

interviews. These include, but are not limited to, the U.S. Customs

Service, the U.S. Immigration and Naturalization Service, and the

U.S. Department of Veterans Affairs.

4. Agencies should evaluate their interview practices for effective-

ness and possible improvement. This recommendation, like the first

two, reiterates an established position of the Board that is particularly

relevant to the structured interview. The structured interview is an

adaptable and flexible assessment tool that can be readily modified to

accommodate changing job requirements and incorporate “lessons

learned.” Evaluation of the interview instrument, process, and out-

comes will help ensure that the interview meets the organization’s

needs, and that the organization treats candidates fairly and defensibly.

A Report by The U.S. Merit Systems Protection Board 5

Background

There are two major types of employment-related interviews: informa-

tional and selection. Informational interviews provide information to

prospective and actual job candidates. The information provided may

cover matters such as application procedures, employment opportuni-

ties, and working conditions and benefits, and typically supports the

recruitment effort. In contrast, selection interviews collect informa-

tion from job candidates. The information collected may cover educa-

tion and credentials, work experience, and accomplishments, and is

used to help the organization make a selection decision. Appendix A

illustrates this distinction, and outlines the various roles the interview

can play in the selection process. The two types can be combined; in

fact, most selection interviews include an informational component.

In this report, the term “interview” refers to an interview, either struc-

tured or unstructured, used at any stage of the selection process,

whether screening, ranking, or final selection.

Selection interviews are

not all alike.

Selection interviews may be placed on a continuum from unstructured

to structured. At one end of the continuum, the unstructured inter-

view is completely unplanned—questions are asked spontaneously

and responses are not evaluated in any systematic manner. At the other

end of the continuum, a highly structured interview uses several ele-

ments of structure—mechanisms such as questions based on job anal-

ysis, and predeveloped rating scales—to create a clear and strong

relationship between performance in the interview and performance

on the job. Table 1 provides key characteristics of unstructured and

structured interviews:

6 The Federal Selection Interview: Unrealized Potential

Interviews are widely used.

The Federal Government has over 1.6 million civilian employees, in

over 800 different occupations.

To the informed observer, it may

seem that there are as many paths to employment as there are employ-

ees. These include—to name only a few—competitive examination,

intern programs, student employment programs, and veterans

employment programs. Yet almost all paths to Federal employment

share one element in common. That element is the selection interview,

a meeting between employer and applicant during which each assesses

the other to determine the possibility and desirability of an employ-

ment relationship.

Data confirm that interviews are a standard part of the Government

selection process. The Board’s 2000 Merit Principles Survey asked

managers about the information they consider when assessing candi-

dates for professional and administrative positions. Responses showed

that the interview is a near-universal source of candidate information

(see Fig. 1).

Table 1. Characteristics of Unstructured and Structured

Interviews

Unstructured Structured

• The factors evaluated by the interview

are implicit, and vary across candi-

dates.

• The factors evaluated are explicit,

based on job analysis, and are the

same for each candidate.

• Questions are not necessarily job-

related.

• Questions are job-related.

• Questions vary from interview to

interview for the same job.

• The same questions are asked of all

candidates for the same job.

• There is no system or guide for evalu-

ating interview results.

• There is a predeveloped system for

evaluating interview results.

• Interviewers may be untrained. • Interviewers have been trained.

Data are from the U.S. Office of Personnel Management, Central Personnel Data File, as reported in

Fedscope (www.fedscope.opm.gov), March 2001. Some 1,641,115 full-time employees are in 814 occu-

pations, including all appointments and pay plans.

A Report by The U.S. Merit Systems Protection Board 7

Interviews are influential.

Table 2 on the following page provides a simplified model

of the typical Federal selection process. Most competitive hiring—

whether delegated examining to fill a vacancy with an outside candi-

date, or merit promotion to advance a current employee—follows this

model. As shown, the selection process has several steps. Because Fed-

eral agencies generally use interviews as the final major hurdle in the

selection process, interviews carry considerable influence in the selec-

tion decision.

Because selection interviews are widely used and highly influential, it’s

important that they be used effectively. In the following section, we

discuss the organizational and fiscal implications of interviews.

Figure 1. Percent of supervisors using information to a “great” or

“moderate” extent

Source: U.S. Merit Sytems Protection Board, 2000 Merit Principles Survey.

10%

15%

20%

34%

36%

38%

39%

69%

71%

19%

28%

37%

28%

41%

44%

46%

26%

25%

W ritten test

Reputation of

e d uc a tio na l institutio n

C o lle ge G P A

Personal

recommendations

Major field of study

Reference checks

Le vel o f e d uc a tio n

Q ua lity o f the

a p p lica tio n

Interviews

Prior work experience

Great Extent Moderate Extent

Source of Information Percent

8 The Federal Selection Interview: Unrealized Potential

Table 2. Typical Steps in the Federal Selection Process

Step 1 Recruitment

—The agency develops a candidate pool and advertises the

vacancy. Candidates’ qualifications are not formally assessed at this stage.

Step 2 Screening

—This is the first hurdle in the selection process. The agency

sorts applicants into two groups: ineligible and eligible. Applicants who do

not meet basic requirements, such as citizenship and minimum

qualifications, are ineligible and thus “screened out.” Applicants meeting

basic requirements are eligible for further consideration. Common

assessments used to screen candidates include written tests and ratings of

training and experience (usually through review of a written application).

Step 3 Ranking

—This is the second hurdle in the selection process, in which

the agency makes quality distinctions among the eligible candidates. The

distinction may be simple sorting (such as assigning candidates into a

“qualified” or “best qualified” group) or a rank ordering of candidates.

The candidates in the highest group (or the top-ranked candidates) are

then referred to the selecting official. The assessment tool used to make

these distinction may be the same one used for screening, or it may be

different. Common assessments used to rank applicants include ratings of

training and experience (such as assigning scores to written descriptions

of desired knowledges, skills, and abilities), and interviews.

Step 4

Selection

—This is the final major hurdle in the selection process.

The

selecting official typically assesses some or all of the referred candidates

through interviews (frequently supplemented by reference checks) and

chooses an applicant from among the referred candidates.

a. There may be subsequent hurdles such as a background investigation, and verification of experience

and credentials. However, these hurdles are generally applied after the candidate has been selected.

A Report by The U.S. Merit Systems Protection Board 9

The Business Case for Effective

Interviews

As discussed in the preceding section, the interview is typically the last

major step in the selection process. Following the interview, the man-

ager will decide which applicant(s) will receive an offer of employ-

ment. The stakes are high: a good decision will confer substantial

benefits on the organization; a poor decision may impose significant

costs.

The benefits of a good selection are straightforward. A good employee

will be capable and productive, and work well with customers and col-

leagues. This outcome, in turn, improves morale and work group

cohesion, increasing the employing organization’s productivity. These

benefits are not merely theoretical: they make a measurable contribu-

tion to the organization’s bottom line. A recent study by the consult-

ing firm Watson Wyatt Worldwide suggests that identifying and

selecting highly qualified candidates is not only consistent with merit

principles—it is also good business. The company surveyed the

human capital practices of over 400 private-sector companies, and

found—not surprisingly—that companies that hired workers well-

equipped to perform their duties created more value for their share-

holders than less-demanding companies.

On the other hand, organizations that make poor selection decisions

will forgo these benefits and bear substantial additional costs. As

shown in Table 3, one private sector organization estimates that a poor

selection can cost as much as three times the employee’s annual salary.

Previous Board studies document additional costs that are borne by

Federal agencies that make a poor selection decision. Federal agencies

do not have the option of simply terminating a poor performer and

“starting over,” unless the employee is serving a trial or probationary

period, and the available alternatives are costly. If the agency chooses

to pursue an adverse action (i.e., demotion or termination), it must

Survey report from “The Human Capital Index: Linking Human Capital and Shareholder Value,” Wat-

son Wyatt Worldwide, 1999, p. 5.

10 The Federal Selection Interview: Unrealized Potential

devote substantial management attention and staff time to that effort.

The remaining options—remediation or inaction—are also costly.

Remediation entails a potentially extended period of reduced produc-

tivity: the organization must devote resources to training and counsel-

ing the employee, and have coworkers (or managers) perform the

employee’s work or make it acceptable. Inaction means that the agency

will bear the costs of a poor selection indefinitely, and may further

reduce the productivity of the work unit:

* * * inaction can create problems far beyond that of a single incompe-

tent worker. It can turn the unit’s better performers into overworked,

resentful employees who, noticing the absence of penalties for inferior

performance, may reduce their own efforts as a result.

Good selections require

good assessments, and

interviews can be excellent

assessments.

Agencies can significantly improve the likelihood of selecting good

employees by using assessment tools with high validity. Validity is the

ability of an assessment tool to predict on-the-job performance.

(Appendix B provides a fuller discussion of the concept of validity.)

The structured interview is among the most valid assessment tools

available, comparing favorably with mental ability tests and work sam-

ple tests, and surpassing such assessment tools as years of experience,

ratings of training and experience, and reference checks.

Interviews are not all

created equal.

Structured interviews use a variety of mechanisms (which we refer to

as elements of structure) to help the interviewer constructively differ-

entiate among job candidates. Research shows that structured inter-

views have a significant edge over unstructured interviews in

predicting on-the-job performance:

Table 3. Estimated costs of hiring the wrong person

For an entry-level full-time employee $5,000 to $7,000

For a $20,000/year FTE

$40,000

For a $100,000/year FTE $300,000

Note: Costs include wasted salary, benefits, severance pay, headhunter fees, training costs,

and hiring time.

Source: Corporate Leadership Council, Literature Review, “Employee Selection Tests,” Cat-

alog No. 070-198-213, Washington, DC, March 1998, p. 2.

a. FTE stands for full-time equivalent, which means 2087 hours of duty per year.

U.S. Merit Systems Protection Board, Federal Supervisors and Poor Performers, Washington, DC, July

1999, p. 15.

Frank L. Schmidt and John E. Hunter, “The Validity and Utility of Selection Methods in Personnel

Psychology: Practical and Theoretical Implications of 85 Years of Research Findings,” Psychological Bulle-

tin, The American Psychological Association, Inc., vol. 24, No. 2, September 1998, p. 265.

A Report by The U.S. Merit Systems Protection Board 11

The average validity of the structured interview is 0.51, vs. 0.38 for the

unstructured interview (and undoubtedly lower for carelessly con-

ducted unstructured interviews).

In practical terms, this means that the structured interview is twice as

effective as an unstructured interview—and, as suggested above, this

is a conservative estimate.

Moreover, research suggests that interview-

ing without the assistance of structure—interviewers relying on

unaided judgment—is ill-advised. Studies have consistently found

that interviewers are inappropriately influenced by factors such as the

performance of previous candidates

and personal feelings.

Inter-

viewers are also vulnerable to universal biases in human cognition,

such as overreliance on first impressions and the tendency to view a

person’s behavior in one situation (such as a social gathering) as indic-

ative of how that person will behave in other, dissimilar situations

(such as at work).

Structure provides a necessary counterweight to problematic influ-

ences and to failings in human judgment. Without this counter-

weight, interviewers will likely have great difficulty accurately

assessing candidates’ actual abilities and performance.

In the follow-

ing section, we discuss common elements of a structured interview,

and how those elements contribute to effective interviewing.

Ibid. p. 267.

“Unstructured interviews” conducted for research purposes almost certainly contain some elements of

structure.

An interviewer who has just interviewed an excellent candidate may rate the next candidate inappro-

priately low. Conversely, an interviewer who has interviewed a poor candidate may rate the next one inap-

propriately high. This rating error is known as “contrast effect.”

Richard D. Arvey and James E. Campion, “The Employment Interview: A Summary and Review of

Recent Research,” Personnel Psychology, vol. 35, 1982, p. 297.

In psychology, this tendency is called the fundamental attribution error. Research shows that people

consistently overestimate the role of personality and underestimate the role of context in determining

behavior. For a non-technical discussion of this phenomenon and its effects on judgments of aptitude and

performance, see Malcolm Gladwell, “The New-Boy Network,” The New Yorker, May 29, 2000.

Michael A McDaniel., Deborah L. Whetzel, Frank L. Schmidt, and Steven D. Maurer, “The Validity

of Employment Interviews: A Comprehensive Review and Meta-Analysis,” Journal of Applied Psychol-

ogy, vol. 79, No. 4, 1994, p. 599.

this page intentionally left blank

A Report by The U.S. Merit Systems Protection Board 13

Elements of a Structured Interview

Below, we discuss eight elements of structure.

These elements

address all stages of the interview process, from pre-interview, such as

job analysis and developing interview questions, to post-interview,

such as rating candidates’ responses. Taken together, the elements

form a “chain” that links the interview to the position being filled.

1. Base questions on job analysis. Job analysis is the process of look-

ing at a position (or, more broadly, the work of an organization) to

identify essential functions and duties, and the competencies, knowl-

edges, skills, and abilities needed to perform work.

A well-known executive recruiter puts the case for job analysis very

succinctly:

Once you know what the real performance needs of the job are, hiring

is relatively easy. When you don’t know what’s required, you substitute

your biases, perceptions, and stereotypes.

Questions based on a sound job analysis will, by definition, be job-

related. Job analysis can also identify characteristics that distinguish

excellent from average employees,

so that the interview questions

will be useful in identifying the best candidates.

Basing questions on job analysis is also an excellent way to avoid ask-

ing inappropriate questions, such as those involving marital and fam-

ily status. For example, for a job that requires face-to-face contact with

customers in several states, job analysis will steer the interviewer

toward asking whether the prospective employee can travel, and away

from asking whether the prospective employee has dependent care

responsibilities.

2. Ask effective questions. As we’ve indicated, effective interview

questions are based on job analysis to ensure that they are job-related.

The eight elements are adapted from Michael A. Campion, David K. Palmer, and James E. Campion,

“A Review of Structure in the Selection Interview,” Personnel Psychology, vol. 50, 1997.

Adapted from 5 CFR 300.103 (January 2001).

Lou Adler, “Hire With Your Head,” John Wiley & Sons, Inc., New York, NY, 1999, p. 5.

This is often accomplished by identifying and analyzing situations (referred to as “critical incidents”) in

which an employee demonstrated high performance.

14 The Federal Selection Interview: Unrealized Potential

Effective interview questions are also usually open-ended and behav-

ioral, so that they will elicit useful responses.

Open-ended questions are questions that require the candidate to pro-

vide details, and cannot be answered in one word (such as “yes” or

“excellent”). Such questions are much more effective than closed-

ended questions at developing insight into a candidate’s experience

and abilities. For example, the closed-ended question, “Can you write

effectively?” can be answered with an uninformative “Yes”—a

response that sheds little light on the candidate’s level of performance

in this area. An open-ended question such as, “Describe the types of

documents you have written, reviewed, or edited,” requires the candi-

date to provide specifics, and provides much more insight into the

candidate’s writing accomplishments.

There is a place for the closed-ended question. For example, to learn

whether a candidate is willing to travel frequently or can start work on

a given date, it is perfectly appropriate to ask a closed-ended question.

Behavioral questions are just that: questions that ask the candidate to

describe behaviors—responses, actions, and accomplishments in

actual situations. The case for the behavioral question is more subtle

than the case for open-ended questions. Although research indicates

that both behavioral questions (“What did you do?”) and hypothetical

questions (“What would you do?”) can be effective, many researchers

and practitioners generally recommend the behavioral question for

two reasons. First, behavioral questions can provide greater insight

into how the candidate will perform on the job, because the best pre-

dictor of future behavior is past behavior. Second, behavioral ques-

tions may be more reliable than hypothetical questions. Because the

response can be verified through reference checks or other means, it is

more difficult to fabricate an inaccurate or untruthful answer to a

behavioral question than to a hypothetical one.

However, hypothetical questions, like closed-ended questions, have

their uses. For example, hypothetical questions can ask a candidate to

respond to a work situation, to yield insight into the candidate’s ability

to reason, as well as his or her “soft” competencies such as flexibility

and cooperativeness. Such questions can also give the candidate a real-

istic “job preview.” Unfortunately, it is not easy to develop reliable

hypothetical questions

because the response to a hypothetical ques-

tion may not reflect what a candidate has actually done or will do on

the job. The problem is that knowledge and behavior are often very

Developing valid hypothetical questions typically involves extensive fact-finding and professional assis-

tance to develop questions and “benchmark” responses. A discussion of the process can be found in

“Developing and Conducting the Structured Situational Interview: A Practical Guide,” U.S. Office of

Personnel Management, Washington, DC, January 1994.

A Report by The U.S. Merit Systems Protection Board 15

different. For example, we all know that we should drive defensively

and exercise regularly—whether we do so is another matter entirely.

3. Ask each candidate the same questions. Using a uniform set of

questions helps ensure that candidates are treated fairly, and that each

candidate provides comparable information. As we note in the boxed

discussion, “How Uniform is Uniform?” this does not mean that

interviewers may never ask tailored (candidate-specific) questions to

develop more information about an individual candidate’s qualifica-

tions. But it does mean that the structured portion of the interview

should use a standardized set of questions. The goal is to eliminate

variation in the questions as a source of variation in interviewees’

answers. Although this approach may appear unduly mechanical, it

ensures that the interviewer does not skip questions, or rephrase them

based on potentially inaccurate and harmful assumptions about the

candidate’s level of knowledge, listening comprehension, or compe-

tence.

Using a uniform set of questions does not preclude repeating or restat-

ing questions, if necessary. However, if the interview questions are

effective—relevant, clear, and free of jargon and acronyms—this

should not be necessary.

4. Use detailed rating scales. A rating scale provides a guide for eval-

uating candidates’ responses. At their simplest, scales provide levels—

numbers, or adjectives such as unacceptable, acceptable, and excellent.

More detailed scales supplement the levels with anchors. Anchors typ-

How Uniform is Uniform?

Can an interview include questions tailored to individual candidates? The quick

answer is “yes.” Our report emphasizes the interview as an opportunity to

develop information across candidates, to even-handedly assess their ability in

specific competencies.

That approach presumes that the candidates come into the interview more or less

“equal”—that they have comparable experience and that the interviewer pos-

sesses a relatively uniform level of information about each candidate prior to the

interview. But this is often not the case. For example, some candidates may have

provided extensive descriptions of relevant training and experience, while others

submitted a one-page resume. Or, one candidate’s experience might be primarily

private-sector, while another candidate has no non-Federal experience.

In such circumstances, the interviewer may want to use the interview to “fill in

gaps” in a candidate’s training and experience—in effect, as an extension of the

rating of training and experience. This will often require candidate-specific ques-

tions. In our view, this is reasonable and acceptable.

Accordingly, we suggest that interviewers who want to collect information across

candidates and obtain further information on individual candidates’ training and

experience divide the interview into two phases: a structured phase, using a uni-

form set of questions, and a candidate-specific phase. This preserves the integrity

of the structured interview while giving the interviewer some flexibility.

16 The Federal Selection Interview: Unrealized Potential

ically include a narrative description of each level, and examples of

behavior and accomplishments that are representative of the level.

Rating scales are valuable for three reasons. First, they promote objec-

tivity. A predetermined standard guards against the tendency to evalu-

ate candidates based on impressions or against each other—both of

which amount to letting the candidate set the standard. Second, they

promote consistency across interviews and interviewers, a particularly

useful outcome if the same person(s) will not be conducting all inter-

views. Finally, detailed rating scales can incorporate behaviors of high-

performing employees, which helps the interviewer distinguish

between acceptable and excellent prospects. Appendix C provides a

sample question and detailed rating scale developed by the Office of

Personnel Management for use in interviewing candidates for infor-

mation technology positions.

5. Train interviewers. Training is needed to ensure that people

involved in developing and conducting structured interviews under-

stand and know how to apply the associated elements and techniques.

Thus, training commonly covers subjects such as:

• Identifying job requirements (job analysis),

• Balancing assessments and judgment,

• Developing interview questions,

• Establishing rapport with candidates,

• Effective questioning,

• Evaluating answers and applying rating scales,

• Avoiding common rating errors,

• Documentation (e.g., note-taking), and

• Making hiring decisions.

Although the list of subjects appears long, the essentials can be covered

in as little as one or two days. The illustration at right, “Interviewer

Training in the U.S. Customs Service,” shows how one Federal agency

prepares its managers to conduct good interviews.

6. Use interview panels so that more than one person conducts the

interview. Many organizations that use structured interviews have a

two- or three-person panel conduct the interview. Using additional

interviewers offers several benefits. First, the additional interviewer(s)

may ask questions and capture information that a single interviewer

might miss or overlook. Second, the additional interviewer can bring a

different and valuable perspective to the rating process, resulting in a

more balanced picture (rating) of the candidate. Finally, additional

Michael A. Campion, David K. Palmer, and James E. Campion, “A Review of Structure in the Selec-

tion Interview,” Personnel Psychology, Volume 50, 1997, p. 685.

A Report by The U.S. Merit Systems Protection Board 17

interviewers can bring demographic diversity to the interview panel,

which can make candidates more comfortable during the interview

and more accepting of the outcome of the interview.

7. Take notes. Notes serve two purposes. First, they help the inter-

viewer capture and recall the content of the interview, and rate the

candidate’s responses accurately. Human memory is imperfect and

selective. Without notes, the interviewer may tend to selectively recall

the candidate’s strengths or weaknesses, or be unable to recall the can-

didate’s responses at all. Second, notes help create an important “paper

trail” that is useful if it becomes necessary to reconstruct the interview

process or defend an employment decision based on the interview.

We suggest taking notes with these purposes in mind. With two

exceptions, notes should reflect what the interviewee says, rather than

how the interviewee says it. The first exception is when the interview

is used to assess a competency such as oral communication, where

both content and delivery are relevant. The second exception, which

should be rare, is when a candidate’s behavior raises questions about

his or her qualifications or suitability. For example, if the candidate

takes an inordinately long time to respond to questions or does not

interact appropriately with the interviewers, this should be noted.

8. Assess candidate responses objectively—use the rating scales,

and use the ratings to score candidates. “Assessing responses objec-

tively” does not mean that interviewers should mechanically review

candidate responses to determine whether the candidate spoke the

Interviewer Training in the

U.S. Customs Service

The United States Customs Service (USCS) uses a structured interview as part of

the selection process for the mission-critical occupations of Customs Inspector

and Special Agent. Interviews are conducted by three-person panels composed of

managers and senior staff. USCS views trained interviewers as essential: a man-

ager must receive training before serving on an interview panel.

The training, delivered over 2 or 3 days, blends theory and practice. Managers are

briefed on potential pitfalls, such as relying on first impressions and the “halo

effect” (allowing the candidate's performance in one competency to influence rat-

ings in another competency). Managers also receive practical advice, covering

topics such as putting the candidate at ease, asking follow-up questions, dealing

with candidates with apparent disabilities, and managing their own “body lan-

guage.”

The training provides many opportunities for “hands on” learning. For example,

managers view videotaped interviews, apply rating scales, and compare and dis-

cuss results. The training concludes with a role-playing exercise where managers

interview and rate “candidates” (played by other USCS employees). Attendees

have given the training excellent reviews, indicating that the it has greatly

enhanced their interviewing skills.

Ibid., p. 679.

18 The Federal Selection Interview: Unrealized Potential

“magic words,” or give undue weight to interview scores. It is appro-

priate for interviewers to discuss responses, resolve differences, and

apply judgment when applying rating scales.

Judgment also comes into play when using the interview results, as we

note in the boxed discussion, “Balancing Assessments and Judgment,”

above. The point is that interviewers should not substitute an intuitive

global judgment for their ratings.

Balancing Assessments and

Judgment

We suggest that agencies use interview results to inform—but not dictate - selec-

tion decisions. For example, suppose that a selecting official can choose between

five qualified candidates with scores of 65, 84, 86, 90, and 91 (on a 100-point

scale) on the interview. The selecting official would be well-advised to eliminate

the low-scoring candidate from consideration, even if that candidate made an

otherwise good impression. But the selecting official could probably make a

merit-based case for selecting any of the four highest scoring candidates.

Assessment tools such as interviews are not a substitute for managerial judgment.

First, minor differences in performance on the interview (or any other assess-

ment) are not necessarily significant—even the best assessments are imperfect.

Second, selecting officials can and should consider a full range of information

when making a selection decision. This information may include things such as

past performance, training and experience, test scores, academic and personal

achievement, and reference checks. Finally, assessment tools cannot measure and

weigh every relevant factor, such as the skills needs and demographic mix of the

employing organization.

A Report by The U.S. Merit Systems Protection Board 19

The Benefits of the Structured Interview

Why go to the time and trouble of adding structure to the interview?

Quite simply, because structured interviews work:

In the 80-year history of published research on employment interview-

ing * * *, few conclusions have been more widely supported than the

idea that structuring the interview enhances reliability and validity.

Structured interviews have several other advantages that contribute to

their effectiveness, advantages that are not merely theoretical. As dis-

cussed below, structured interviews are not only valid but provide

important practical benefits. These include:

Fairness and objectivity. Structured interviews use job-related ques-

tions, treat interviewees consistently, and assess interviewees’ responses

in a thorough, systematic manner. Structured interviews also focus on

the interviewees’ answers rather than on their behavior during the

interview. The result is that a good structured interview treats

candidates—and their responses—fairly and objectively, with little or

no adverse impact.

Professionalism. Structured interviews are businesslike; they focus

strictly on the candidate’s qualifications in relation to the job require-

ments. Such interviews can help “sell” the organization by conveying

the message that it is serious about evaluating candidates carefully and

selecting the best person for the job.

We recognize that “selling” the job also involves providing information

about the job and work environment, and showing a genuine interest

in the candidate. Structured interviewing does not preclude doing

these things. For example, an interview could begin with the inter-

viewer providing information about the job and the work setting, fol-

lowed by structured questions, and conclude with an opportunity for

the interviewee to ask questions. The interviewer simply needs to

ensure that providing information to the candidate does not over-

shadow collecting information from the candidate.

Ibid., p. 655.

20 The Federal Selection Interview: Unrealized Potential

Compliance. Laws and regulations set a high standard for Federal

employment practices. The statutory merit system principles state that

“selection and advancement should be determined solely on the basis

of relative ability, knowledge, and skills, after fair and open competi-

tion which assures that all receive equal opportunity.”

To achieve this

objective, good intentions are not sufficient. Federal agencies need to

ensure that their employment practices, including interviews, are (1)

based on job analysis; (2) relevant to the position being filled; and (3)

nondiscriminatory.

A structured interview can meet all these require-

ments.

Acceptance and defensibility. Candidates expect prospective employ-

ers to assess their qualifications thoroughly and fairly—and candi-

dates have recourse against employers who fail to do so. Candidates

will tend to accept, and be less likely to challenge, employment prac-

tices that are clearly job-related and even-handed. From this perspec-

tive, interviews that are objective—even if somewhat impersonal—

are to be preferred to interviews that are more sociable and subjective.

The structured interview contains several highly visible elements to

promote fairness and objectivity, and thus ensure candidate accep-

tance.

Structured interviews are not only fair and compliant; they are

demonstrably so. Documentation (a “paper trail”) is an integral part of

the structured interview. A written record of how an interview was

developed, conducted, and applied is extremely valuable if it becomes

necessary to formally justify an employment decision based on an

interview.

Title 5, United States Code, Section 2301(b)(1)

5 CFR 300.103.

A Report by The U.S. Merit Systems Protection Board 21

The Case Against the

Unstructured Interview

For most of us, hiring someone is essentially a romantic process * * *

We are looking for someone with whom we have a certain chemistry

* * * We want the unlimited promise of a love affair. The structured

interview, by contrast, seems to offer only the dry logic and practicality

of an arranged marriage.

As noted above, the traditional, unstructured—or loosely-

structured—interview is very appealing. It seems to provide a coun-

terweight to impersonal, mechanical assessments such as training and

experience evaluations and written tests, and (therefore) to provide a

less abstract and more realistic picture of the candidate.

Unfortunately, researchers have found that the validity of unstructured

interviews is lower than that of structured interviews. At best, perfor-

mance in an unstructured interview explains only 14 percent of varia-

tion in on-the-job performance.

Unstructured interviews are also

risky, even when well-intentioned. The absence of structure leaves the

interviewer vulnerable to cognitive biases (that is, patterns in how we

receive and process information that can lead to errors in judgment),

which include:

• Reliance on first impressions. Research shows that people make

judgments about each other when they first meet—whether in per-

son or on paper—and that those judgments have great staying

power. In effect, the handshake (or job application) colors the inter-

viewer’s recollection and evaluation of the subsequent interview.

This introduces the possibility that the interview will not add to the

selection process, but will simply serve to confirm existing, possibly

mistaken impressions of the individual. It also introduces the possi-

bility that the resulting selection may reflect—unintentionally—

personal liking, in addition to or instead of ability to do the job.

Gladwell, op. cit., p. 86.

Schmidt and Hunter, op. cit., p. 37.

22 The Federal Selection Interview: Unrealized Potential

• Reading too much into the interviewee’s behavior. Often, people

assume that other people behave in ways that reflect their underly-

ing character, and fail to give due weight to the context in which

that behavior occurs. Richard Nisbett, one researcher, describes the

resulting problem as follows:

When you have an interview with someone * * * you don’t conceptual-

ize that as taking a sample of a person’s behavior, let alone a possibly

biased sample, which is what it is. What you think is that you are see-

ing a hologram, a small and fuzzy image but still the whole person.

At best, a lack of structure reduces the value of the interview as an

assessment tool. At worst, a lack of structure can lead to perceptions—

and quite possibly the reality—of disparate or unfair treatment.

Gladwell, op. cit., p. 72.

A Report by The U.S. Merit Systems Protection Board 23

The Structured Interview Process

Appendix D provides a simplified model of the structured interview

process. The model places the elements of the structured interview in

the context of the selection process and shows how these elements “fit”

together. The ordering of steps in the model is more illustrative than

prescriptive. Steps such as job analysis, determining the role of the

interview, and interviewer training may be completed before a vacancy

occurs. The appendix is not a detailed, “how-to” guide to structured

interviewing. As we note in the box below, “Developing Interviews—

A Job for Professionals?,” Federal employment practices are held to a

high standard and agencies must take considerable care when develop-

ing any assessment tool.

We believe that the requirement that an assessment tool be profession-

ally developed should be viewed as an ideal rather than as an absolute

requirement. It is neither realistic nor desirable for agencies to discon-

tinue using an interview simply because it was not developed by a per-

sonnel psychologist or an equivalent professional. However, agencies

should take steps to meet the spirit of the requirement, ensure that

assessment tools are developed by informed and knowledgeable staff,

and make their interviews as good as reasonably possible. With this

goal in mind, we offer the following suggestions on how agencies can

make better use of interviews, in addition to adding structure.

Developing Interviews—

A Job for Professionals?

Agencies are expected to use employment practices—including assessment tools,

such as interviews—that are based on job analysis, relevant, and nondiscrimina-

tory. To demonstrate relevance, the agency must show that the employment prac-

tice was “professionally developed.”

This does not mean that developing a structured interview is a job for “HR

experts” only. The insights and perspectives of knowledgeable managers and sub-

ject matter experts are indispensable. But it does mean that agencies should treat

interviews with respect, and devote appropriate attention and resources to their

development and use.

24 The Federal Selection Interview: Unrealized Potential

Determine the purpose of the interview. The hiring organization

must determine the role of the interview in the selection process,

including:

• Which job-related competencies

the interview will assess. This

will depend on several factors, such as the role of other assessments

in the selection process, the nature of the candidate pool, and rela-

tionship between the competency and high performance. For exam-

ple, if candidates already have been evaluated on knowledge of

accounting principles through review of the written application,

asking questions about accounting principles may add little value,

unless the interviewer wants to confirm or expand on existing infor-

mation. On the other hand, if candidates have not been systemati-

cally assessed on oral communication—and the organization’s job

analysis indicates that excellent oral communication is a characteris-

tic of high performers—then it makes sense to use the interview to

assess candidates on this competency.

• How interview results will be used. Interviews may be used to rank

or group candidates, and thus narrow the candidate pool, or they

may be used to make a selection from a pool of ranked or grouped

candidates. The same principles apply to both.

However, the level

of precision needed, resource requirements, and the effect on the

candidate pool may be substantially different. For example, an

interview used to select from a small group of final candidates may

focus on high-level competencies and be relatively lengthy. In con-

trast, an interview used to sort a large pool of candidates into

groups (such as “qualified” and “highly qualified”) may focus on

basic competencies and be relatively brief (in the interest of effi-

ciency).

• How the interview will support the organization’s recruiting strat-

egy. Interviews can be used to assess job candidates, or to support

recruitment efforts by providing information and generating candi-

date interest. These two functions are not mutually exclusive, but

they must be balanced. That balance will depend on several factors,

including the context of the interview, the nature of the candidate

pool, and the stage of the selection process. For example, interviews

at a visit to a university campus may be primarily informational,

intended to educate students about Government employment

opportunities. It would make little sense to use interviews to assess

Here, “competency” refers to all aspects of qualifications (e.g., related knowledges, skills, abilities,

behaviors, and other characteristics), other than suitability, that an organization may assess as part of the

employment process.

Because OPM retains jurisdiction over qualification standards, assessments used to screen candidates

(i.e., to determine whether a candidate meets minimum qualification requirements) may be subject to

OPM review and approval.

A Report by The U.S. Merit Systems Protection Board 25

students who have not applied for—and may not even be inter-

ested in—a position with the organization. On the other hand, a

final interview for a position will most likely center on

assessment—identifying the best qualified candidate. However,

even interviews of this type should have an informational compo-

nent in order to keep candidates informed and interested through-

out the selection process. There is little point in identifying the

best-qualified candidates if the organization cannot sustain those

candidates’ interest in employment.

Provide background information. Providing background informa-

tion can serve as an “ice-breaker” and help the candidate understand

the interview process and its purpose as well as basics about the job.

Appropriate background information includes:

• The current status of the selection process,

• The role of the interview in the selection process,

• The interview format,

• Job duties and responsibilities, and

• Work environment.

We note that many organizations elect to provide background infor-

mation in advance of the interview. That option makes efficient use of

the interviewer and interviewee’s time, promotes consistency, and can

help put candidates at ease by eliminating the element of “surprise.”

Probe. Candidates’ responses are likely to vary in length and level of

detail. This is acceptable, as long as the interviewer obtains enough

information to rate each candidate fairly and accurately. The inter-

viewer may need to request additional examples or more specifics to

fully understand the candidate’s answer. Probing, done tactfully, also

gives a candidate who is less talkative the opportunity to describe his

or her relevant experience.

Answer candidate questions. This report focuses on the interview as

an assessment tool for the employer. The interview is also a recruit-

ment tool—or, viewed from the candidate’s perspective, a way for the

candidate to assess the prospective employer. Providing an opportu-

nity for the candidate to ask questions helps the candidate make a

well-informed employment decision, thus increasing the odds of a

good fit between employee and employer. It also shows interest in the

candidate, which is part of effective recruiting.

Follow up. Interviewers should ensure that they have obtained suffi-

cient and reliable information from the interview. First, the interview-

ers should verify that the candidate’s responses and the record of the

interview are sufficiently detailed. (This should rarely be a problem, if

26 The Federal Selection Interview: Unrealized Potential

interviewers ask well-designed questions and use appropriate probing.)

However, because interviewers’ note-taking and memories are not

always perfect, it may be appropriate in some cases to contact the can-

didate for the necessary information. Second, interviewers should

determine whether the candidate’s responses are consistent with infor-

mation from other sources, such as the application and reference

checks. Inconsistencies may be readily resolved or explained—but

they may also raise questions about candidate qualifications and truth-

fulness which warrant further fact-finding.

Evaluate and refine the interview. We suggest that organizations

view the structured interview as a “living assessment.” Periodically

evaluating the structured interview process and its outcomes can help

identify strengths and opportunities for improvement. (This evalua-

tion need not be as elaborate as a formal validation study, which is a

rigorous statistical analysis of the interview and its outcomes to dem-

onstrate that it measures job-related criteria and distinguishes between

candidates based on ability to perform. Formal validation may not be

feasible or cost-effective for low-volume jobs or occupations. However,

organizations should seriously consider validating interviews used for

high-volume jobs or occupations, where the cost of validation will be

compensated by the interview’s increased validity and defensibility.)

Some questions to ask could include:

• Do the interview questions make useful job-related distinctions

among candidates?

• Are interviewers comfortable with the interview questions and pro-

cess?

• Are interviewers applying the rating scales consistently?

• How are candidates performing on the interview? Are different

groups performing differently?

• How are selecting officials using the results of the interview?

• Do selecting officials believe that the interview is successful at iden-

tifying good employees?

• How do candidates perceive the interview?

• Do the benefits outweigh the costs?

Asking such questions—and acting on the answers—will ensure that

the structured interview remains relevant, useful, and defensible.

A Report by The U.S. Merit Systems Protection Board 27

The State of Federal Employment

Interviewing

Federal managers make extensive use of interviews, and they believe

that interviews are valuable. As shown in Fig. 2, managers responding

to our 2000 Merit Principles Survey believed that the interview is an

excellent predictor of on-the-job performance, surpassing most other

commonly used assessments.

As discussed previously, research provides conditional support for the

high regard Federal managers have for the interview: a well-designed,

properly conducted structured interview is a very good assessment

tool. However, information from the Board’s Merit Principles Survey

and past studies suggests that many agencies are not taking full advan-

tage of the interview’s potential. Indicators include:

Figure 2. Percent of supervisors indicating that information

predicts performance “to a great extent”

Source: U.S. Merit Sytems Protection Board, 2000 Merit Principles Survey.

13%

22%

28%

34%

36%

57%

61%

Reputation of

e d uc a tio na l institutio n

W ritten test

C o lle ge G P A

Personal

recommendations

Level of education

Major field of study

Q ua lity o f the

application

Reference checks

Prior work experience

Interviews

Source of Information Percent

28 The Federal Selection Interview: Unrealized Potential

Excessive reliance on personal observation and judgment. The

responses to our Merit Principles Survey shown in Figure 2 suggest

that managers prefer assessments that allow for “hands on” managerial

involvement, such as interviews and reference checks, over “hands off”

assessments such as written test scores and college grade-point average.

This preference is understandable: we tend to prefer direct observation

over indirect observation, and our own perceptions and judgments

over those of others. But this preference may also lead managers to be

overly confident about their own impressions and judgments, and

view the “assistance” of structure as unnecessary to good

interviewing—and to be comfortable with a relatively unstructured

approach to interviewing. This would be unfortunate, because struc-

ture helps interviewers make better judgments about job candidates,

and structured interviews are considerably more effective than

unstructured interviews.

(We note that this preference may also lead to poor selection of assess-

ment tools. Given the choice, managers are unlikely to use or give

appropriate consideration to assessments that they do not value.

Unfortunately, managers’ valuations of assessment tools are not consis-

tent with current research on the ability of assessment tools to predict

on-the-job performance. For example, managers view the written test

with considerable disdain, although it is one of the best assessment

tools available—and far better than indicators such as level of educa-

tion and reference checks.)

A tendency to sacrifice quality for speed in the hiring process. A

Board report examining how Federal supervisors are fulfilling their

human resource management responsibilities observed that “supervi-

sors too often succumb to the pressures of filling a vacancy quickly by

* * * using assessment or selection processes that get fast results, but

may not produce the best candidates available.”

This suggests that

agencies may resort to relatively unstructured interviews when trying

to fill jobs quickly. Structured interviews require advance planning

and an upfront investment of time and resources—inputs that may be

overlooked or dispensed with under pressure. This is not to say that

structured interviewing is incompatible with a streamlined hiring pro-

cess. Structured interviews can be done almost as quickly as unstruc-

tured interviews, if the agency has laid the groundwork, such as

performing job analysis, developing questions and rating scales, and

training interviewers, beforehand. But, as we have noted above, man-

agers may be reluctant to invest much time in the hiring process. And,

U.S. Merit Systems Protection Board, “Federal Supervisors and Strategic Human Resources Manage-

ment,” Washington, DC, June 1998, p. 2.

A Report by The U.S. Merit Systems Protection Board 29

as we discuss below, agencies are often reluctant to invest resources in

the hiring process.

Insufficient resources devoted to developing and using assessments.

The Board’s recent studies

of delegated examining units

highlight

two resource-related issues that could lead to suboptimal interviewing

practices. First, primary responsibility for developing and administer-

ing assessment tools has been shifted from the Office of Personnel

Management to Federal agencies.

However, this responsibility was

not necessarily accompanied by supporting resources needed to meet

it. The outcome is that managers may not have access to high-quality

assessment tools, such as a good structured interview:

Agencies vary widely in their ability to develop and apply good training

and experience (or any other) assessment instruments. Agencies with

little in-house expertise in this field, and little or no discretionary

money to pay OPM or anyone else for the needed expertise, are at a

distinct disadvantage.

Second—and more troubling—is the fact that some delegated exam-

ining units appear to be under great pressure to minimize operating

costs. Such delegated examining units may be unable or unwilling to

invest in assessment tools, even when the investment could be recov-

ered almost immediately:

* * * almost all of the DEU officials who filled [positions in occupa-

tions where written tests are required at lower grade levels] told us that

the cost of the certificates is a significant factor in the decision not to

fill these positions at the lower grades.

This suggests that some organizations find it easier to fill positions at

higher grades—and incur thousands of dollars in additional salary

costs—than to pay for and use an assessment tool that could have

increased the likelihood of hiring a good employee at a lower grade

level. If this logic is applied to interviews, agencies will tend to use

unstructured instead of structured interviews, because structured

interviews require a greater initial investment.

See U.S. Merit Systems Protection Board, “Assessing Federal Job Seekers in a Delegated Examining

Environment,” Government Printing Office, Washington, DC, December 2001 and U.S. Merit Systems

Protection Board, “The Role of Delegated Examining Units: Hiring New Employees in a Decentralized

Civil Service,” Government Printing Office, Washington, DC, August 1999.

A delegated examining unit (DEU) is an organization that evaluates (examines) applicants for employ-

ment in the Federal competitive service under authority delegated by the U.S. Office of Personnel Man-

agement. There are currently nearly 700 DEU’s, most located in agency human resources offices.

Most written tests currently in use were developed centrally by OPM. However, agencies must now

bear the costs of test administration and scoring.

U.S. Merit Systems Protection Board, “Assessing Federal Job Seekers in a Delegated Examining Envi-

ronment,” Government Printing Office, Washington, DC, December 2001, p. 31.

U.S. Merit Systems Protection Board, “The Role of Delegated Examining Units: Hiring New Employ-

ees in a Decentralized Civil Service,” Government Printing Office, Washington, DC, August 1999, p. 13.

30 The Federal Selection Interview: Unrealized Potential

Inexpert interviewers. Interviews are typically conducted by supervi-

sors and managers. A 1992 Board study of first-line supervisors found

that a vast majority—nearly 90 percent—are involved in selecting

new employees.

Unfortunately, a recent OPM study finds broad

deficiencies in how Federal supervisors are prepared for their responsi-

bilities, stating that “[Supervisors] come to the job without leadership

development and they get very little after assuming their new responsi-

bilities.”

This suggests that relatively few supervisors are trained in

developing or conducting interviews—a specialized and, for many

supervisors, infrequently performed task. The situation does not

improve greatly at higher levels of management. As part of its study of

assessment tools

, the Board recently surveyed a small sample of

senior Federal managers (members of the Senior Executive Service, or

their representatives). Barely half of these managers indicated that they

had ever received training in conducting interviews.

The lack of training is troubling, because interviewing is neither intui-

tive nor easy. In fact, it has been characterized as “uniquely difficult

among managerial tasks.”

The interviewer must ask questions, be

attentive to verbal and nonverbal cues, interpret and record a great

deal of information, “sell” the job and the organization, and accurately

rate the candidate’s responses—often in the space of one hour or less.

An untrained interviewer will be hard-pressed to do all these things

well, and realize the full potential of even a well-conceived interview.

There are many good

interviewing practices, and

improvements are being

made.

As we have indicated, selection interviewing in the Federal Govern-

ment is highly variable. The positive aspect of this variation is that

some agencies are making good use of interviews. Some agencies,

including the U.S. Border Patrol, the U.S. Customs Service, and the

Social Security Administration, have recognized the importance and

potential of the interview for many years. The illustration at right,

“Investing in Workforce Quality,” discusses how interviews contribute

to high-quality selections at the U.S. Border Patrol.

Other agencies have taken steps to make better use of the selection

interview. For example, the U.S. Department of Veterans Affairs has

conducted an extensive education and marketing campaign to encour-

Data from MSPB survey of Federal first-line supervisors (See U.S. Merit Systems Protection Board,

“Federal First-Line Supervisors: How Good Are They?,” Government Printing Office, Washington, DC,

March 1992).

U.S. Office of Personnel Management, “Supervisors in the Federal Government: A Wake-Up Call,”

Washington, DC, January 2001, p. 16.

U.S. Merit Systems Protection Board, “Assessing Federal Job Seekers in a Delegated Examining Envi-

ronment,” Government Printing Office, Washington, DC, December 2001, p. 34.

Wayne F. Cascio, “Applied Psychology in Human Resource Management,” fifth ed., Prentice-Hall,

Inc., Upper Saddle River, NJ, 1998, p. 195.

A Report by The U.S. Merit Systems Protection Board 31

age managers to use structured interviews. The discussion on page 32,

“Improving Selections One Interview at a Time,” highlights key ele-

ments of this campaign.

Also, many agencies are using structured interviews as part of their

assessment strategy under OPM’s Competency-based

Information

Technology (IT) Job Profile Pilot program.

The pilot program uses a

job profile (a competency-based qualifications standard) in lieu of the

current qualifications standard, which expresses minimum qualifica-

tions in terms of years of experience and/or years of education. Under

the pilot, agencies have several assessment options, including ratings of

training and experience, structured interviews,

and an on-line objec-

tive test. OPM reports positive feedback from agencies that are using

structured interviews to assess candidates.

OPM defines a competency as “an observable, measurable pattern of knowledge, skills, abilities, behav-

iors, and other characteristics that an individual needs to perform work roles or other occupational func-

tions successfully.”

To provide agencies an incentive to join the pilot program, OPM worked with agencies to develop

benchmarks for more than 50 general and technical competencies. OPM also developed a structured

interview for use with these competencies.

OPM provided required training in structured interviewing to prospective interviewers in agencies that

elected to use structured interviews. OPM offers similar training, on a reimbursable basis, outside the

pilot project.

Investing in Workforce

Quality

The U.S. Immigration and Naturalization Service (INS) in the U.S. Department

of Justice uses structured interviews to assess candidates for entry-level Border

Patrol Agent and Immigration Inspector positions. The INS hires an average of

2,000 Border Patrol Agents and 2,000 Immigration Inspectors each year.

After initial screening based on a written test, candidates are interviewed by a

three-person panel of Border Patrol Agents who have been trained in interview-

ing. The interview is designed to complement the written test, and focuses on

important competencies such as:

• Judgment and decision-making,

• Interpersonal skills,

• Emotional maturity,

• Cooperativeness and sensitivity to others, and

• Oral Communication.

Responses are evaluated against benchmarks developed by experienced Border

Patrol agents and supervisors in cooperation with personnel psychologists. Can-

didates who perform acceptably on the interview, which is scored on a pass/fail

basis, remain eligible for further consideration.

The Border Patrol has used this approach successfully for many years. The Immi-

gration Inspections program implemented its structured interview in the spring

of 2002. Managers consider the interview an indispensable part of the assessment

and selection process, and view it as an investment in a high-quality workforce.

32 The Federal Selection Interview: Unrealized Potential

Improving Selections One

Interview at a Time

The U.S. Department of Veterans Affairs (VA) has a large, diverse, geographically

dispersed workforce, employed in occupations ranging from housekeeping aid to

loan specialist to physician, in locations ranging from urban offices to rural med-

ical centers. Implementing structured behavioral interviewing (which VA refers

to as performance-based interviewing, or PBI) in such an environment is a daunt-

ing task—but VA has taken steps to do just that.

VA's size and scope make it impossible to implement PBI through an “act of

HR.” Instead, VA's strategy is to “market” PBI directly to the managers and

supervisors who will benefit from it, and disseminate the knowledge and tools

needed to make interviews more structured. Elements of this strategy include:

• Identifying line and senior managers at each installation who understand the

importance of effective interviews and who can champion structured inter-

viewing;

• A satellite broadcast, introduced by a high-level executive, to educate manag-

ers on the concept of the structured interview and its benefits. The broadcast

was videotaped and distributed to all field installations;

• Purchasing two commercially-available videos on structured interviewing and

providing one copy to each Veterans Integrated Service Network (VISN, a

field organization with management and operating responsibility for several

medical centers and clinics);

• Developing guidance and training materials, which are available on VA's web

site and on CD-ROM;

• Training designated staff (typically one manager and one human resources

specialist) at each field installation. These staff can train other employees who

are involved in developing and conducting interviews;

• Employee training, to make employees familiar with PBI’s intent and format

so that they can “put their best foot forward” in job interviews; and

• Managerial accountability. Although VISN directors are evaluated primarily

on outcome measures, they are also evaluated on steps they have taken to

improve organizational performance. This may include promotion and use of

PBI.

Although PBI is a work in progress, initial feedback is positive. One long-time

user reports that staff quality and the organization's reputation have risen sub-

stantially, making the organization a more attractive employer. Another user

reports that the PBI techniques can be applied effectively in other areas, such as

in discussions with contractors.

A Report by The U.S. Merit Systems Protection Board 33

Recommendations

This paper focuses on a single aspect of the assessment and selection

process—the interview. Over the years, the Board has repeatedly

encouraged agencies to use good assessment tools, as in the following

statement from 1999:

Agencies need to seriously examine the way they assign people to jobs.

They need to devote resources to finding ways to assess job candidates

that permit a good fit between worker and work; they need to evaluate

how well they accomplish these tasks; and they need to adjust their

methods accordingly.

This counsel has particular relevance to interviewing, because inter-

views are widely used and highly influential in selection decisions. The

following recommendations simply place the interview in this broader

context.

1. Agencies should decide which purpose(s) the interview will serve,

and design and conduct the interview accordingly. Interviews can

be purely informational, supporting recruitment efforts by educating

candidates about employment opportunities and conditions of

employment. Or, interviews can focus purely on assessing candidates’

qualifications. Or they can do both. Agencies must make a conscious

and informed choice, because that choice has significant implications

for the design and conduct of the interview. Interviews that function

only as a recruitment tool do not require a high level of structure.

On the other hand, interviews that function as assessment tools—that

are used to screen candidates, rank or group candidates, or make a

final selection decision—call for careful design and considerable

structure, if the organization is to realize their full potential.

2. Agencies that use interviews to assess job candidates should use

structured interviews. In making this recommendation, we distin-

guish between interviews that function as a continuation of the evalu-

ation of training and experience—for example, an interview that

U.S. Merit Systems Protection Board, “Federal Supervisors and Poor Performers,” Washington, DC,

July 1999, p. 33.

Even here, some structure—such as recruiter training and “talking points”—will help the organization

project a clear, consistent image and maximize the effectiveness of the informational interview.

34 The Federal Selection Interview: Unrealized Potential