2000 USMLE Performance Data
As published in the 2000 NBME Annual Report,
Copyright 2001 by the National Board of Medical Examiners®

USMLE Administration, Minimum Passing Scores, and Performance

Full implementation of computer-based administration of the USMLE was achieved in 1999. Prometric, Inc.®, a division of Thomson LearningT, provides scheduling and test centers for the USMLE. By the end of 2000, more than 100,000 examinations had been administered at 327 US/Canadian Prometric test centers and 192 international Prometric test centers. Additional test centers are located at eight medical schools in the United States. The Step 1 and Step 2 examinations are administered worldwide; the Step 3 examination is administered only in the United States.

Prior to computer-based testing (CBT), USMLE examinees were restricted to two test administration dates per year for each of the three Steps. With CBT, continuous test administration is now available to all examinees. After completion of the registration process and receipt of certification of eligibility, applicants for Step 1, Step 2, or Step 3 contact the Prometric scheduling network to make an appointment to take the examination at a location and time most convenient to the applicant's schedule. Nearly all US applicants are able to test at a center within 50 miles of their home address and within 10 business days of their most desired date.

Examination results are processed continuously, and scores are reported weekly. Most Step 1 and Step 2 examinees receive their scores approximately three to four weeks after their test date and most Step 3 examinees, four to six weeks. Individual score reports are mailed to examinees; medical schools and state licensing authorities can receive score information by using a secure website. Paper score rosters are no longer provided.

The USMLE program recommends a minimum passing score for each Step of the USMLE. Minimum passing scores are based on achievement of specified levels of proficiency established prior to administration of examinations. Statistical procedures are employed to insure that for each Step, the level of proficiency required to pass remains uniform across forms of the examination. As noted in the USMLE Bulletin of Information and at the USMLE website, the score required to meet the recommended level of proficiency is reviewed periodically and may be adjusted without prior notice if necessary prior to score reporting. Notice of adjustments is posted on the Internet at www.usmle.org.

In setting the minimum passing score for USMLE examinations, the USMLE Step Committees employ information gathered from a number of sources including standard setting surveys and content-based standard setting panels. A standard setting survey for all three Steps was conducted in 2000; and content-based standard setting panels were convened for the Step 1 and Step 2 examinations. Reviews of USMLE standards by the Step 1 and Step 2 Committees were conducted in 2000. A content-based standard setting panel and review of the Step 3 standard by the Step 3 Committee are scheduled for 2001.

In February 2000, a survey regarding the appropriateness of current USMLE pass/fail standards for each of the Steps was sent to (1) random samples of 2000 examinees taking each Step in 1999, (2) approximately 3000 directors of basic science courses and clinical clerkships, (3) 250 associate deans for academic and student affairs, (4) directors and chief residents from approximately 1850 residency programs, (5) 250 members of USMLE test material development committees, (6) executive directors and presidents of all state medical boards, and (7) 72 members of the National Board of Medical Examiners and Federation of State Medical Boards Board of Directors. Of the more than 13,000 persons surveyed, roughly 40% responded. Only the responses from individuals who indicated that they felt qualified to answer items regarding a particular Step and completed correctly all associated portions of the survey were included in analyses. Samples of 3359, 2694, and 1886 resulted for Steps 1, 2, and 3, respectively.

On average, across groups, almost 70% of respondents indicated that current initial failure rates for first-time takers from US schools were "about right" for all three Steps, and 25% indicated that initial failure rates were "too low." For ultimate failure rates, 60% indicated that the rates were "about right," and 40% indicated that the rates were "too low." Respondents were also asked to indicate "ideal," "minimum," and "maximum" initial and ultimate failure rates for first-time takers from US schools; current failure rates were provided on the survey form. Results for the Step 1 and Step 2 programs are given below; Step 3 results will be reported in 2001 after review of the Step 3 minimum passing score.

Step 1

In 1998, 1999, and 2000, the pass rates for first-time takers from US and Canadian allopathic medical schools were 95%, 94% and 93%, respectively. Approximately 16,500 first-time takers were tested each year. For the same time period, the pass rates for first-time takers from US osteopathic schools were 87%, 82%, and 77%, respectively; osteopathic examinee counts grew from 320 in 1998 to 776 in 2000. For foreign medical graduates (FMGs), 65% passed the examination on their first attempt in 2000, compared with 62% in 1998 and 65% in 1999. Details of the performance of US/Canadian and foreign medical graduates are provided below. Because failing examinees generally retake Step 1, the ultimate failure rate across test administrations is expected to drop to 1% for US and Canadian graduates of allopathic schools, as it has since USMLE was introduced in 1992.

TABLE 1 - 1999-2000 STEP 1 ADMINISTRATIONS Number Tested and Percent Passing
	1999		2000*
	#Tested	%Passing	#Tested	%Passing
US/Canadian Examinees
Allopathic Students	17,552	91%	18,267	90%
1st Takers	16,254	94%	16,412	93%
Repeaters**	1,298	54%	1,855	58%
Osteopathic Students	559	80%	823	75%
1st Takers*	543	82%	785	77%
Repeaters**	16	19%	38	40%
Total US/Canadian	18,111	91%	19,090	89%
FMG Examinees
1st Takers*	6,510	65%	8,767	65%
Repeaters**	3,334	32%	4,688	36%
Total FMG	9,844	53%	13,455	55%
* '1st Takers' represents examinees with no prior Step 1 attempts. ** 'Repeaters' represents examinations given, not number of different examinees.

As noted above, performance standards for Step 1 were reviewed in 2000. In reviewing the Step 1 standards, the Step 1 Committee considered the results of the USMLE standards survey conducted in 2000, and the recommendations of Step 1 standard setting panels convened in 2000. Data from the standard setting survey indicated that when averaged across groups for Step 1, the recommended ideal initial failure rate was 6%; the recommended minimum was 4%; and the recommended maximum was 10%. The recommended ideal ultimate failure rate was 2%; the recommended minimum was 1%; and the recommended maximum was 4%.

Three panels of medical school faculty participated in the Step 1 content-based standard setting panels. The 24 panelists were selected to represent a range of medical schools, basic science and medical specialties. Panelists had a broad range of roles, including course and clerkship directors, residency program directors, department chairs, and practitioners; a range of gender and ethnic groups were also included. Each panel met for 1 ½ days, engaging in a modified version of the "Angoff" standard-setting procedure. The same procedure was used to determine the minimum passing score for Step 1 in 1991 and 1998. Each panelist reviewed a subset of the items appearing on a single form of the Step 1 CBT examination and developed a content-based, percent-correct standard that, in their view, reflected the minimum level of performance representing mastery of the material for the purpose of licensure. Results were provided to the Step 1 Committee for consideration.

At its December 13-14, 2000 meeting, the USMLE Step 1 Committee reviewed analyses of recent trends in examinee performance, results of the standard-setting survey, recommendations from content-based standard-setting panels, and the effect of measurement imprecision on pass/fail decisions. This review resulted in the decision to raise the Step 1 minimum passing score from 179 to 182; the new minimum passing score took effect for test administrations on or after January 1, 2001. Although the Step 1 Committee may adjust the minimum passing score at any time, no further adjustment is expected for several years.

Step 2

For examinees taking Step 2 between August 16, 1999, and June 29, 2000, the pass rate for first-time takers from US and Canadian allopathic medical schools was 95%. This is comparable to the pass rate for the last paper-and-pencil administrations in 1998-1999. The mean three-digit score performance on the first computerized Step 2 administration in 1999-2000 was slightly better than on past paper-and-pencil administrations (mean score of 213 [SD =24] versus 210 [SD= 23]). For foreign medical graduates, 70% of first-time takers passed Step 2 in the 1999-2000 CBT administration cycle, up from 59% for the 1998-99 paper-and-pencil administrations.

Failing examinees generally retake Step 2. As a result, the ultimate pass rate across test administrations is expected to be higher. It is anticipated that 99% of examinees from US and Canadian allopathic schools will ultimately pass the examination. However, because computerized Step 2 administrations were only recently introduced, it is too soon to report ultimate pass rates for examinees failing for the first time on this form of the examination.

Performance standards for Step 2 were reviewed in 2000. Data from the standard setting survey indicated that when averaged across groups for Step 2, the recommended initial failure rate was 6.7%, the recommended minimum was 4.1% and the recommended maximum was 10.5%. The recommended ultimate failure rate was 2.4%. The recommended minimum ultimate failure rate was 1.8%; the recommended maximum was 4.8%.

TABLE 2 - 1999-2000 STEP 2 CBT ADMINISTRATIONS Number Tested and Percent Passing
	August 16, 1999 - June 29, 2000
	#Tested	%Passing
US/Canadian Examinees
Allopathic Students	17,406	93%
1st Takers*	16,279	95%
Repeaters**	1,127	66%
Osteopathic Students	183	90%
1st Takers*	178	92%
Repeaters**	5	40%
Total US/Canadian	17,589	93%
FMG Examinees
1st Takers*	6,027	70%
Repeaters**	3,172	41%
Total FMG	9,199	60%
* '1st Takers' represents examinees with no prior Step 2 attempts. ** 'Repeaters' represents examinations given, not number of different examinees.

Three panels of physicians participated in content-based standard setting for Step 2. The 22 panelists were selected to represent a range of medical schools and medical specialties. In addition, panelists had a broad range of roles such as clerkship directors, program directors, department chairs, practitioners, and state board members; a range of gender and ethnic groups were also included. Each panel met for 1 ½ days, engaging in a modified version of the "Angoff" standard-setting procedure. The same procedure was used to set standards in 1991 and 1996. In the most recent review, each panelist saw a subset of the items appearing on a single form of the Step 2 CBT exam and developed a content-based percent-correct standard that, in their view, reflected the minimum level of performance representing mastery of the material for licensure purposes. Across panels, these deliberations would have resulted in a standard of approximately 174. The results were provided to the Step 2 Committee for consideration.

At its April 17, 2000 meeting, the USMLE Step 2 Committee reviewed the analyses of recent trends in examinee performance, the results of the standard-setting survey, and the recommendation from the content-based standard-setting panels. This review resulted in the decision to raise the Step 2 minimum passing score to 174. Although the minimum passing score is reviewed annually and the Step 2 Committee may adjust this value at any time, it is not expected to change again for several years.

Step 3

Based upon examinees for whom Step 3 scores were reported in 2000, the pass rate for first-time examinees from U.S. allopathic medical schools was 95%. The pass rate for first-time examinees from U.S. osteopathic medical schools was 89%. On all examinations taken by allopathic graduates who were repeating Step 3, 59% of the reported scores were at or above passing. The comparable figure for osteopathic graduates was 86%. The pass rate for first-time takers who were graduates of foreign medical schools was 58%; the pass rate for repeat examinations for this group was 38%.

The minimum passing score for USMLE Step 3 was originally set in 1994 to be a three-digit score of 176. A periodic, in-depth review of the minimum passing score was conducted in 1997, and resulted in an increase in the minimum passing score to 177. The Step 3 Committee is scheduled to review the minimum passing score in March 2001. If the Committee determines that a change is appropriate, the new standard will be applied to Step 3 administrations beginning in April 2001. (Click here for updated information.)

In its review of the pass/fail standard, the Step 3 Committee will consider information from an analysis of recent trends in examinee performance, results of the standard-setting survey, recommendations from the content-based standard-setting panels, and the effect of measurement imprecision on pass/fail decisions.

TABLE 3 - 2000 STEP 3 ADMINISTRATIONS Number Tested and Percent Passing
	November 1999 - December 2000
	#Tested	%Passing
US/Canadian Examinees
Allopathic Students	14,365	92%
1st Takers*	13,135	95%
Repeaters**	1,230	59%
Osteopathic Students	60	88%
1st Takers*	53	89%
Repeaters**	7	86%
Total US/Canadian	14,425	92%
FMG Examinees
1st Takers*	5,133	58%
Repeaters**	4,608	38%
Total FMG	9,741	49%
* '1st Takers' represents examinees with no prior Step 3 attempts. ** 'Repeaters' represents examinations given, not number of different examinees.