Date: January
30, 2024
To: President Sian Leah Beilock, Dean of Admissions Lee Coffin
From: Professors Elizabeth Cascio, Bruce Sacerdote, Doug Staiger, Michele Tine
Report From Working Group on the Role of Standardized Test Scores in Undergraduate
Admissions
Dear President Beilock and Dean Coffin:
We are responding to your request for researcher input and detailed empirical analysis on the role
standardized test scores play in undergraduate admissions. Thank you both for the significant
time you have spent providing us with context, details on the admissions process, and extensive
deidentified microdata linking Dartmouth applicants to their undergraduate careers. Thanks also
to Libby Barlow and Rett Weber in Institutional Research for their informative White Paper that
uses Dartmouth data and for the construction and provision of data sets.
1
The inputs into our investigation were numerous. They included: interviews and commentary
from Admissions, the White Paper produced by Institutional Research, dozens of published and
unpublished academic papers on the subject of SAT and ACT scores and admissions and related
subjects, detailed data on Dartmouth applicants and Dartmouth students, and extensive
discussions with colleagues at other institutions who have examined the same questions with
similar data.
Our overall conclusion is that the use of SAT and ACT scores is an essential method by which
Admissions can identify applicants who will succeed at Dartmouth. Importantly, these test
scores better position Admissions to identify high-achieving less-advantaged applicants and
high-achieving applicants who attend high schools for which Dartmouth has less information to
interpret the transcripts. The data suggest that, under an SAT/ACT optional (hereafter
1
The White Paper is an internal document that formed the starting point for our analysis. The White Paper
examined the correlations between admissions data and first-year GPA. Many of the variables we use were
calculated for us by Institutional Research.
“test-optional”) policy, many high-achieving less-advantaged applicants choose not to submit
scores even when doing so would allow Admissions to identify them as students likely to
succeed at Dartmouth and in turn benefit their application.
SAT and ACT scores are highly predictive of academic performance at Dartmouth. This is
consistent with previous research (e.g., Comeaux and Sánchez, 2020; Bettinger et al., 2013;
Saboe and Terrizzi, 2019). It is also a key finding of the White Paper and a result we have
reproduced in the data provided. Using detailed admissions data from IvyPlus institutions,
2
Chetty, Deming, and Friedman (2023) show that SAT and ACT scores also predict career
success, including high levels of earnings and attendance at elite graduate schools, holding
family income constant. Importantly, the relationship between first-year college GPA and
SAT/ACT scores is likewise quite similar across neighborhood income and other demographic
subgroups at Dartmouth. By contrast, Chetty, Deming, and Friedman (2023) show that certain
non-test score inputs in the admissions process, such as guidance counselor recommendations, do
not predict college performance even though they do advantage more-advantaged applicants at
IvyPlus institutions, increasing their admissions chances.
By necessity, test-optional policies thus put more weight on components of the application that
increase the admissions chances of groups that: (a) have historically been better represented at
Dartmouth; and (b) do not predict success at Dartmouth or later in life. Moreover, Dartmouth
Admissions reads SAT/ACT scores “in context,” or in relation to the local educational
environment. For example, an applicant with an SAT score of 1400 has a higher probability of
admission if from a high school where average SAT scores are relatively low. Under a
test-optional policy, these students are less likely to be identified and admitted.
Below, we summarize the findings from our independent investigation of data on Dartmouth
applicants and Dartmouth students. Institutional Research provided us with detailed data on
applicants (including enrolled students) who applied (and in some cases enrolled) during
calendar years 2017-2022. This period helpfully spans the test-required and test-optional years.
The data items include SAT/ACT test scores, high school GPA and other admissions indicators,
demographics for the applicants, and College Board measures of high school and (separately)
2
Ivy-Plus institutions are defined as the eight Ivy League colleges – Brown University, Columbia University,
Cornell University, Dartmouth College, Harvard University, Princeton University, University of Pennsylvania, and
Yale University – plus Stanford University, MIT, Duke University, and the University of Chicago.
2
neighborhood advantage and income. We also have indicators for first generation and
international student status. In addition, for enrolled students, we have first-year GPA and
detailed data on grades earned. In the analysis that follows, we define “less-advantaged”
students as those who are any of: U.S. first-generation college going, from a neighborhood with
median income below the 50th percentile for the U.S., or attended a U.S. high school in the top
20% of the College Board's index for challenge.
3
Our key findings are as follows:
1) SAT (and ACT) scores are highly predictive of academic achievement at Dartmouth.
Figure 1 shows a linear relationship between SAT scores and cumulative first-year GPA for
Dartmouth students. SAT scores are grouped into 16 equal-sized bins, and we plot mean
first-year GPA (y-axis) against mean SAT score (x-axis) in the bin.
4
Within any smaller range of
scores, first-year GPA and the SAT maintain the same linear relationship.
SAT scores have significant predictive value for academic achievement over and above other
measures such as high school GPA. Table 1 shows regressions with first-year GPA as the
dependent variable. In column 1, SAT by itself explains about 22% of the variation in first-year
GPA. High school GPA by itself explains 9% of the variation (column 2). The explained
variation in first-year GPA rises to 25% when we include both high school GPA and SAT scores
as predictors (column 3). In other words, the marginal contribution of high school GPA above
SAT is only 3%. High school class rank (column 4) adds very little additional predictive power
and has an additional challenge for use in admissions since we do not observe class rank for a
substantial number of students. In short, consideration of SAT scores allows Admissions to
identify applicants who will thrive academically at Dartmouth better than does the use of high
school GPA alone.
These findings are consistent with a series of studies on the subject. For example, Bettinger et al.
(2013) (“Improving College Performance and Retention the Easy Way: Unpacking the ACT
4
By “equal-sized,” we mean the same number of observations per bin. From here forward, we use the term “SAT
scores” for expository ease. The variable is actually the SAT composite score. When only ACT is available, we
rescaled ACT to the SAT composite scale.
3
The College Board's index for challenge incorporates the median income of students who attend the high school
(based on the American Community Survey (ACS)), crime rates, housing stability (from the ACS), college
attendance for students from the high school (from the College Board), and education level for adults in the
neighborhood of the high school (from the ACS).
3
Exam”) offer empirical evidence on the predictive power of Math and English standardized test
scores for first-year GPA and the likelihood of college dropout. In a task force report on the
fairness of standardized testing in UC school admissions (“Report of the UC Academic Council
Standardized Testing Task Force (STTF)”), Comeaux and Sánchez (2020) also conclude that
standardized test scores are a better predictor of first-year college GPA than high school GPA.
Furthermore, the task force found that high school GPA is less predictive of first-year GPA when
compared to earlier studies by the California Board of Admissions and Relations with Schools,
likely due to variations in grading policy and grade inflation.
2) SAT is a strong predictor of academic success at Dartmouth for all subgroups.
The SAT predicts first-year GPA at Dartmouth similarly for all subgroups we have examined.
For example, Figure 2 shows the relationship between the SAT and first-year GPA separately for
the broad index of more- versus less-advantaged students defined above. For a given SAT score,
less-advantaged students have on average a slightly lower first-year GPA than more-advantaged
students. If SAT scores were a downward biased predictor of academic performance for lower-
income or first-generation students, we might have expected the opposite pattern. Moreover, both
Figure 2 and Table 1 show that the positive relationship between SAT scores and first-year GPA
is similar across the two groups. Table 1 also shows that SAT scores alone predict between 14
and 21% of the variation in first-year GPA within groups defined by disadvantage (columns 5
and 6).
3) A test-optional policy is likely a barrier to Dartmouth identifying less-advantaged
students who would succeed at Dartmouth.
There are three reasons. First, in the absence of having a common test score metric across
students, Admissions is left having to place more weight on other factors that have been shown
to be biased toward higher-income students. These include guidance counselor recommendations
and non-academic ratings created by Admissions officers (Chetty, Deming, and Friedman, 2023).
Second, lower-income students and international students are more likely to be from high
schools where Admissions has less information to interpret the transcript. This is concerning
because under test-optional policies an assessment of the high school transcript becomes even
4
more important. Hence, without SAT scores, it is significantly less clear which students from less
well-known high schools are the most academically capable.
Third, under test-optional policies, some less-advantaged students withhold test scores even in
cases where providing the test score would be a significant positive signal to Admissions.
Importantly, Dartmouth Admissions uses SAT scores within context; a score of 1400 for an
applicant from a high school in a lower-income community with lower school-wide test scores is
a more significant achievement than a score of 1400 for an applicant from a high school in a
higher-income community with higher school-wide test scores. Admissions uses numerous
detailed measures of outcomes at the high school and neighborhood levels to account for these
known disadvantages. As one example, Admissions computes a measure of how each applicant
performs on standardized tests relative to the aggregate score of all test-takers in their high
school, using data available from the College Board.
Even well-informed students cannot be expected to know how reading SAT scores in context is
operationalized. Published test score ranges for Dartmouth suggest that a score of 1400 places a
student well below the 25
th
percentile of test scores, and our data show that most students who
score (or would score) 1400 or less opt not to submit a score. Table 2 gives the total number of
applicants by year who did and did not submit an SAT score. Figure 3 shows a histogram of
submitted scores under test-required versus test-optional policies for enrolled Dartmouth
students. Under a test-optional policy, about 31% of enrolled students have not submitted a
score, and most of the missing mass of the distribution is at a score of 1450 and below.
Figure 4 shows estimates of the rate of score submission within 50-point bins of underlying SAT
scores, both overall and separately for various subsamples of applicants. To arrive at these rates,
we divided the number of scores received in each bin under the test-optional policy by the
number of scores received in each bin under the test-required policy. Since the denominator
ignores the 35% increase in applications in the test-optional period, this approach almost surely
overstates the score submission rate in each bin.
The message from Figure 4 is that all groups (e.g. students from higher- versus lower-income
neighborhoods or students from more- versus less-advantaged backgrounds)
5
submit scores at
5
Recall that we define less-advantaged as an indicator for any of: U.S. first-generation college-going student, from a
neighborhood with median income below the 50th percentile for the U.S., or attended a U.S. high school in the top
5
roughly the same rate for a given SAT score. This similarity actually hinders the admission
chances for less-advantaged applicants, who would raise their admissions probabilities by
revealing their scores at some range of lower SAT scores.
This point is shown explicitly in Figure 5. In the left two panels of the figure, we again show
that score submission rates by underlying SAT score are quite similar for more- and
less-advantaged applicants and by first generation status. However, based on data from the
test-required cohorts, the right panels show that admissions probabilities vary greatly across
groups holding SAT scores constant. For example, at an SAT score of 1400, less-advantaged
students have twice the probability of admission of more-advantaged students. These relatively
high-achieving less-advantaged students likely should submit their scores, as their score would
benefit their application.
Figure 6 reinforces this conclusion, drawing on a specialized subsample of applicants in the
test-optional cohorts who initially submitted scores but then asked Admissions not to consider
them in the admissions decision.
6
The figure plots admissions probabilities for more- and
less-advantaged students (panels a and b) and by first generation status (panels c and d) by SAT
score and whether or not the scores were revealed to Admissions officers. Consider students
with a score of 1450-1490 from less-advantaged backgrounds. These students increased their
admission probability by a factor of 3.7x (from .02 to .074) by revealing their score. The
test-optional policy thus led to Admissions not identifying these high-achieving applicants as
highly prepared. This pattern does not hold for applicants from more-advantaged backgrounds;
students from more-advantaged backgrounds had similar admissions probabilities with and
without submitting their scores. This may be because Admissions has more experience reading
transcripts from the schools these students attend.
These data imply that there are hundreds of less-advantaged applicants with scores in the 1400
range who should be submitting scores to identify themselves to Admissions, but do not under
test-optional policies. Figure 7 shows the frequency of scores by bin of SAT for those applicants
who opted to not use the score in admissions but for whom we observe the score ex post. To
convert these frequencies to our estimates of total unobserved scores, we first divide by two
6
Admissions obliged by not using the scores.
20% of the College Board’s index for challenge. The College Board's index of challenge factors for the high school
incorporates median family income, family structure, crime, and homeownership.
6
since the histogram is for two years of data. We then scale up to the full set of applicants under
the assumption that the 19% of hidden scores we see ex post are a random sample of total hidden
scores.
7
The estimates imply that there are 1,000 less-advantaged students in each applicant pool
who have a score greater than 1400 but do not reveal their score to Admissions under
test-optional.
4) Test-optional policies do not necessarily increase the proportion of less-advantaged
students in the applicant pool.
Figure 8 shows the neighborhood income distribution of the applicant pool for the test-required
and test-optional years. Each bar represents a decile of neighborhood income, with the tenth
decile being the highest-income group. The neighborhood income distribution of the applicant
pool is similar under test-required and test-optional policies. In addition, the U.S.
first-generation share among applicants was also similar for the test-required and test-optional
cohorts. These findings are consistent with Saboe and Terrizzi (2019), who find that the
adoption of test-optional policies results in a brief upsurge in applications but does not
significantly influence diversity metrics within an institution.
Summary
Overall, we find that test scores add significant value to the Admissions process at Dartmouth.
They are significantly predictive of academic success at Dartmouth and increase the likelihood
that Admissions will be positioned to identify high-achieving less-advantaged applicants. In
particular, the data suggest that a test-optional policy leads large numbers of less-advantaged
applicants not to submit scores when it would benefit them to do so.
Bibliography
Bettinger, E.P., Evans, B.J., and Pope, D.G. "Improving college performance and retention the
easy way: Unpacking the ACT exam." American Economic Journal: Economic Policy 5, no. 2
(2013): 26-52.
7
While this is a strong assumption, we could alternatively estimate the number of missing scores by SAT bin by
comparing the score distributions in the test-required and test-optional periods (see Figure 4). The resulting
estimates of missing test scores are quite similar.
7
Carey, John M., Katherine Clayton, and Yusaku Horiuchi. Campus diversity: The hidden
consensus. Cambridge University Press, 2019.
Chetty, R.D., Deming, D.J. and Friedman, J. "Diversifying Society’s Leaders? The Determinants
and Causal Effects of Admission to Highly Selective Private Colleges." accessed at
https://www.nber.org/papers/w31492 (2023).
Comeaux,E. and Sánchez, H. "Report of the UC Academic Council Standardized Testing Task
Force (STTF)." University of California Academic Senate (2020).
Friedman, John N., Bruce Sacerdote, and Michele Tine. “Standardized Test Scores and
Academic Performance at Ivy-Plus Colleges”, Opportunity Insights (2024) accessed at
https://opportunityinsights.org/wp-content/uploads/2024/01/SAT_ACT_on_Grades.pdf .
Leonhardt, David. “The Misguided War on the Sat.” The New York Times, January 7, 2024.
https://www.nytimes.com/2024/01/07/briefing/the-misguided-war-on-the-sat.html.
Saboe, M. and Terrizzi, S. "SAT optional policies: Do they influence graduate quality, selectivity
or diversity?." Economics Letters 174 (2019): 13-17.
8
Figures
Figure 1.
Relationship Between Cumulative First-Year GPA and Composite SAT Scores:
Dartmouth Students
Notes: Figure displays bin scatter plot of cumulative first-year GPA against the SAT for 16 equal-sized bins of SAT
for enrolled Dartmouth students in the 2017-2018 (test-required) and 2020-2021 (test-optional) cohorts. The SAT
Composite is used for all applicants where reported. Where only ACT is available, scores are rescaled to the SAT
scale. Non-reporters have average GPAs at the 31st percentile of cumulative GPA. Key takeaway: Cumulative
first-year GPA is roughly linear in the SAT score at Dartmouth.
9
Figure 2.
Relationship Between Cumulative First-Year GPA and Composite SAT Scores:
Less- versus More-Advantaged Dartmouth Students
Notes: Figure displays bin scatter plot of cumulative first-year GPA against the SAT for 16 equal-sized bins of SAT
for enrolled Dartmouth students in the 2017-2018 (test-required) and 2020-2021 (test-optional) cohorts. Separate bin
scatters are shown for less-advantaged (yellow) and more-advantaged (navy) students. We define “less-advantaged”
students as those who are any of: U.S. first-generation college going, from a neighborhood with median income
below the 50th percentile for the U.S., or attended a U.S. high school in the top 20% of the College Board's index for
challenge. The SAT Composite is used for all applicants where reported. Where only ACT is available, scores are
rescaled to the SAT scale. Key takeaway: The linear relationship between cumulative first-year GPA and SAT score
is similar across more- and less-advantaged Dartmouth students.
10
Figure 3.
Histograms of Composite SAT Scores:
Dartmouth Students in the Test-Optional and Test-Required Cohorts
Notes: Chart displays histograms of SAT scores for Dartmouth Classes of 2017-2018 (test-required; blue) and
2021-2022 (test-optional; red). The overlap of the two histograms appears as purple. The SAT Composite is
included for all students reported. Where only ACT is available, scores are rescaled to the SAT scale. 31% of
enrolled students in test-optional cohorts opted not to submit a score. The two test-optional cohorts of enrolled
students are also 3% smaller than the two test-required cohorts. Key takeaway: The “missing mass” in the SAT
score distribution for Dartmouth students in the test-optional cohorts is at lower scores.
11
Figure 4.
Estimated Fraction of Dartmouth Applicants Reporting Scores Under Test Optional,
by SAT Score: Overall and by Subpopulation
Notes: Chart displays estimates of the fraction of applicants opting to report scores in the test-optional period,
separately for 50-point bins of the SAT composite score. Analysis uses data for Dartmouth applicants in years
2017-2018 (test-required) or 2021-2022 (test-optional). The SAT composite is included for all applicants where
reported. Where only ACT is available, scores are rescaled to the SAT scale. Estimates are derived by dividing the
number of scores for the test-optional cohorts by the number of scores for the test-required cohorts, separately by
bin. 46% of test-optional applicants opted not to submit a score. The two test-optional cohorts of applicants are 35%
larger (see Table 2), so total applicants in each bin are likely higher for test-optional cohorts than test-required
cohorts, making these estimates an upper bound. Low (high) advantage HS=U.S. high school in top 20% (bottom
80%) of the College Board's index for challenge; low (high) income=neighborhood median income below (at or
above) the U.S. 50th percentile; less advantaged=U.S. first-generation college-going or low advantage HS or low
income; more advantaged=not U.S. first-generation college-going and high advantage HS and high income. Key
takeaway: The likelihood of reporting a given SAT score appears similar across groups of Dartmouth applicants.
12
Figure 5.
Score Reporting and Admissions Rates of Dartmouth Applicants, by SAT Score:
By Advantage and First-Generation Status
a. Fraction submitting scores:
by advantage
b. Fraction admitted:
by advantage
c. Fraction submitting scores:
by first generation status
d. Fraction admitted:
by first generation status
Notes: Panels a and c display estimates of the fraction of applicants opting to report scores in the test-optional
period, separately for 50-point bins of the SAT composite score. Estimates are calculated as described in the notes
to Figure 4. Analysis uses data for Dartmouth applicants in years 2017-2018 (test-required) or 2021-2022
(test-optional). Panels b and d display the probability of admission for the test-required cohorts (not including
recruited athletes), separately for 50-point bins of the SAT score. The SAT composite is included for all applicants
where reported. Where only ACT is available, scores are rescaled to the SAT scale. We define “less-advantaged”
students (panels a and b) as those who are any of: U.S. first-generation college going, from a neighborhood with
median income below the 50th percentile for the U.S., or attended a U.S. high school in the top 20% of the College
Board's index for challenge. We define “first generation” as U.S. first generation (panels c and d). Key takeaway:
Under test-optional, less-advantaged and U.S. first-generation college-going students with SAT scores around 1400
and below are more likely to be admitted to Dartmouth but not more likely to submit scores than their
more-advantaged counterparts.
13
Figure 6.
Admissions Rates of Dartmouth Applicants Submitting and Not Submitting Scores in the
Test Optional Cohorts, by SAT Score: By Advantage and First-Generation Status
a. Less-Advantaged Applicants
b. More-Advantaged Applicants
c. First-Generation Applicants
d. Not First-Generation Applicants
Notes: Charts display admission rates by whether applicants opted for SAT scores to be considered in the
application decision, separately for 50-point bins of the SAT composite score. Analysis uses data from Dartmouth
applicants in the test-optional years, 2021-2022. For these cohorts, we have SAT scores both for students who
submitted scores (blue lines) and for a small sample (19%) of applicants who chose to exclude their score from the
admission decision but for whom we observe their scores ex post (red lines). The SAT composite is included for all
applicants where reported. Where only ACT is available, scores are rescaled to the SAT scale. We define
“less-advantaged” students (panels a and b) as those who are any of: U.S. first-generation college going, from a
neighborhood with median income below the 50th percentile for the U.S., or attended a U.S. high school in the top
20% of the College Board's index for challenge. We define “first generation” (panels c and d) as U.S.
first-generation college-going students. Key takeaway: High-achieving less-advantaged and U.S. first-generation
college-going applicants lowered their Dartmouth admissions probabilities by opting not to submit a score
14
Figure 7.
Histograms of Composite SAT Scores in the Test-Optional Cohorts:
Subsample with Known but Non-Submitted Scores in the Test Optional Cohorts
a. Applicants From Less Advantaged
Backgrounds
c. Applicants From Less Advantaged
High Schools
b. Applicants From More Advantaged
Backgrounds
d. Applicants From More Advantaged
High Schools
Notes: Charts display histograms of SAT scores for the 19% of Dartmouth applicants in the test-optional years
(2021-2002) who opted not to submit scores but for whom we observe scores ex post. Where only ACT is available,
scores are rescaled to the SAT scale. We define “applicants from less-advantaged backgrounds” (panels a and b) as
those who are any of: U.S. first-generation college going, from a neighborhood with median income below the 50th
percentile for the U.S., or attended a U.S. high school in the top 20% of the College Board's index for challenge. We
define “applicants from less advantaged high schools” (panels c and d) as those who attended a U.S. high school in
the top 20% of the College Board's index for challenge. Key takeaway: There are large numbers of high-achieving
less-advantaged students who lowered their Dartmouth admissions probabilities by opting not to submit a score.
15
Figure 8.
Histogram of Neighborhood Income Decile:
Dartmouth Applicants in the Test-Optional and Test-Required Cohorts
Notes: This figure uses data for Dartmouth applicants in years 2017-2018 (test-required) or 2021-2022
(test-optional). The two test-optional cohorts are on average 35% larger than the two test-required cohorts (see
Table 2). Key takeaway: The neighborhood income distribution of Dartmouth applicants looks similar for the
test-required and test-optional cohorts.
16
Table 1.
Value of High School GPA and the SAT Composite in
Predicting Cumulative First-Year GPA at Dartmouth
(1)
(2)
(3)
(4)
(5)
(6)
First Year
GPA
First Year
GPA
First Year
GPA
First Year
GPA
First Year
GPA
Less Adv
First Year
GPA
More Adv
SAT Composite
0.00158***
0.00139***
0.00140***
0.00129***
0.00158***
Score
(4.64e-05)
(4.88e-05)
(6.56e-05)
(0.000132)
(5.31e-05)
HS GPA
0.550***
0.322***
0.220***
(0.0261)
(0.0263)
(0.0485)
Class Rank
-0.490***
(0.124)
Constant
1.271***
1.447***
0.304***
0.703***
1.630***
1.275***
(0.0687)
(0.101)
(0.103)
(0.197)
(0.185)
(0.0794)
Observations
4,051
4,507
3,937
1,920
580
3,471
R-squared
0.222
0.090
0.255
0.260
0.141
0.203
Notes: This table uses data for enrolled Dartmouth students who applied in 2017-2018 (test-required) or 2021-2022
(test-optional). We define “less-advantaged” students as those who are any of: U.S. first-generation college going,
from a neighborhood with median income below the 50th percentile for the U.S., or attended a U.S. high school in
the top 20% of the College Board's index for challenge. The SAT Composite is used for all applicants where
reported. Where only ACT is available, scores are rescaled to the SAT scale. *** Statistically significant at the 1%
level. Key takeaway: The SAT Composite offers more predictive power for first-year GPA at Dartmouth than high
school GPA.
17
Table 2.
Number of Applicants Overall and Opting Not to Use an SAT Score in the Admission
Decision, by Application Year
Application Year
Reported Score
Total
2017
20,067
20,067
2018
22,075
22,075
2019
23,691
23,691
2021
15,414
28,531
2022
15,493
28,367
Total
96,740
122,731
Notes: This table uses data for Dartmouth applicants in years 2017-2019 (test-required) or 2021-2022 (test-optional).
46% of test-optional applicants opted not to submit a score. In addition, the test-optional cohorts were on average
29% larger than the test-required cohorts. In the 2021 and 2022 cohorts, respectively, 2,416 and 2,544 applicants
reported a College Board score that they later requested not be considered by Admissions.
18