Where’s The Overlap?
Mapping the SAT9 and TAAS
4th Grade Test Objectives
Lesli Hoey, BA
Patricia B. Campbell, PhD
Lesley Perlman, BS
(This docment is also available in Adobe Acrobat (ver. 4 and above .pdf).)
As districts and states implement testing, there is value in determining similarities and differences across the tests used. The following example explores the overlap in the math and science objectives of two tests used by districts participating in a study of effective schools.
Introduction
As part of the Effective Schools Study currently underway, analysis is being done on fourth grade student achievement in the four participating districts(1). In the School District of Philadelphia, fourth grade student achievement is measured by the Stanford Achievement Test (SAT9): Intermediate I, while in the El Paso, Socorro and Ysleta Independent School Districts, student achievement is measured by Texas’ statewide test, the Texas Assessment of Academic Skills (TAAS). The TAAS was developed to assess the degree to which students meet the objectives stated in the Texas Essential Knowledge and Skills (TEKS).
To explore the degree of commonality across the two tests, the TEKS: TAAS fourth grade math and science objectives were mapped to the math and science objectives of the SAT9: Intermediate I, and the SAT9: Intermediate I math and science objectives were mapped to the TEKS: TAAS fourth grade math and science objectives.
The Method
TEKS: TAAS objectives were first mapped against the SAT9; then, separately, the SAT9 objectives were mapped against the TEKS: TAAS objectives. This was done because rarely are individual objectives similar enough to match each other exclusively or totally. For example, the SAT9 objective, Read a weather chart, is partially matched by the following two TEKS: TAAS objectives: Identify patterns of change such as in weather, metamorphosis, and objects in the sky and Construct graphs, tables, maps, charts to organize, examine, evaluate information. Objectives from one test may partially match several different objectives of the other test because they are more specific or more general or because they cover similar topics but ask for different skills. The above example is a partial match because: A weather chart is more specific than charts in general; to read a chart can be different than to construct one, and to identify patterns of change such as in weather may not call for reading a weather chart.
The Results
Overall, there is a great deal of overlap in the content of the objectives of the TEKS: TAAS and the SAT 9.
Math
As Table 1 indicates, almost 3/4 (74%) of the SAT9 math objectives are either partially or totally matched to the TEKS: TAAS math objectives, while 83% of the TEKS:TAAS objectives are totally or partially matched by SAT9 objectives.
Table 1. Overlap in SAT9/TEKS: TAAS Math Objectives
SAT9 Math (N=51)
TAAS Math (N=40)
Total Match
22/43%
17/43%
Partial Match
16/31%
16/40%
No Match
13/25%
7/18%
The TEKS: TAAS objectives fall into six general categories while the SAT9 objectives fall into 12 general categories. Some general areas are covered by both tests as are Measurement and Statistics and Probability and most have significant overlap (i.e. the TAAS category Numbers, Operations, and Quantitative Reasoning - Fractions and Decimals is similar to the SAT9 category Fraction and Decimal Concepts, while the TAAS Geometry and Spatial Reasoning category is similar to the SAT9 Geometry and Spatial Sense category).
Science
A similar process was used to map the 4th grade TEKS: TAAS science objectives to the science objectives of the SAT9: Intermediate I. As seen in Table 2, overall overlap in the content science objectives is also high, though there are fewer ‘total’ matches. Ninety percent of the SAT9 science objectives were either partially or totally matched to the TEKS: TAAS science objectives, with 23% of the SAT9 objectives being totally matched by TEKS: TAAS science objectives. Seventy-one percent of the TEKS: TAAS science objectives were either partially or totally matched by SAT9 objectives, with 19% being totally matched.
Table 2. Overlap in SAT9/TEKS: TAAS Science Objectives
SAT9 Science (N=39)
TEKS Science (N=31)
Total Match
9/23%
6/19%
Partial Match
26/67%
16/52%
No Match
4/10%
9/29%
Unlike math, the general areas into which science objectives are categorized are quite different for the SAT9 and the TEKS: TAAS. While the SAT9 lists science objectives under different scientific disciplines (i.e. Life Science, Earth and Space Science, Physical Science), the TEKS: TAAS lists science objectives according to general scientific skills or concepts that are necessary to understand science (i.e. Scientific Inquiry, Complex Systems and Parts, Critical Thinking, Problem Solving and Decision-making). Too, many SAT9 science objectives have multiple components, making it difficult to find ‘total’ matches. For example, the TAAS objective Construct graphs, tables, maps, charts to organize, examine, evaluate information, remains a partial match even when mapped with five related SAT9 objectives: Analyze a graph about sound, Read a weather chart, Read a weather map, Use a graph to predict weather and Read a graph about motion. It is important to note that currently there is no TEKS: TAAS science test given at the fourth grade level. Such a test is being piloted and it is expected that it will be administered statewide in the near future.
In Conclusion
The matching was done on the objectives of the two tests, not on the items themselves. Thus any conclusions on the degree of overlap between the two tests are based on the assumption, not checked in this analysis(2), that the items reflect the objectives. If this is the case, then it appears that in fourth grade mathematics, the two tests are measuring students’ achievement over approximately the same content. Over 40% of the mathematics objectives of both tests are fully matched, with only seven of 40 TEKS: TAAS objectives not being at least partially covered by SAT9 objectives and 13 of the 51 SAT9 objectives not being at least partially covered by TEKS: TAAS objectives. While there is not enough overlap to predict individual SAT9 math results based on TEKS:TAAS scores, it does seem clear that there is enough overlap to assume that classes or schools that score high on one test would most likely also score high on the other.
(1)The Effective Schools Study, funded by the National Science Foundation, is being conducted by the Urban Institute and Campbell-Kibler Associates in conjunction with the School District of Philadelphia and the Philadelphia Urban Systemic Program and the three El Paso area districts that make up El Paso Urban Systemic Program, El Paso Independent School District, Socorro Independent School District and Ysleta Independent School District.
(2) It was not possible to get the access to the test items themselves, which would be necessary for such an analysis to be done.
Production of this material was made possible by a
grant from the National Science Foundation. Opinions expressed are those of the
authors and not necessarily those of the funders.
Campbell-Kibler Associates, Inc. 80 Lakeside Dr., Groton, Ma. 01450, campbell@campbell-kibler.com, www.campbell-kibler.com
© 2001, Campbell-Kibler Associates, Inc.. All Rights Reserved