NCEA

The National Center for Educational Achievement (NCEA), a department of ACT, helps educators build capacity within their school systems and provides research-based resources to support them in their school improvement efforts. NCEA's Higher Performing Schools List recognizes schools that are raising student achievement levels more quickly than schools that enroll similar student populations.

This overview outlines NCEA's approach for identifying higher performing schools. The final selection of schools also incorporates individual state's AYP and accountability information, which varies by state.

General Approach

The guiding question of NCEA's analysis is: Which schools are preparing students better than expected? In order to answer this question, NCEA employs the statistical technique of Hierarchical Linear Modeling (HLM). HLM is commonly used to identify the value-added effects of schools. The model is based on the school's student performance on the state assessment and is run separately for each tested subject. Typically, these subjects include reading, mathematics, and science. In addition, the model controls for school demographics.

Additionally, the NCEA methodology focuses on a value-added analysis of the schools based on prior-year student achievement. The value-added analysis examines whether students enrolled for the full academic year (in that particular school) have performed above what was predicted, based on their state assessment scores from the previous year. For Grade 3, prior-year achievement scores are not available in the initial tested grade-level, so NCEA adjusts its analysis to include continuously enrolled students--those students who have been enrolled in the school or district for a minimum of three years.

Due to ceiling effects on state assessments, the amount of growth students performing at the top of the test scale can show is often restricted. For this reason, NCEA identifies a school as higher performing based on the school's continuously enrolled student performance on the state test's achievement level that falls closest to NCEA's College and Career Readiness (CCR) Targets. In addition, NCEA controls for the percentage of students who are eligible for free or reduced lunch.

Methodology

NCEA recognizes a school as higher performing across all tested grades per subject. The steps in this analysis are as follows:

Step 1: Dataset creation

Each student's current-year test results are merged with their math and reading scores from the previous year. For example, Grade 8 mathematics scores are matched with Grade 7 mathematics and reading results.

Fall enrollment records are also used to identify whether the student has been enrolled at the same school for the full academic year and/or has been continuously enrolled. Students without prior-year scores in the relevant subjects or who were not enrolled for long enough, were excluded from the analysis.

If the school's grade span does not allow for three years of continuous enrollment (e.g., a 7th grader at a Grade 6 to 8 campus), continuously enrolled students are those who have been enrolled in the district at least three years and at the school as long as the grade span allows. For charter schools, this last requirement changes to drop the continuous enrollment in the district requirement.

Step 2: Performance measure

For each subject, every student's converted scale score (z-score) is regressed in each grade and year on the following variables:

• The student's prior-year z-scores in mathematics and reading
• Whether the student is eligible for free or reduced lunch
• Whether the student is African--American or Hispanic
• The percentage of students that:
  • Were tested,
  • Were present for the full academic year,
  • Had prior test scores in math and reading (except for the initial tested grade),
  • Qualified for free or reduced lunch,
  • Were English language learners (for elementary schools only),
  • Were African-American, and
  • Were Hispanic
• Whether the school is a magnet school

SAS software is used to fit the HLM model and to calculate--for each school in each subject--a separate set of grade-and-year school measures and a combined school effect across grades and years.

In addition, the percentage of students at or above the achievement level closest to the CCR Targets is compared to the percentage eligible for free or reduced lunch.

Step 3: Identification of eligible schools

Schools are eligible for ranking on a given subject if they have an adequate number of qualifying students in each of their grade-levels for the most recent school year. Furthermore, a school must have no more than the maximum allowable number of missing grade-levels (shown in Table 1 below) for the previous two school years in order to be eligible for the analysis.

A "grade-year combination" is one grade for a specific year. For example, Grade 3 in 2010 is one grade-year combination.

A grade is considered "missing" for a given school year if either:

1. Test score records are available for fewer than 10 qualifying students
2. More than 20% of the records in that grade are deleted based on masking rules applied by the state

Step 4: School rankings

After eligibility is determined, schools are divided into two low-income groups: 0-50% low-income and 50-100% low-income. The schools are then ranked within each of these low-income groups by the performance measures calculated in Step 2.

Step 5: School selection

In addition to requiring that the school's value-added performance be in the top 10% of its low-income group and have better than predicted performance across grades in all years or have performed in the top 5% of its low-income group on the achievement level closest to the CCR Targets, NCEA also includes the following requirements:

1. Met Adequate Yearly Progress (AYP) or other ESEA requirements.
2. Had 90% of enrolled students take the state assessment in non-AYP subjects.
3. Met their state's requirements on the accountability system.