95% confidence interval for the predicted effectiveness rate:

This range is a benchmark to help districts judge their comparative performance on postsecondary effectiveness. There is a 95 percent probability that, after accounting for influential risk factors, the true mean district effectiveness rate falls within this range.

If your district’s five-year effectiveness average is within or above the predicted range, then your district’s effectiveness rates can be consider average, or maybe better than average, when compared to similar districts. On the other hand, if your district’s five-year effectiveness average is below this predicted range, then its performance, when compared to similar districts, is either average or below average.

How do I know if my district is average, much better or much worse than similar districts?

This formula and example will give you a more precise estimate:

District A’s five-year effectiveness average = 54

The 95% confidence interval for its predicted effectiveness rate = 36.7 to 42.7

Step 1: estimate your district’s predicted effectiveness rate:

Lower bound + ((upper bound – lower bound) / 2)

For District A: 36.7 + ((42.7 – 36.7) / 2) = 39.7

Step 2: get the difference between the actual effectiveness rate and the predicted effectiveness rate

For District A: 54 – 39.7 = 14.3

Step 3: divide this difference by the standard deviation of the differences between the effectiveness rates and the predicted effectiveness rates (sd of the residuals = 6.51)

For District A: 14.3 / 6.51 = 2.19 sd units

Our calculation tells us that District A is about 2 standard deviations above its predicted effectiveness rate. After accounting for risk factors like students’ cumulative poverty, District A is performing well above average. While the categories below are somewhat arbitrary because we are dealing with a normal curve and a continuous distribution, we could categorize different performance levels like this:

• Far above average = ≥ 1.5 standard deviations (sd) above the mean or predicted rate
• Above average = > 1 sd but < 1.5 sd above the predicted rate
• Typical or average = ≤ 1 sd above and ≥ -1 sd below the predicted rate
• Below average = < -1 sd below and > -1.5 sd below the predicted rate
• Far below average = ≤ -1.5 sd below the predicted rate

Why did KSDE account for risk factors?

KSDE researchers used a set of risk factors and linear regression to predict the average postsecondary effectiveness rates for school districts. By including the risk factors as independent variables in the prediction, the variance of the predicted effectiveness rates is accounted for in the calculation. Once we have accounted for risk factors in the predicted rates, we can compare districts' actual effectiveness rates to their predicted ones. It is a way to compare all districts on their effectiveness rates after factors known to depress effectiveness rates, over which districts have limited influence, have been accounted for.

What were the risk factors included in the regression?

KSDE researchers used linear regression to identify factors that depress districts' postsecondary effectiveness rates. Based on this year’s data the significant detractors were (1) cumulative poverty, (2) cumulative student mobility, (3) cumulative chronic absenteeism, and (4) cumulative identification of disability. These four factors explained 54.3 percent of the variance of accredited districts' effectiveness rates. Students' cumulative poverty was by far the strongest detractor of effectiveness rates.

Cumulative poverty: The proportion of the students' school years spent in poverty. A school year in which the student spent any period of time eligible for free lunch was valued at one. If the student was not eligible for free lunch during the school year, but was for reduced-price lunch, the school year was valued at 0.5. For the selected five cohorts, these values were combined into the numerator. The denominator was the count of the total number of school years the selected five cohorts have attended Kansas schools.

Cumulative student mobility: The proportion of school changes during the school year. After selecting for the grades and years the five cohorts attended Kansas schools, the numerator is the number of times these students changed schools during a school year. The denominator was the count of the total number of school years the selected five cohorts have attended Kansas schools.

Cumulative chronic absenteeism: After selecting the five cohorts, the numerator was the count of student years in which students missed at least 10 percent or more of their days of membership in a school year. The denominator was the count of the total number of school years.

Cumulative identification of disability: The proportion of the students’ school years reported to have a primary area of disability. A school year in which the student was reported to have an active Individual Education Plan (IEP) which documents that the student received special education was valued at one. For the selected five cohorts these values were combined into the numerator. The denominator was the count of the total number of school years the selected five cohorts have attended Kansas schools.