Growth Models: CCSSO Summary of Questions and Answers

The purpose of this document is to assist states in their planning and response to the opportunity for submitting growth model proposals for school accountability under NCLB, as outlined in a November 21, 2005 "dear colleague" letter from U.S. Secretary of Education Margaret Spelling. The following list represents an ongoing process of identifying key questions and answers that have emerged in our shared discussions with states about developing growth models that can contribute to the achievement of the goals of NCLB by “measuring school accountability, giving schools credit for improvement over time, and measuring individual student progress” (Secretary Spelling’s Letter). The answers represent our best thinking in understanding the intent of this opportunity, but it is not official guidance from the U.S. Department of Education (ED). Interested states and stakeholders are strongly encouraged to review these questions and answers and to send additional questions and/or comments to Rolf Blank, CCSSO, rolfb@ccsso.org Growth Models in Comparison to Improvement/Status Models What are the essential differences between growth models vs. improvement/status models? The questions asked and answered by these models are different. Improvement/status models focus on the school ‑ how well is the school educating the students it receives? Is the school doing better at educating the students it has this year than last year? That is, improvement/status models are explicitly focused on school improvement and school performance. Growth models also address school performance, but are based on a different question ‑ is the school helping students learn more each year? Is the learning going to get the student to proficient at the end of a specific grade? Growth Models as “an Additional Safe Harbor” Is it possible to use a growth model as an additional safe harbor? A state would continue its current NCLB AYP implementation and add an additional growth model AYP option. It is not clear if ED will approve adding growth models as a third option to status and safe harbor approaches under the statute. The state would need to justify the educational sense of this proposal; for example, using the third option to avoid false negative AYP determinations. ED will likely want to closely examine data on the impact on AYP determinations of growth as a third option. It may be interested in the use of growth models as a third option to differentiate possible sanctions for schools found to be missing AYP under the status and safe harbor models Vertical Scaling Do assessments have to be vertically aligned? Is vertical scaling essential for consideration of a growth model proposal? ED has clarified that the intent is to know that observed growth is due to an increase in the student's knowledge and skills and not to a change in the test scale. Vertical equating is one way to provide evidence that the growth is real, but there may be simpler ways to validate growth (e.g. equating for each grade pair separately or using expert judgment to establish growth targets.) While vertically scaled assessments are not an absolute prerequisite, the proposal would have to be peer reviewed and convince experts of the ability to measure growth from one grade to the next in a coherent way. ED did not specify what it means to have linkage from grade to grade. Vertical equating is one way to provide evidence that the growth is real, but there may be simpler ways to validate growth (e.g. equating for each grade pair separately or using expert judgment to establish growth targets.) Setting Growth Targets What amount of growth is acceptable in the calculation of AYP? Will this vary state by state? What will the impact be of one state allowing a student 3 years to achieve proficiency, and another state 5 years? The amount of time provided for students to achieve proficiency under a growth model may vary in different states. States need to make the decision based on what is educationally appropriate for their state. Nothing is off the table. However, states will be in a stronger position to be approved if they measure growth towards proficiency in a reasonable amount of time, such as three years or the time at which a student graduates from his school. States also may want to examine possible use of “averaging” so that a student generally on a path to proficiency is not scored negatively for a temporary set-back. The rationale for averaging is that student learning is not linear, and is often characterized by fluctuations within a specific period of time. Averaging student progress adjusts for this fluctuation and gives a more accurate measure of the growth in student learning. Growth Trajectories Do states have to set individual trajectories by subgroup? One of the things off the table is generating an expected performance on the basis of student or school demographics or background characteristics. If expectations are based simply on individual trajectories needed to make certain levels of progress, this is not off the table Establishing separate trajectories for each subgroup is not required. Generally, to measure growth, the progress of individual students toward proficiency is measured, based on a target of when the student should be proficient; for example, in 3 years or by the time the student graduates from his or her school. In effect, there is a separate trajectory for each student 100% Proficiency Goal for 2013-14 Is there an expectation of all students proficient by the close of the 2013-14 school year? Core principle #1 in the Secretary's letter is that the accountability system must ensure that all students are proficient by 2013-14 and that annual goals ensure that the achievement gap is closing for all groups of students. The proposal needs to show that the system will maintain fidelity to that principle. This does not mean in practical effect that that all lines lead to proficiency by 2013-14. The current safe harbor provision, for example, can be met without all students being proficient by that year. The Secretary's letter expressly permits growth models that establish a “point in time goal” – for example, within three years – to move the student to the proficient level. A growth model could, for example, require annually increasing percentages of students to be proficient or on a pathway to proficient, with reference to that point in time goal. However, as noted, the proposal needs to demonstrate fidelity to the goal of all students being proficient by 2013-14. The clearest way to do that may be to use a growth model as a substitute for safe harbor and continue to use the status model that calls for all students to be proficient by 2013-14. Data Systems and Issues Do you need to have two years of data for every tested grade? What are the expectations in regard to how states handle the problem of “missing data,” in which the data for the same cohort narrows over time due to factors such as student mobility? The states that have assessed in all NCLB grades (3-8, HS) in 2004-2005 are clearly well-positioned to have growth models for 05-06. For states that performed field tests during 2004-2005, whether they have 2 years of data depends on the characteristics of the field test such as type of fit, standards set, extent of sample; some of these states may also be ready for growth model. But, while ED is looking for a model that can assess growth across all tested grades, states that have two years of data for only some of the tested grades, e.g., Grades 4 and 7, are still encouraged to submit a proposal, as meeting the minimum requirements of this pilot project. Again the strength of the proposal rests on the educational foundation for the growth model and how well it addresses the core principles detailed in the Secretary’s letter. States are required to have as a core element of a growth model a data system that tracks student progress from grade to grade (e.g., the use of individual student identifiers). The proposal should clearly demonstrate how its data system manages this problem in terms of intra-district and intrastate student mobility. However, tracking students between states is still a challenge to be addressed. Issue of 3rd Graders in Growth Models with Grades 3-8 Assessments Given 3-8 testing, how can third graders be included in a growth model since they will have only one test score? The Department has spent much time considering the issue of 3rd graders and does not have specific answers for how to include them. It has invited the states to address this issue. One answer may be a hybrid that maintains the status model and safe harbor approaches at the 3rd grade level and uses growth in conjunction with one or the other of these approaches at other grade levels. States may have other creative ways to include these students. Growth Proposals for High School Please describe what growth models would look like in high schools? ED has informally indicated that it would consider for approval proposals that look at growth for 3-8 and maintain current AYP approaches for high school. However, we would suggest that states consider possible pilot projects at the high school level as well ED has announced that high school reform is a major priority area. It may be very interested in proposals that promote that goal in ways that may complement and reinforce a growth model at the earlier grades (for example, looking at multiple measures and college-ready graduation rates). Standards and Assessment Peer Review Several states had their assessment systems peer reviewed and some states will be reviewed in February. Is “deferred approval” of assessment system at the time of submission sufficient to qualify for consideration of a proposal? What’s the timeline on getting the peer review decisions? And submitting information for final approval? The Department has clarified that ED will consider states with "full approval", "approval with recommendation" and possibly "deferred approval" provided the reason for deferral (such as the lack of a technical manual) can be remedied quickly. Reviews of assessments systems and other high priority AYP work at the Department are concentrated in relatively few people. Ideally, they want to get through assessment reviews in early spring. In further conversations with ED, it has been determined that states interested in submitting a proposal but scheduled for peer reviews in May (there are about 6-7 states) should contact ED as soon as possible to move these peer reviews up in the schedule. Regarding peer review process, is there a well-developed rubric at this point for evaluation of these proposals? What will the peer review process be like? ED has stated that the details of the peer review process are still being worked out. But it will be similar to the accountability and assessment reviews; they will include outside experts and state assessment, accountability, and data analysis staff. States are encouraged to send nominees for peer review, as well as any ideas for the evaluation, to the Department. Evaluation of Growth Model Pilot Projects What are the plans for evaluation of the pilots? There will be some kind of summary report of the findings of the pilots. The states with pilot projects will be announced as they are approved and what their model is doing.

Council of Chief State School Officers
One Massachusetts Avenue, NW · Suite 700
Washington, DC 20001-1431
voice: 202.336.7000 · fax: 202.408.8072

Legal Conditions · Feedback · Sitemap

document last updated 8/21/2009