ADMINISTRATOR PERCEPTIONS OF SCHOOL IMPROVEMENT POLICIES IN A HIGH-IMPACT POLICY SETTING

This study investigated school administrators’ perceptions of school improvement policies in a high-impact policy environment by measuring the impact of accountability, site-based management, professional development, and scheduling reform on the three dependent variables of a) academic outcomes, b) staff morale, and c) parent and community involvement. Using a convenience sampling method, 49 public school principals from Texas participated and an online survey was constructed to gather both quantitative (i.e., Likert scale) and qualitative (i.e., open ended response) data. The findings clearly point to principals, regardless of geographical district type and grade level school type, viewing less controversial and more intrinsically oriented policies (i.e., site-based management and professional development) as having a greater positive impact on outcomes as a whole than more radical alternatives (i.e., accountability and time and schedule reform). The evidence suggests that more aggressive school improvement policy approaches are likely failing to generate enough convincing outcomes to generate high commitment and confidence from school leaders. Further studies may look at the interaction of policy impact with minority student enrollments and with subgroup populations.


Introduction
Principals are under a lot of pressure running their schools.One source of this pressure is the principals' responsibility for implementing state and district initiatives for school improvement.However, in the last two decades, the political climate has altered the policy emphasis from one based on ensuring adequate resources to one based on outcomes and accountability.In response, states have experimented with efforts such as site-based management (SBM), staff development, and schedule reforms in hopes of increasing student achievement.
Yet in contrast to the above reform proposals, the current wave in educational policy--emphasis on standards, testing, and consequences--has the potential to threaten the normative function and operation of public schools.In addition to state initiatives, principals have mandates by the federal government, most notably those associated with the No Child Left Behind Act (NCLB).While many states are embarking on systemic accountability measures, other states, such as Texas, are regularly accustomed to high-impact policy initiatives aimed at demanding minimum levels of productivity.There is a need to more fully understand the effect policy initiatives are having on principals and their schools in high-impact policy environments.

Purpose
The purpose of this study is to investigate school administrators' perceptions of the impact of school improvement policies.This study measures the impact of accountability, SBM, professional development, and scheduling reform on three dependent variables: a) academic outcomes, b) staff morale, and c) parent and community involvement.Because policy contexts differ greatly across states, Texas was selected for its lengthy and substantial contact with centralized policy initiatives.Texas is one of a handful of states that adopted ambitious, high controversy, high-visibility policies years before NCLB mandates were conceived.Some of the more notable features of the Texas accountability system predating NCLB included school ratings, publicized school, district, and state report cards, and tracking subgroup assessment performances.While many of these features exist presently in NCLB, Texas is a unique setting given its longer exposure to these requirements.This study assumes that Texas school administrators will have a greater amount of experience to provide a more precise assessment of policy impact.
The findings of this study can offer researchers, policymakers, and practitioners a greater awareness of the relative contributions of various policy initiatives.In an era where policy development appears to be mostly in the hands of policy elites, less attention is focused on the perceptions of the practitioner.In sum, the school reform discussion seems to demand a broader civic and professional engagement.Given the critical role school principals play in school improvement, an examination of their perspective seems critical if informed policy development is to occur.This study aims to gauge perceptions of the personnel in the most pivotal roles implementing traditional and not-so-traditional reform policies: school principals.

Policy Attractiveness
Although prior research has reported varying perceptions as to the benefit of school improvement policies (e.g., accountability), fewer studies have examined practitioner appeal by policy features.For instance, some contend teachers resist policies that impose changes to the normative operation of schools (Hess, 2003;Moe, 2003;O'Day, 2004).Others claim that high-impact poli-cies such as accountability foster greater organizational focus, clarity, and cohesiveness and have largely been embraced by teachers (Goertz & Duffy, 2003;Kelley, Heneman, & Milanowski, 2002).One study of the perceptions of principals in a high-impact state accountability program found that greater focus and attention to results paid dividends as far as test score improvement, but not without unanticipated outcomes.The findings of the study suggest that accountability seemed to diminish the importance of other worthy educational pursuits where no reward was available, as well as discouraged talented teachers from seeking employment in academically vulnerable schools (Ladd & Zelli, 2002).
Hess' research on policy attractiveness provides a promising framework from which to gauge policy appeal.Hess (1999) contends policy preference and selection can be explained through an interaction of two factors: a) policy visibility, and b) policy controversy.In a four qundrant matrix format, the possibilities can be high and low for both factors.For instance, according to Hess' survey of school district internal and external observers nationally (i.e., local education newspaper reporters, teacher union chiefs, a high profile school administrator, Chamber of Commerce heads, heads of minority organizations with most influence, and school board members), policies such as scheduling changes (i.e., time and day scheduling adjustments) scored low in visibility and high in controversy.Because scheduling reform lacks the captivating appeal of other more visible policies and tends to be disrupt the routine nature of schooling, superintendents are more inclined to select policies with greater visibility and less controversy, such as SBM, according to Hess.SBM generates a high degree of notice and less friction because, as Hess (1999) contends, "[SBM] is perceived by observers as the bigger and bolder gambit than is scheduling reform" (p.109).Hess emphasizes that although SBM appears to be ambitious and innovative, it is--ironically--less ambitious than scheduling reforms because of its less threatening nature.Because the impact of SBM has been largely "piecemeal" and "symbolic" according to Hess, the "grammar of schooling" (Tyack & Cuban, 1997) has remained relatively intact.
Student evaluation scored high on both controversy and visibility while professional development scored low on both.Accountability measures, such as student performance requirements set forth by the NCLB, would thus be placed in the high visibility, high controversy quandrant.Conversely, professional development generates little appeal and attention and little controversy according to Hess' s findings.Such activities in the form of benign "teacher inservices" and "staff workshops" rarely generate publicity and most importantly leave the status quo nature of schooling fully intact.Although it seems intuitive to hypothesize that school personnel would tend to also favor highly visible, less controversial policies over the contrasting case, this question has not been specifically tested on school leaders within a highimpact policy context.Moreover, because Hess limited the scope of the analysis to urban school districts, it would seem necessary to assess the perception of common school improvement reform efforts from a broader segment of stakeholders.

Method Sample
The study employed a convenience sampling method.Approximately 49 Texas public school principals participated in the study.Subjects selected were contacted by e-mail and or fax between mid-May and September 2005 requesting participation in the survey.Upon consent, the participants participated in the survey via an online survey system.As for the overall profile of the sample, 45 percent of the surveyed principals represented elementary schools, 20 percent middle school, and 25 percent high school.The remaining 10 percent served in schools where grade levels were consolidated (e.g., elementary and middle grades grouped together).Sixty-nine percent of the principals served schools in mid-sized cities or smaller, while the remaining led schools in large city environments.
The instrument was constructed in a Likert style format allowing for five possible responses: completely negative, mostly negative, mostly positive, completely positive, and not observed. 1The survey was organized in four sections, each representing a major school improvement policy area (national and state accountability, sitebased management (SBM), professional development, and schedule reform).Operational definitions were assigned to each of these policy areas.For the variable "national and state accountability," it was defined as "a national (e.g., NCLB) or statewide system (e.g., TEKS and the AEIS indicators in Texas or PSSA in Pennsylvania) that focuses on student performance."Site-based management was defined as "any initiative on the part of the state or local school district to provide teachers, parents, and communities greater participation and decision-making control over educational matters such as hiring personnel, budget, and academic related issues."Professional development was defined as "actions on the part of the state or local school district to provide teachers and staff opportunities to acquire knowledge and skills to inform practice (e.g., inservice activities, teacher/supervisor conferences)."Scheduling reform was defined as "changes made in school time arrangements on a daily, weekly, or yearly basis (e.g., extended day, block schedule, year-round school, etc.)."These policy areas were treated as independent variables assessing their impact on three dependent variables: a) academic outcomes, b) staff morale, and c) parent and community involvement (see Table 1).
Academic outcomes were defined as "attainment of evidences of academic success, including but not limited to performance on norm referenced and criterion referenced standardized tests, SAT and ACT scores, successful entry into community colleges and universities, enrollment and success in advanced placement classes, etc." Staff morale was defined as "the mental and emotional condition (as of enthusiasm, confidence, or loyalty) of teachers with regard to school tasks."Parent and community involvement was defined as "the degree to which parents are engaged in activities that make a significant positive impact on the school program."Improvements to the survey, which included calculating an estimated time needed for survey completion, were addressed in a pilot application.Appropriate modifications were made on the survey form, and the average

Method of Analysis
This study employed a mixed method analysis.In the quantitative portion, descriptive statistics were used to evaluate policy impact.In the qualitative portion, data analysis in the study was guided by a "grounded theory" approach to investigation.(author, 1993;Glaser & Strauss, 1967;and Strauss & Corbin, 1990).A set of data coding procedures called "open coding," "axial coding," and "selective coding" were used in (a) analyzing the written responses in the questionnaire, (b) defining specific themes, and (c) addressing research questions of the study (Strauss & Corbin, 1990).First, the researchers assessed overall policy perception calculating the mean Likert score for each dependent variable and composite policy area.The second stage assessed the relationship between policy perception and school district geographical type or metropolitan statistical area (e.g.urban v. mid-sized/small town/rural) again calculating the mean Likert score for each dependent variable and composite policy area.Metropolitan statistical area information for each partipant school was collected from the National Center for Education Statistics Common Core data files (2005).The eight area desigations were then collaped into binary form: a) large city and b) mid-sized and smaller.Stage three examined the association between policy perception and type of school by grade level using the mean Likert score (i.e., elementary, middle, and high schools).As for the five principals in the sample leading multigrade level schools, their responses were eliminated from this portion of the analysis to minimize bias.

Overall Perception of Policy Impact
The first stage of the analysis examined the overall perception of various policy areas on academic outcomes, staff morale, and parent and community involvement.SBM (3.15) and professional development (3.08) oriented policies fared most favorably among all principals surveyed (see Table 2, page 9).SBM had the greatest positive impact, particularly on academic outcomes (3.20).Accountability had the most negative impact on the whole (composite average, 2.55).Staff morale (2.24) suffered the lowest impact score among all dependent variables in the accountability category, but it also had the greatest impact from SBM (3.14) and professional development (3.08)--a difference of nearly .9 of a Likert point above the accountability indicator.
Assessing composite indicators on the whole, SBM and professional development fared approximately ,3 to .5 Likert points greater in positive impact than scheduling reform and accountability policies.

Perception of Policy Impact by Geographical Location of District
The second stage of the analysis examined the overall perception of the impact of various policy areas on academic outcomes, staff morale, and parent and community involvement using the geographical location of the principal' s school as a factor.SBM and professional development policies scored highest in both large city schools and mid-sized and smaller school contexts (see Table 3, page 10).In large city schools, principals rated SBM as having the most positive impact out of all policy areas, especially pertaining to academic outcomes (3.46).Accountability was perceived as the least effective overall impact policy in regard to academic outcomes in large city schools (2.93)--a difference of more than .5 of a Likert point.Non-large city school principals attributed academic outcomes more positively to professional development (3.24) than SBM (3.10).By contrast, they gave a much lower rating to the impact of accountability on academic outcomes (2.82).
Accountability negatively impacted staff morale more than all other policy areas in large city (2.00) and non-large city (2.35) schools according to principals.Principals in large city schools in particular rated SBM' s positive impact on staff morale nearly 1.3 Likert points greater on average than accountability (2.00).As for parental and community involvement, both large city and non-large city schools identified SBM as having the greatest positive effect on external participation while accountability was perceived as the greatest limitation to such involvement.Judging the composite indicators together, the differences in mean Likert averages between the two types of schools were far from dramatic.Accountability fared only slightly better in non-large city schools while principals in large city schools (3.30) favored SBM--exceeding the non-large city mean by only .2 of a Likert point.

Perception of Policy Impact by School Grade Level
The third and final stage of the analysis examined the overall perception of various policy areas on academic outcomes, staff morale, and parent and community involvement using the school grade level of the principal' s school as a factor.Similar to principal perceptions in contrasting geographical school contexts, SBM and professional development policies rated most favorably in impact among principals in each of the three grade levels (see Table 4, page 11).Professional development had its greatest impact on academic outcomes in elementary schools (3.45).Middle and high school principals saw SBM having the most beneficial impact on staff morale (3.38) and academic outcomes (3.18) respectively.For all grade levels, staff morale was most negatively impacted by accountability policies.SBM and professional development were also identified as having the most beneficial impact on parent and community involvement at all three levels.Overall, elementary school principals perceived greater positive policy impact than their counterparts.Principals' perceptions of policy impact declined on the whole as the school grade level increased.

Policy Impact Rank
The rank order of policy area based on positive impact reflects remarkably similar perceptions of principals despite geographical location and school grade level (see Table 5).In sum, principals identified SBM and professional development as having the greatest positive impact on the three outcomes.By contrast, schedule reform and accountability policies were perceived to offer the least benefit.SBM and schedule reform rated highest for principals in large city schools while professional development and accountability policies fared best for principals in elementary schools.As for the overall impact of the policy environment, elementary school principals (2.97) embraced policy intervention at a greater degree than principals for the two remaining grade levels (middle (2.84) and high (2.79)).The overall impact was nearly the same for large city (2.91) and nonlarge city (2.87) school principals.

Summary of Qualitative Data
The qualitative data collected in this study (see Table 6, page 12-15) reflect the overall quantitative findings (see Table 2, page 9) and add meaning.For example, the overall perceptions of the impact of SBM showed high positive mean ratings for each of the three dependent variables.Positive qualitative comments on the impact of SBM outnumbered negative comments more than 3 to 1 in terms of its impact on academic outcomes, nearly 3 to 1 in terms of its impact on staff morale, and somewhat less than 2 to 1 in terms of impact on parent and community involvement.A similar, though less pronounced, positive pattern of comments was found in relation to the impact of staff development.The comments on the impact of accountability and schedule reform were generally more balanced, though there are a greater number of negative comments in regard to the impact of accountability on teacher morale and a greater number of positive comments on the impact of schedule reform on academic outcomes.
The qualitative data also give insight into why the principals rated the items the way they did.Thirty-nine positive comments about the impact of accountability on academic outcomes centered around a greater focusing on student outcomes and the means for reaching those outcomes.By contrast, the negative comments were more scattered.Interestingly, accountability was seen as impacting staff morale positively by empowering teachers, but was viewed negatively in its impact on staff morale because it puts undue pressure on teachers.We obviously need to know more about what school conditions interact with what teacher characteristics to produce one type of impact or the other.
The comments about the impact of accountability on parents and community suggests a similar interaction.Site-based management is seen as having a positive impact on academic outcomes and teacher morale through its empowerment of teachers.It is also seen as having a similar impact on parents and community members.The greatest number of negative comments regarding SBM relate to the amount of time it takes for teachers and the unrealistic expectations it creates for parents and community members.
Positive comments about staff development indicate that, by improving teaching skills, it positively impacts academic outcomes, improves the teacher feelings of empowerment and self-efficacy, and engenders parental support and involvement.The smaller number of negative comments suggest that professional development often wastes teacher time and leads to teacher depression.Once again, we need to further explore what kinds of staff development interact with school and teacher characteristics to make positive impacts on outcome variables.
Similarly, schedule reform is seen by principals as having great potential for meeting student needs and thereby impacting academic outcomes, but it also creates the conditions for disrupting routines and causing anxiety among teachers and parents.In regard to the impact of schedule reform on teacher morale, time is seen as both a positive and a negative factor.Some principals believe that schedule reform allows a better use of teacher time while others feel that it uses too much time.Further investigation needs to be conducted to better understand how schedule reforms actually impact time and why their impact may be perceived differently among teachers and principals.
A more in-depth qualitative study of how these four types of reform (state accountability systems, site-based management, professional development, and schedule reform) actually work in a small number of selected schools would be a valuable venture for future research.Each type of reform means something different in each of the schools in which it takes place.School culture, individual teacher characteristics, and the particular configuration that the reform takes in a particular school or school district all impact the nature of the reform and affect the impact that it has.If meaningful progress is to be made in tracking the impact of reform, the interaction of these school and school district factors with the reform must be thoroughly understood.

Discussion and Implications
These findings show principals, regardless of geographical district type and grade level school type, viewing less controversial and more intrinsically oriented policies (i.e., SBM and professional development) as having a greater positive impact.This finding was not surprising in light of prior research suggesting school personnel select school improvement approaches that tend to cause minimal disruption to normative school practice and structure.On the other hand, schedule reform and accountability policies are typically perceived as burdensome and threatening because each aims to considerably alter the status quo (i.e., fundamentally transform the "grammar of schooling").The findings of the study seem to suggest this as well.
The evidence however seems to more importantly imply that more aggressive school improvement policy approaches are likely failing to generate enough convincing positive outcomes to gain and maintain commitment and confidence from school leaders.The findings are consistent with past research illustrating the precariouness of policies that are transformative and punitive in character.This correlation suggests principals see greater benefit from policies having greater human resource orientation (i.e., SBM and professional development).For instance, staff morale (2.24) overall fared poorly as a result of accountability but was impacted favorably by human resource-based policy (SBM (3.14) and professional development (3.08)).Hess (1999) mentions that policies geared to professional development or decentralized governance structures are less controversial and thus attract more political appeal.Non-controversial policy initiatives may not have much impact and can generally be accomodated within the existing culture of schools.But since they are so easily assimilated, they are also the ones that are least likely to produce needed changes in the schools.Policymakers who wish to improve schools are caught in the dilemma of initiating change that will negatively affect teacher morale, which in turn may destroy the purposes for which the policy was designed.This study of Texas principals tends to confirm Hess' material on policy attractiveness.
Another question is whether or not it is possible to design low visibility, high-impact policy initiatives in such a way that they will have high-impact.One potential difficulty that this study identifies is the fact that certain low controversy initiatives (e.g., SBM or staff development) have very loose definitions.There is evidence that high powered staff development initiatives may make a significant difference, but while it may boost teacher morale, staff development provides little evidence that it actually makes a change in academic out-comes.If all staff development activites were intense and followed with regular monitoring and feedback on performance, would those activities still be so overwhelmingly non-controversial?
If teachers were empowered to participate in the design and structuring of accountability strategies, would they then be less controversial?Perhaps one of the problems with the current accountability structure is that local school professionals have had so little input into the establishment of their goals and the specification of either the strategies by which they are attained or the criteria for their attainment.Is it possible that in attempting to change "the grammar of schooling," that we have not involved key stakeholders in the negotiation of that change?
The generally more positive impacts seen by elementary school principals raises questions: Does the level of the school make a difference in perceptions of positive impact or is it possibly the size of the school?Or is it the size of the school and the greater likelihoond of the presence of a cohesive learning community that makes the difference?The argument suggesting communities ought to be smaller in the 1996 Breaking Ranks report may be applicable here.The unique organizational, cultural, and social characteristics of high schools raises questions about differentiated programming, curricular scope and grade alignment, and student subject choice as well (Anagnostopoulos, 2003;Ingram, Louis, & Schroeder, 2004;Siskin, 2004).
These and other questions that arise from this study need to be further examined by both quantitative and qualitative approaches.Qualitative studies, using focus groups and participant observers, may help flesh out some of the reasons and assumptions behind the perceptions identified in this study.Further studies may look at the interaction of policy impact with minority student enrollments and with subgroup populations.The perception of high-impact needs to be compared to other measures of impact.This study needs to be replicated and extended as well.This study reports the perceptions of principals in one high-impact state.What might be found in other states?What would be found if a similar study were conducted with school superintendents?

Implications
Several implications can be made from this study: a) Less controversial, high visibility policies are perceived as having greater positive impact.The results of this particular study as well as the importance of understanding school improvement policy and its impact on perceptions can be used to: a) Guide policy makers in designing and structuring educational policy.b) Give researchers direction for further exploring the shaping and exploration of policy initiatives.c) Provide principals with a basis of understanding the impact and use of policy initiatives to improve schools.

Table 1 .
Summary of Variables Examined

Table 5 .
Policy Area Ranked by Composite Positive Impact * means for SBM and professional development impact are equal

Table 3 .
Policy Perception by District Geographical Location