MAKE YOUR DAY:

A SCHOOLWIDE CITIZENSHIP PROGRAM

Evaluation Report 2006

Elizabeth Vale

Michael T. Coe, Ph.D.

Center for Research, Evaluation, and Assessment

Dr. Kim Yap, Director

January 2006

Northwest Regional Educational Laboratory

101 S.W. Main Street, Suite 500

Portland, Oregon 97204

EXECUTIVE SUMMARY

“Make Your Day” (MYD) is a schoolwide citizenship program developed by Earl Brown.  This program promotes the development of internal locus of control in students by emphasizing human dignity, responsibility for one’s actions, and an understanding that all actions result in consistent reasonable consequences.

This study was conducted in fall 2005, to provide a preliminary view of the impact of the Make Your Day program and to begin developing instruments and methods for future research on the program. The evaluation research consisted of a review of existing data available from a small random sample of participating schools, as well as a limited collection of new data from these schools using surveys and interviews.

The small number of schools sampled do not necessarily comprise a representative sample of all schools using the program. Schools in the sample did represent a wide range of possible outcomes, primarily in terms of how well the program is implemented and accepted in different contexts. Some of the schools in this study had experienced high staff turnover, inconsistent implementation of the Make Your Day program, and less than ideal outcomes for students and teachers. Other schools had stronger implementations, greater commitment and consistency in applying the Make Your Day citizenship model, and stronger results. The findings, therefore, are most relevant to the question of which parts of the model are easier or harder to implement. For schools interested in the program, it is likely that full commitment to the model and consistency among staff members are keys to success. This report points out some of the potential challenges to successful adoption of the program, as well as some of the program features that are seen by participants as both easy to adopt and effective.

A detailed description of the program and the study methodology are included in this report, along with findings from individual schools, aggregate findings across all five schools, and recommendations for further program development and evaluation. The major areas of focus for the study, and highlights from the findings, are listed below:

Training and Support. School personnel gave high ratings to the Make Your Day program training. Not all respondents knew about other available support mechanisms such as follow-up visits and telecommunication from trainers, program newsletters, and the program Web site. In schools where high staff turnover undermines the effectiveness of the program over time, continued training and support may help to maintain consistency and successful use of the model.

Specific Program Components. One focus for the study was to provide information about the ease of implementation and perceived effectiveness of specific components of the program. Some of the core elements of the program, such as establishing a common “school rule,” using common language regarding discipline in all classrooms and common areas, establishing classroom expectations, and using the program “steps” were rated as being easy to consistently implement and effective. These features of the program were not likely to require modification by individual schools or teachers. Some of the other program components were rated as more difficult to implement, and were perceived as less effective; these may require further program development and/or more training and support.

Impact on Student Discipline Problems and Student Achievement. The survey and interview results showed that teachers and administrators believe the program is effective and helpful. As noted above, this perception varies when the program is broken down into specific components. In general, those program features that are considered easy to implement are also those most consistently used and most likely to be seen as effective. Impact also varied according to how thoroughly the program was actually implemented; one school with high staff turnover reported a greater level of problems with implementation and lower ratings of effectiveness.

Data on actual frequencies of discipline problems and levels of student achievement were inconclusive. Some schools in this sample were unable to provide detailed and consistent data on discipline problems and achievement from both before and after their use of the Make Your Day program. No clear pattern emerged, and in any case the retrospective, descriptive nature of the research design would not allow strong conclusions about cause and effect. Recommendations about how to address these issues in the future are included in the report.

Impact on School Climate and Parent Involvement. A majority of school staff members reported that their school is safer in general, and they as individuals were happier and more productive, as a result of the Make Your Day program. Most reported wanting to continue the program at their school. A majority also reported that as a result of the program, parents and school staff work together more effectively to handle student behavior problems. Other items addressing the school atmosphere, productivity, and relations with parents were given mixed ratings, particularly in the school with high staff turnover and inconsistent implementation of the program.

These findings can be used to inform further program development, as well as to provide additional recommendations to schools on how to successfully implement the program. Finally, the methods used here can serve as a basis for further evaluation research on the effectiveness of the Make Your Day approach.

SUMMARY ACROSS SCHOOLS

Individual school-level survey data was combined so that some general trends could be illuminated. Percentages were calculated by averaging percentages across schools. Tables 56-65 display these results.

Across all schools, survey responses indicated that:

· Respondents were highly satisfied with the quality, relevance, and amount of training they received in preparation to implement the program.

· Data on use of the MYD Web site and newsletter, phone and e-mail support, and yearly visits by trainers are confounded because many respondents were unaware that these support structures were available, or they did not access them. Therefore, the relatively low ratings for these five items may be due simply to lack of use of those features of the program, rather than lack of satisfaction. Future studies will use a branching structure to determine first whether respondents have participated in each of these support services provided by the MYD organization, then ask those who have accessed each service to answer questions about the level of satisfaction they experienced.

Table 56

Summary Across Schools of MYD Training and Support Offered by MYD Facilitators

 Across all schools, survey responses indicated that:

· Overall, most respondents agreed or strongly agreed that they were personally consistent and through in applying the MYD components.

· Staff members were the most personally consistent and thorough in applying the program components involving using common language and having students take responsibility for their behavior choices. Other components that staff members also indicated they were consistent and thorough in implementing were Steps 1–3 and establishing the School Rule.

· Staff members were least likely to agree that they were personally consistent and thorough in sending home a note to parents when the student does not “Make his/her Day” and following up with further communication if the note does not return.

Table 57

Summary across Schools of Consistency of Implementation of MYD Program Components

 Across all schools, survey responses indicated that:

· Overall, only a modest number of respondents indicated that they had to make modifications to the MYD components.

· Modifications were made most often to “Sending home a note to parents when the student does not “Make his/her Day” and following up with further communication if the note does not return,” “Student self-evaluation when assigning points,” and “Steps”.

Table 58

Summary Across Schools of Modification of MYD Program Components

 Across all schools, survey responses indicated that:

· Most of those surveyed agreed or strongly agreed that most of the MYD components were easy to implement.

· “Establishing the school rule” was most often cited as the easiest MYD component to implement.

· The components: “Sending home a note to parents when the student does not “Make his/her Day” and following up with further communication if the note does not return,” “Student self-evaluation when assigning points,” and “Student expression of concerns” were least often cited as the easiest components to implement.

Table 59

Summary Across Schools of Ease of Implementing MYD Program Components

 Across all schools, survey responses indicated that:

· Overall, most respondents agreed that their school has been effective in implementing most MYD components.

· “Establishing the school rule” was most often cited as the component schools were most effective in implementing.

· “Sending home a note to parents when the student does not “Make his/her Day” and following up with further communication if the note doesn’t return” was the component that respondents were least likely to agree that their school was effective in implementing.

Table 60

Summary Across Schools of Effectiveness of Implementation of MYD Program Components

Across all schools, survey responses indicated that:

· Overall, the majority of respondents agreed that most MYD components have caused a significant decrease in student disciplinary problems.

· The “Step 4—Parent conference” was most frequently cited as the component responsible for a significant difference in student disciplinary problems.

· The component respondents were least likely to agree lead to a significant decrease in disciplinary problems was “sending home a note to parents when the student does not “Make his/her Day” and following up with further communication if the note doesn’t return.”

Table 61

Summary Across Schools of Impact of MYD Program Components on Student Disciplinary Problems

 Across all schools, survey responses indicated that:

· Overall, almost half of those surveyed believed that MYD had a significant impact on student achievement.

Table 62

Summary Across Schools of Impact of MYD on Student Achievement

Across all schools, survey responses indicated that:

· Overall, a modest number of those surveyed indicated that parent involvement, support, and attitudes were positive because of MYD.

· The interface of parents with the MYD program is an area where improvements can be made.

Table 63

Summary Across Schools of Impact of MYD on Parent Involvement/Support/Attitudes

 Across all schools, survey responses indicated that:

· Many staff members agreed that aspects of their school climate had improved because of MYD, in particular “school safety” and the “work environment.”

· Respondents were less likely to agree that “teacher self-esteem”, “teacher collaboration within and across grade levels” and “student self-esteem” had increased because of MYD.

Table 64

Summary Across Schools of the Impact of MYD on School Climate

Across all schools, survey responses indicated that:

· Generally, the majority of those interviewed indicated that they hope their school continues with MYD next year and that they are happy that their school has adopted MYD.

Table 65

Summary Across Schools of Satisfaction with the MYD Program

RECOMMENDATIONS

The findings in this report may be helpful in further program development as well as assisting current and future Make Your Day schools to better implement the program and achieve better results. Each of the case studies presented here, as well as the data aggregated across all five schools, illuminate strengths and potential weaknesses of the program that may be encountered by participating schools. The following list highlights some recommendations based on these findings.

n     The role of ongoing support from Make Your Day trainers, in the form of regular retraining, site visits, newsletters, and online communications with schools, may need more emphasis, particularly in schools where staff turnover threatens to erode the program after initial training and implementation.

n     Program components which received low ratings for consistency, ease of implementation, and effectiveness may need to be modified, and/or may need additional emphasis during initial training and ongoing support. These program features also were among those most likely to be altered by individual schools or teachers. They include sending notes home to parents, student self-evaluation when assigning points, and student expression of concerns about the behavior of other students.

n     Parent understanding, involvement and support of the program might also be a worthwhile focus for more emphasis in program development and implementation. These are difficult issues for any school or school program, but might be an area in which more (or more effective) communication could yield better results. The scope and design of this study did not provide conclusive evidence about program effects on discipline problems or student achievement. Additional research could be undertaken to provide such evidence. This could be done using an approach similar to that used here, but with refined measures and a larger sample of schools. Alternatively, an experimental or quasi-experimental design would provide an even more accurate view of program effectiveness.

n     Schools that were randomly sampled for this study generally had data on student discipline problems that were of limited use for tracking change over time before and after their use of the Make Your Day program. This makes it difficult for school personnel or outside evaluators to generate even a descriptive, post-hoc assessment of how the program and other school factors may have affected discipline issues. It may be worth exploring the development of an additional program component which would provide guidelines, resource materials, and consultation to assist schools with better collection and management of student discipline data, so they can make more informed school management decisions.

Send mail to clarnold@ix.netcom.com with questions or comments about this web site.
Copyright © 1981-2005 MYD, Inc.  Make Your Day - Schoolwide Citizenship Program
Last modified: 07/28/06