Translating Knowledge From an Evaluation Report

profileYaya0625
Reading2week11.pdf

Research Article

Process Evaluation of a Positive Youth Development Program: Project P.A.T.H.S.

Ben M. F. Law 1

and Daniel T. L. Shek 2,3,4

Abstract There are only a few process evaluation studies on positive youth development programs, particularly in the Chinese context. Objectives: This study aims to examine the quality of implementation of a positive youth development program (Project Positive Adolescent Training through Holistic Social Programs [P.A.T.H.S.]) and investigate the relationships among program adherence, process factors, implementation quality, and success. Method: Process evaluation of 20 Secondary 3 classroom- based programs was conducted in 14 schools. Results: Overall program adherence, individual evaluation items, quality, and success had high ratings. Principal components analysis showed that two components, namely, implementation process and implementation context were extracted from 11 evaluation items. The correlational analysis indicated that program adherence, implementation process, and context were highly correlated with quality and success. Multiple regression analyses show that teaching process and program adherence predicted quality, whereas teaching process, teaching context, and program adherence predicted success. Conclusions: The implementation quality of the Tier 1 Program of Project P.A.T.H.S. was generally high.

Keywords Project P.A.T.H.S, process evaluation, positive youth development program

Introduction

Social work programs are specific sets of strategies and actions

that can be implemented to enhance social functioning and

problem-solving capabilities among individuals, families, and

groups. Program evaluation is a systematic assessment of the

process and outcomes of the programs with the aim of contri-

buting to the improvement of the programs, such as in deciding

whether to adopt the program further, enhancement of existing

intervention protocols, and compliance with a set of explicit or

implicit standards (Zakrzewski, Steven, & Ricketts, 2009).

This article documents the process evaluation of a large-scale

positive youth development program in Hong Kong called Pos-

itive Adolescent Training through Holistic Social Programs

(P.A.T.H.S.).

Process Evaluation in Prevention Science and Social Work Practice

Outcome evaluation focuses mainly on the results of the

programs, whereas process evaluation is concerned with how

the program is actually delivered (Dane & Schneider, 1998;

Domitrovich & Greenberg, 2000). Process evaluation is widely

adopted in prevention science, such as nursing care (Huryk,

2010; Painter et al., 2010), chronic illness prevention programs

(Braun et al., 2010; Karwalajtys et al., 2009; Mair, Hiscock,

& Beaton, 2008; Shevil & Finlayson, 2009), smoking cessation

programs (Gnich, Sheehy, Amos, Bitel, & Platt, 2008; Kwong

et al., 2009; Quintiliani, Yang, & Sorensen, 2010), dietary

programs (Allicock et al., 2010; Bowes, Marquis, Young,

Holowaty, & Issac, 2009; Hart et al., 2009; Muckelbuer,

Libuda, Clausen, & Kersting, 2009; Salmela, Poskiparta,

Kasila, Vahasarja, & Vanhala, 2009), and AIDS rehabilitation

programs (Bertens, Eiling, van den Borne, & Schaalma, 2009;

Fraze et al., 2009; Hargreaves et al., 2009; Konle-Parker, Erien,

& Dubbert, 2010; Mukoma et al., 2009). In social work prac-

tice, process evaluation has been used in family programs

(Cohen, Glynn, Hamilton, & Young, 2010; Kumpfer,

Pinyuchon, de Melo, & Whiteside, 2008) but is not commonly used

in youth programs (Beets et al., 2008; Frazen, Morrel-Samuels,

1 Department of Social Work and Social Administration, The University of

Hong Kong, Hong Kong 2 Department of Applied Social Sciences, The Hong Kong Polytechnic

University, Hong Kong 3 Public Policy Research Institute, The Hong Kong Polytechnic University, Hong

Kong 4 College of Medicine, University of Kentucky, KY, USA,

Corresponding Author:

Ben M. F. Law, Department of Social Work, The Chinese University of Hong

Kong, Hong Kong

Email: [email protected]

Research on Social Work Practice 21(5) 539-548 ª The Author(s) 2011 Reprints and permission: sagepub.com/journalsPermissions.nav DOI: 10.1177/1049731511404436 http://rsw.sagepub.com

Reischl, & Zimmerman, 2009; Johnson, Lai, Rice, Rose, &

Webber, 2010).

Process evaluation consists of five components, namely,

program adherence, implementation process, intended dosage,

macro-level implication, and process-outcome linkage

(Scheirer, 1994).

Program adherence deals with whether the program is being

delivered as intended according to the original program design.

It is an important factor affecting the quality of program imple-

mentation (Domitrovich & Greenberg, 2000; Fagan, Hanson,

Hawkins, & Arthur, 2008). True program fidelity is not easily

achieved because program implementers often change or adapt

the program content during actual implementation, whether

intentionally or otherwise. Studies have shown that a number

of preventive programs do not follow the prescribed program

content entirely, and adaptation is made to specific target

groups (Elliot & Mihalic, 2004; Nation et al., 2003). A study

has found tension between the desire of the program implemen-

ter to adhere to the manualized plan and to make adaptations in

accordance with the needs of clients (Wegner, Flisher,

Caldwell, Vergnani, & Smith, 2008). Although it is not an

easily resolved issue, program fidelity is generally encouraged,

especially when programs are designed with vigorous trial runs

and repeated success rates (Griffin et al., 2010; Johnson et al.,

2010; Wilson et al., 2009).

Process factors are those that can be observed during the

implementation process and are contingent to implementation

quality or success. There is a variety of process factors accord-

ing to the program characteristics and the needs of program

developers. Some programs even design their own process

measurements (Yamada, Stevens, Sidani, Watt-Watson, & de

Silva, 2009). There are two main groups of process indicators

in prevention science and social work programs. First is the

implementation process. It is the direct observation of the inter-

action between the program implementer and the program

receivers, such as the program receivers’ engagement and the

program implementer’s use of feedback. Second is the imple-

mentation context. It involves context factors critical to imple-

mentation, including goal attainment and background

knowledge, such as the program implementer’s familiarity with

the program receivers and the program implementer’s program

preparation.

Program dosage refers to the effort by program implemen-

ters to follow the required time prescribed for a program, as

inadequate time affects the quality of program implementation

(Bowes et al., 2009; Johnson et al., 2010). Dosage also refers to

the group size of program receivers. A discrepancy between the

intended and actual program receiver to program implementer

ratio affects the program delivery process (Frazen et al., 2009).

Process evaluation can provide important findings with

macro-level program implications, such as the importance of

engagement of different community stakeholders (Carswell,

Hanlon, O’Grady, Watts, & Pothong, 2009; Zani & Cicognani,

2010), client needs (Kwong et al., 2009), assessment of the

environment (Eisenberg, 2009; Stewart, 2008), and challenges

of the programs for a particular context (Louis et al., 2008).

Process evaluation and outcome evaluation are strongly

linked. Process evaluation sheds light on which types of inter-

ventions strategies or process are related to the program success

(Kwong et al., 2009; Painter et al., 2010). These factors can be

amplified during program reimplementation.

The components of process evaluation point toward its

importance. First, outcome evaluation provides inadequate

hints on the quality of program implementation. Process eva-

luation demystifies the ‘‘black box’’ of intervention and aids

in the understanding of the elements of program success or fail-

ure (Harachi, Abbot, Catalnao, Haggerty, & Fleming, 1999).

Process evaluation facilitates program developers to under-

stand fully the strengths and weaknesses of the developed pro-

grams. Program implementers can follow the suggestions from

the process evaluation for further program delivery. This is one

essence of evidence-based practice. It is also the foundation of

bridging the gap between research and practice (Saul et al.,

2008; Wandersman et al., 2008). Second, process evaluation

can inform program developers about whether the programs are

delivered according to some standardized manuals. The exis-

tence of other activities different from those intended by the

program developers will not truly reflect the effectiveness of

the prescribed programs. Third, different human organizations

and communities arrange the programs in various settings,

levels of involvement by the stakeholders, perceptions of the

program among program implementers and program receivers,

as well as the levels of support. Process evaluation can document

the variety of implementations in real human service settings for

the same manualized plans. Finally, process evaluation provides

insights for program developers and implementers into the link-

age between process and outcome. These insights allow both

program developers and implementers to delineate the success

and improvement areas during the process and connect them

with the program outcomes.

Project P.A.T.H.S.

Many primary prevention programs and positive youth devel-

opment programs have been developed in the West to address

the growing adolescent development problems, such as sub-

stance abuse, mental health problems, and school violence

(Shek, 2006a; Shek & Merrick, 2009). However, in Hong

Kong, there are very few systematic and multiyear positive

youth development programs. To promote holistic develop-

ment among adolescents in Hong Kong, The Hong Kong

Jockey Club Charities Trust approved the release of HK$750

million (HK$400 million for the first phase and HK$350 mil-

lion for the second phase) to launch a project entitled

‘‘P.A.T.H.S. to Adulthood: A Jockey Club Youth Enhancement

Scheme.’’ The acronym ‘‘P.A.T.H.S.’’ denotes Positive Adoles-

cent Training through Holistic Social Programs. The Trust invited

academics from five universities in Hong Kong to form a research

team to develop a multiyear universal positive youth program

(Shek & Merrick, 2009).

The project commenced in 2004 and is targeted to end by

2012. There are two tiers of programs in this project. The Tier

540 Research on Social Work Practice 21(5)

1 Program is a universal positive youth development program

where students from the Secondary 1 (Grade 7) to Secondary

3 (Grade 9) participate in a classroom-based program, normally

with 20 hr of training in the school year in each grade. Around

one fifths of adolescents with more psychosocial needs will

join the Tier 2 Program. The Tier 2 Program consists of inten-

sive training on volunteer service, adventure-based counseling

camp, and other experiential learning activities.

The overall objective of the Tier 1 Program is to promote

holistic development among junior secondary school students

in Hong Kong. The programs are designed according to 15 con-

structs conducive to adolescent development (Shek, 2006b):

promotion of bonding, cultivation of resilience, promotion of

social competence, promotion of emotional competence,

promotion of cognitive competence, promotion of behavioral

competence, promotion of moral competence, cultivation

of self-determination, promotion of spirituality, development

of self-efficacy, development of a clear and positive identity,

promotion of beliefs in the future, provision of recognition for

positive behavior, provision of opportunities for prosocial

involvement, and promotion of prosocial norms.

The Tier 1 Program has several characteristics. First, there

are 40 units per grade (each lasting for 30 min), with a total

of 120 units, for the entire Tier 1 program. The time fits well

with the Hong Kong secondary school time slots. Second, each

school can choose to implement all 40 units (full program) or

20 units (core program), according to school needs. Third, the

program content was developed by the research team and

underwent extensive integration of existing research findings,

adolescent needs, cultural characteristics, and trial teaching

runs. Fourth, relevant adolescent developmental issues, such

as drug issues, sexuality, and financial management, are incor-

porated into the program content so that it fits the current real-

life experiences of Hong Kong adolescents. Fifth, the program

implementers are either social workers or teachers who had to

undergo intensive 20-hr training before program delivery.

There are two implementation phases: the experimental

implementation phase (EIP) and the full implementation phase

(FIP). The EIP aims at accumulating experience from trial

teaching and administrative arrangement. Program materials

are revised and refined during this phase. The FIP aims at

executing the programs in full force. There are several lines

of evidence that support the effectiveness of the Tier 1

Program, including the evaluation findings based on rando-

mized group trials (e.g., Shek & Ma, 2011; Shek & Yu, 2011),

subjective outcome evaluation (e.g., Shek & Sun, 2007), quali-

tative findings based on focus group interviews with program

implementers and students (e.g., Shek & Lee, 2008), interim

evaluation (e.g., Shek, Sun, & Siu, 2008), analyses of the weekly

diaries of students (e.g., Shek, Sun, Lam, Lung, & Lo, 2008), and

case studies (e.g., Shek & Sun, 2008). The evaluation findings

based on different evaluation strategies indicate that Project

P.A.T.H.S. promotes the development of its program

participants.

Process evaluation has already been carried out in the EIP

and FIP for Secondary 1 (Shek, Ma, Lui, & Lung, 2006) and

2 students (Shek, Lee, & Sun, 2008). The evaluation results

indicate that the quality of implementation and program adher-

ence are high. The current study focuses on the Secondary

3 Tier 1 Program.

Process Evaluation for the Secondary 3 Tier 1 Program

Process evaluation for Secondary 3 students is important.

First, the Secondary 3 curriculum is different from the others.

It requires students to develop self-reflexivity during the

process. Thus, the findings of the process evaluation may be

different. Second, Secondary 3 students are cognitively more

mature and have more life exposure than their Secondary 1 and

2 counterparts. Their perception of program implementation

quality can be different. Finally, Secondary 3 students have

participated in Project P.A.T.H.S. continuously for 3 years and

have all completed the entire Tier 1 curriculum. Their feedback

represents an overall evaluation for the entire Project P.A.T.H.S.

curriculum.

The current process evaluation focuses on program adher-

ence, process factors, program quality, and success. Program

adherence is the objective estimation of the adoption percentage

from the manualized plan for real service delivery. A variety of

process factors exist. A review of literature indicates that the

following program attributes can affect the quality and success

of the positive youth development program implementation

(Collaborative for Academic, Social and Emotional Learning,

2010; Harachi et al., 1999; Nation et al., 2003; Ringwalt et al.,

2003; Tobler, Lessard, Marshall, Ochshorn, & Roona, 1999):

1. Student interest: A successful program usually elicits the

interest of students.

2. Active involvement of students: The more involved the

students are, the higher the possibility that the program

can achieve its outcomes.

3. Classroom management: The program implementer can

manage student discipline during student activities. Stu-

dents obey the requirements set by the program imple-

menter and are attentive.

4. Interactive delivery method: Interactive delivery is better

than didactic delivery for positive youth development

programs.

5. Strategies to enhance the motivation of students: The use

of various learning strategies can enhance the engage-

ment of students and result in positive learning outcomes.

6. Positive feedback: The use of praise and encouragement

throughout the lessons by the program implementers can

promote the engagement of students.

7. Familiarity of implementers with the students: All other

things being equal, a higher degree of familiarity with the

students is positively related to student learning outcome.

8. Reflective learning: The program implementer should

engage students in reflection and deeper learning.

This can lead to growth and meaningful changes among

the students.

Law and Shek 541

9. Program goal attainment: The achievement of program

goals constitutes program success.

10. Time management: Efficient time management ensures

that the majority of the program materials are carried out

with high program adherence.

11. Familiarity of program implementers with the implementa-

tion materials: Familiarity with the material ensures that

the messages are conveyed effectively to the students.

Program quality is the subjective appraisal of the program

implementation process. It can be reflected from the implemen-

tation atmosphere and the interaction between program imple-

menters and students.

Program success refers to the extent of unit objective

attainment and the subjective evaluation of the response of the

students to the program.

Against this background, the current study aims to explore

the factors related to the implementation quality and imple-

mentation success of the Secondary 3 Tier 1 Program during

the Full Implementation Phase. There are two research

questions:

1. What is the implementation quality of the Secondary 3

curriculum of the Tier 1 Program of Project P.A.T.H.S.

in Hong Kong?

2. How are program adherence and other indicators related to

the implementation quality and success of the Secondary 3

Tier 1 Program?

Method

Participants and Procedure

In total, 14 schools were randomly selected from among the

167 secondary schools that joined the Secondary 3 program

in the school year 2008/2009 for the process evaluation.

Process evaluation was carried out using systematic obser-

vations of actual classroom program delivery. For each school

joining the process evaluation, one to two program units were

evaluated by two independent observers who are project

colleagues with master’s degrees. A total of 20 units were

observed for this study. The learning units of these units

are shown in Table 1. During the observation, observers sat

at the back of the classroom and evaluated the method by which

the units were actually implemented by completing several

instruments.

After the psychometric properties of the instruments were

explored, program adherence and implementation process

components were associated with implementation quality and

success. In addition, program and implementation process

components were used to predict implementation quality and

implementation success separately.

Instruments Program adherence. Observers were requested to rate program

adherence in terms of percentage (i.e., the correspondence

between actual program delivery and stipulated program mate-

rials). Pearson correlation analyses showed that the ratings of

program adherence were highly reliable (r ¼ .86, p < .001) between raters.

Implementation Process Checklist (IPC)

The IPC consists of 11 items, which are shown in Table 2.

Items 1, 2, 3, 4, 5, 6, and 8 are conceptually related to the imple-

mentation process, whereas items 7, 9, 10, and 11 are related to the

implementation context (items 7, 9, 10, and 11). Observers were

requested to report their observations using a 7-point Likert scale

ranging from 1 (extremely negative) to 7 (extremely positive).

To explore whether the conceptual distinction of these two

components is reflective from the data, principal components

analysis (PCA) with varimax rotation was used to summarize

the effects of the 11 process evaluation items. Two components

were identified with eigenvalues greater than 1.0. In addition,

the resulting scree plot of the eigenvalues revealed that the

leveling off to a straight horizontal line occurred after the

second eigenvalue. These two factors could explain 75.02% of variance.

The components emerged to reflect clearly the factors

originally proposed. The items from each subscale were loaded

on the intended components. Consistent with the conceptual

model, two components were formed, namely, implementation

process (items 1, 2, 3, 4, 5, 6, and 8) and implementation

context (items 7, 9, 10, and 11).

The internal consistency of the overall IPC, as shown by

Cronbach’s a, was .93. The inter-rater reliability of the IPC, as shown by Pearson correlation, was .87 (p < .001). The inter-

nal consistency of the Implementation Process subscale was

.92 and that of the Implementation Context subscale was .82.

Process Outcomes

Two items were used to evaluate the observation outcome:

implementation quality and implementation success. Observers

were requested to indicate their observations using a 7-point

Likert scale ranging from 1 (poor) to 7 (excellent). A higher

score represents better implementation quality or success.

The inter-rater reliability for implementation quality, as shown

by Pearson correlation, was .73 (p < .001), whereas that for

implementation success was .69 (p < .001).

Results

The inter-rater reliabilities of the scores were high, allowing the

ratings of each item by the two observers across all units to be

averaged. Table 3 shows the descriptive profile of the evalua-

tion indicators for process evaluation. The overall program

adherence to the established manual ranged from 12.5% to 95.0%, with an average overall adherence of 76.18%. All other items used the 7-point scale. We set 4.50 as the cutoff point

as an indication of high or low rating; it is a more stringent

criterion instead of using the mid-point. This can differentiate

542 Research on Social Work Practice 21(5)

some factors from others and provide a more balanced

picture. The scores for implementation quality and success

were 4.63 (SD ¼ .94) and 4.68 (SD ¼ .82), which are high. The scores of the 11 process evaluation items ranged from

4.48 to 5.60. Classroom management (5.60) and familiarity

with students (5.40) had the highest scores, whereas reflective

learning (4.48) and time management (4.55) had the lowest

scores. Apart from reflective learning, all scores were on the

high side.

The 11 items were divided into the two groups of the PCA:

implementation process and context. The mean score for

implementation process was 5.03 (SD ¼ .97), whereas that for

the implementation context was 4.95 (SD ¼ .99). Both scores were on the high side.

Table 4 shows the inter-correlations among program adher-

ence, implementation process, implementation context, imple-

mentation quality, and implementation success. All variables

were highly related to each other. Quality versus success

(.98) and process versus quality (.83) had the highest correlations,

whereas process versus adherence (.51) and process versus

context (.67) had the lowest.

In addition to correlational analysis, multiple regression

analyses were also performed using program adherence,

implementation process, and implementation as independent

Table 1. Summary of the Program Objectives of the Observed Units

School Program Units Program Objectives

A MC 3.1 To discuss the differences between fairness in our ideals and in reality To understand that a system or situation of ‘‘absolute fairness’’ does not exist in reality

MC 3.2 To learn how to exercise self-reflection and how to help others To discuss ways of helping others in society

B PN 3.1 To understand that prosocial and moral consideration and analysis are essential when making decisions C PN 3.2 To understand that society has different expectations of different roles

To investigate the potential conflict between being prosocial and socially accepted behaviors D RE 3.3 To state how Mencius looked at adversity

To reflect upon oneself and how Mencius’ teachings can be applied in daily life RE 3.4 To construct a vision of one’s future family

To recognize that one needs to work hard and use resources properly so as to achieve their aspirations E BC3.1 and

BC 3.2 To understand the importance of forgiving others sincerely To learn how to forgive others for their offenses against us To learn how to observe and appreciate people and things around us

F BF 3.1 To adopt a realistic and positive attitude in exploring future careers G BC 3.2 To understand the importance of sincere forgiveness

To understand the negative influence of taking revenge on those who have offended us H BC 3.1 To understand that appreciation brings joy to oneself and others.

To learn how to observe and appreciate people and things around us, and to express sincere appreciation To learn how to respond to appreciation in a proper manner

BC 3.2 To understand the importance of sincere forgiveness To understand the negative influence of taking revenge on those who have offended us

I BF 3.2 To understand that different jobs have different requirements To be aware of the issue of gender stereotypes and their impact(s) on career choices

J BC 3.1 To understand that appreciation brings joy to oneself and others. To learn how to observe and appreciate people and things around us, and to express sincere appreciation To learn how to respond to appreciation in a proper manner

BC 3.2 To understand the importance of sincere forgiveness To understand the negative influence of taking revenge on those who have offended us

K SE 3.1 To understand that successful wealth management relies on the ability to exercise self-control and delayed gratification To understand the importance of controlling desires for unnecessary material things

SE 3.2 To understand the meaning of dreams and their importance in life To identify the personal qualities that help one overcome environmental constraints and realize dreams

L MC 3.3 To learn to cherish love relationships and to love with commitment instead of quitting easily To discuss the proper attitudes to end a love relationship

M SC 3.3 To understand the reasons for conflict among siblings To learn the proper attitude to get along with siblings

N SC 3.3 To understand the reasons for conflict among siblings To learn the proper attitude to get along with siblings

SC 3.4 To understand the reasons for conflict among friends To learn how to face and handle conflict with friends

Note. MC ¼ moral competence; PN ¼ prosocial norms; RE ¼ resilience; BC ¼ behavioral competence; BF ¼ beliefs in future; SE ¼ self-efficacy; SC ¼ social competence.

Law and Shek 543

variables. Implementation quality and implementation success

were used as two separate dependent variables. Table 5 shows

the results for the prediction of implementation quality.

Both implementation process and program adherence could

predict the quality with a large variance explained. Implemen-

tation context could not predict the quality. The effect size for

the implementation process (b ¼ .51), Cohen f 2, was .35, which is large. The effect size for program adherence (b ¼ .34) was .13, which is medium. Table 6 shows the results of the

prediction of implementation success. Implementation process,

implementation context, and program adherence could all

predict success with a large variance explained. The effect size

for implementation process (b ¼ .47) was .28, which is large. The effect size for both context and adherence (b ¼ .29) was .10, which is medium.

Discussion and Social Work Implications

Project P.A.T.H.S is a huge evidence-based project of positive

youth development in Hong Kong. This article attempts to

examine the adherence and quality of implementation of the

Tier 1 Program (Secondary 3 curriculum) of Project P.A.T.H.S.

in the second year of the FIP.

We find that the range of program adherence is wide from

12.5% to 95.0%. There are various reasons for the difference in the range of program adherence. First, some program mate-

rials are overpacked within 30 min. By the time the program

implementers have finished part of the program, the lesson time

is up. The fact that the implementers cannot complete the pro-

gram directly affects the adherence rate. Second, the units

observed cover a great variety of constructs. Some schools

have already some established programs covering these con-

structs before the P.A.T.H.S. Hence, these schools may use

these materials instead of manual materials. For example, the

construct of ‘‘beliefs about future’’ is related to career planning

(i.e., BF 3.1 and BF 3.2). The research team designed some

exercises for students to reflect on their career choices.

However, schools may have similar career planning exercises

from the career and guidance team and may use their materials

instead. Another example is the construct of ‘‘moral competence.’’

Unit MC 3.2 is related to prosocial behavior. Almost every

school in Hong Kong arranges volunteer services for students

(Law & Shek, 2009). Some schools may use their school

Table 5. Regression Table of Implementation Quality

Predictors b

Implementation process .51*** Implementation context .22 Program adherence .34***

Note. R2 ¼ .90. ***p < .001.

Table 3. Descriptive Statistics of Evaluation Items

Evaluation Items Min Max M SD

Interest 3.0 6.5 5.02 1.02 Involve 3.5 6.5 5.38 .94 Class 4.0 7.0 5.60 .82 Interact 2.5 6.5 4.73 .94 Motivation 3.0 6.5 5.10 1.01 Feedback 2.5 6.5 4.93 1.03 FStudents 3.0 7.0 5.40 1.02 Reflect 2.5 6.0 4.48 1.03 Goal 2.0 6.5 4.83 1.18 Time 2.5 6.5 4.55 .96 FMaterials 2.5 6.5 5.03 .82 Adhere 12.50% 95.00% 76.18% 24.00% Quality 2.0 6.0 4.63 .94 Success 2.5 6.0 4.68 .82

Note. Interest ¼ student interest; Involve ¼ active involvement of students; Class ¼ classroom management; Interact ¼ interactive delivery method; Motivation ¼ strategies to enhance the motivation of students; Feedback ¼ positive feedback; FStudents ¼ familiarity of implementers with students; Reflect ¼ reflective learning; Goal ¼ program goal attainment; Time ¼ time management; FMaterials¼ familiarity of program implementers with the program materials; Adhere ¼ program adherence; Quality ¼ implementation quality; Success ¼ implementation success.

Table 2. Factor Loadings for Principal Components Analysis With Varimax Rotation of Implementation Process Checklist

Evaluation Items Implementation Process Implementation Context

Interest .85 .38 Involve .96 .02 Class .71 .28 Interact .74 .41 Motivation .84 .23 Feedback .90 .22 FStudents �.02 .79 Reflect .77 .45 Goal .56 .66 Time .32 .67 FMaterials .41 .82

Note. Factor loadings > .50 are in boldface. Interest ¼ student interest; Involve ¼ active involvement of students; Class ¼ classroom management; Interact ¼ interactive delivery method; Motivation ¼ strategies to enhance the motivation of students; Feedback ¼ positive feedback; FStudents ¼ familiarity of implementers with students; Reflect ¼reflective learning; Goal¼program goal attainment; Time¼ time management; FMaterials ¼ familiarity of program implementers with the program materials.

Table 4. Summary of Intercorrelations for Scores of Process Evalua- tion, Implementation Quality, and Success

Measure 1 2 3 4 5

1. Implementation process .67*** .51** .83*** .81*** 2. Implementation context .77*** .77*** .79*** 3. Program adherence .81*** .78*** 4. Implementation quality .98*** 5. Implementation success

Note. Bonferroni correction was used to evaluate the significance of the correlations. ***p < .001, **p < .005.

544 Research on Social Work Practice 21(5)

volunteer service experience to deliver similar messages.

Thus, the same program objectives are achieved with different

materials. Another issue is the use of current examples. The

manual was written 2–3 years prior to this study. Thus, some

examples may not be up-to-date. Program implementers may

opt to choose current cases (such as news clips) rather than the

cases provided by the manuals. Although these adaptations aim

at a more effective message delivery, they may affect the rate of

program adherence.

Despite the discrepancy in the rate of program adherence,

the overall degree of adherence to the program units is

on the high side. This observation is generally consistent

with the previous findings generated from process evalua-

tions conducted by observers (Shek et al., 2006, 2008) and

subjective outcome evaluations reported by the program

implementers (Shek & Sun, 2008; Shek, Sun, & Siu,

2008). Most program contents are well designed for implemen-

tation. This can be attributed to the fact that all program mate-

rials have gone through trial teaching. They have already been

revised and refined according to prior teaching experience.

Thus, program implementers did not have great difficulty in

following the plans. These findings dispute the common

myth that curricula-based positive youth development

programs cannot be used easily and require major adaptations

or modifications.

Different aspects of the program delivery were perceived to

be positive, highlighting the fact that positive youth programs

were well received by both the program implementers and stu-

dents. Nevertheless, there were relatively low average ratings

on time management and reflective learning. These findings

are similar to those based on the EIP (Shek et al., 2006,

2008). There are two possible explanations for these observa-

tions. First, due to the usual didactic teaching style in Hong

Kong, students are not used to reflecting on their everyday life

practice in classroom settings. Hence, the students cannot eas-

ily shift their learning modes from one-way knowledge disse-

mination to reflective learning. Second, the overpacking of

the curriculum may have prevented the students from carrying

out reflections on their learning. Overpacking could have also

contributed to the unsatisfactory rating of time management.

The current study has found that program adherence,

implementation process, and implementation context are

closely associated with implementation quality and success.

Implementation quality and success had the highest correlation.

For positive youth programs, an interactive program delivery is

the key milestone for program quality and success (Collaborative

for Academic, Social and Emotional Learning, 2010). Thus, these

factors are highly correlated with each other.

Conversely, program adherence is associated with both

implementation quality and process. The manualized plan,

along with the skills of the program implementer, is effective

in bringing quality program delivery. Heavy modification is

not required.

Among the correlations, the correlation between implemen-

tation process and context was relatively lower. This is consis-

tent with the findings of the PCA that these two variables are

distinctive. For example, a stern and distant teacher can achieve

all the contextual qualities, such as achieving the goals and

familiarity with programs, but this does not reflect good

implementation without process-oriented program delivery

(Collaborative for Academic, Social and Emotional Learning,

2010).

Program adherence and implementation process had the

lowest correlation. This reflects the dilemma of program imple-

menters. In general, a few program materials are overpacked

within the time limit. When a program implementer focuses

on following the manual through, he or she will run out of time

for discussion, self-reflection, and other interactions, affecting

the learning process in the end.

The current study utilizes three groups of evaluation vari-

ables (i.e., implementation process, implementation context,

and program adherence) to predict implementation quality or

success. Both process and adherence can predict quality,

whereas all variables can predict success. The effect size of all

predictors is either medium or large. Implementation process

refers to the processes and dynamics during classroom activi-

ties, whereas implementation context refers to the background

knowledge of students, familiarity with materials, time man-

agement, and goal attainment. The implementation process

emphasizes the interaction between implementer and students

and is critical to the quality and success of positive youth pro-

gram delivery. Program adherence can predict quality and suc-

cess, implying that the quality of the curriculum manual is high.

Implementation context cannot predict quality, but it can pre-

dict success. Context items, including time management, goal

attainment, and program preparation, are not closely related

to process. However, process is highly related to quality. Thus

explains the relatively low relationship to quality. However,

implementation success is about whether the message of the

lesson is effectively delivered to students and consists of con-

textual elements. Thus, context can predict implementation

success. Other plausible explanations can be explored from

further studies.

The findings in the present study have several social work

implications. The first implication is on the conceptual level.

When we focus on program implementation regardless of

external environment (i.e., macro-level implication and

dosage), we can focus on three variables: program adherence,

implementation process, and context. These variables are all

related to implementation quality or success. The present findings

provide conceptual insights for understanding program quality

or success. These can be generalized to other social services.

Table 6. Regression Table of Implementation Success

Predictors b

Implementation process .47*** Implementation context .29* Program adherence .29*

Note. R2 ¼ .86. *p < .05. *** p < .001..

Law and Shek 545

For instance, the implementation quality and success of an

evidence-based volunteer service manual used by a profes-

sional social worker with a group of adolescent volunteers can

be evaluated using these three areas.

The second implication is on a practical level. These 11 pro-

cess evaluation statements can actually be used in other social

work contexts, especially in educational and developmental

groups. Social workers in general have to develop and imple-

ment many group work plans and psychosocial interventions.

All these measures are important for positive youth programs

and should be brought to the fore in the group work training

of social workers working with the youth. Social workers

should be made aware that both implementation process and

context are important for classroom-based psychosocial inter-

vention programs. With reference to the 11 evaluation items,

program implementers should consider ways of enhancing

reflective learning of students as well as observe good time

management.

Another practice implication is program adherence.

O’Connor, Small, and Cooney (2007) suggested that there are

certain risky adaptations for program adherence, such as reduc-

ing the number or length of sessions, lowering the level of par-

ticipant engagement, eliminating key messages, removing

topics, and changing the theoretical approach. The present find-

ings suggest the importance of program adherence. Program

adherence coupled with the effective use of the self, and good

interaction between implementers and students, can be more

difficult than expected. This requires intensive training and

personal reflexivity on the part of the social worker. If over-

packing of the material prevents program adherence, then Tier

1 materials can be trimmed down during the revamping process

so that the messages can be delivered clearly with sufficient

program materials.

This study has several limitations. First, school factors

should be controlled in the regression analysis. The authors

were not able to locate the academic profile of the schools,

as this information is regarded confidential. Thus, the results

of regression should be treated with caution and stringently

specified in further analysis. Second, only 14 randomly

selected schools participated in this study. Although the num-

ber of schools can be regarded as respectable, the inclusion

of a greater number of schools with different characteristics

to participate in the study is advisable. Third, process evalua-

tion with reference to macro-level implication, dosage issues

(Saunders, Evans, & Joshi, 2005), and school characteristics

can help program developers to understand the quality of the

program implementation process further. Fourth, the observa-

tion may have a confounding effect. Students may be more

cooperative when there are visitors or outside observers

because the students do not want to ruin the reputation of their

schools. As Chinese students, they also want to ‘‘give face’’ to

the program implementers (Leung & Chan, 2003). They inten-

tionally perform better in front of the raters. Fifth, additional

variables can be devised for the implementation context and

implementation process, such as the effect of using computer

games and self-disclosure of the program implementers.

Despite these limitations, the current process evaluation find-

ings suggest that the quality of implementation of the Tier 1

Program is generally high. The findings are conducive for pro-

gram reimplementation as well as the training and conceptual

enlightenment of social workers on the importance of process

evaluation in social work practice.

Acknowledgments

The preparation for this article and Project P.A.T.H.S. were financially

supported by The Hong Kong Jockey Club Charities Trust.

Declaration of Conflicting Interests

The authors declared no potential conflicts of interests with respect to

the authorship and/or publication of this article.

Funding

The authors received financial support from the Hong Kong Jockey

Club Charities Trust.

References

Allicock, M., Campbell, M. K., Valle, C. G., Barlow, J. N., Carr, C., &

Meier, A., Gizlice, Z. (2010). Evaluating the implementation of

peer counseling in a church-based dietary intervention for African

Americans. Patient Education and Counseling, 81, 37-42.

Beets, M. W., Flay, B. R., Vuchinich, S., Acock, A. C., Li, K., &

Allred, C. (2008). School climate and teachers? Beliefs and

attitudes associated with implementation of the positive action

program: A diffusion of innovations model. Prevention Science,

9, 264-275.

Bertens, M. G. B. C., Eiling, E. M., van den Borne, B., & Schaalma, H.

P. (2009). Uma toril evaluation of an STI/HIV-prevention inter-

vention for Afro-Caribbean women in the Netherlands. Patient

Education and Counseling, 75, 77-83.

Bowes, D., Marquis, M., Young, W., Holowaty, P., & Isaac, W.

(2009). Process evaluation of a school-based intervention to

increase physical activity and reduce bullying. Health Promotion

Practice, 10, 394-401.

Braun, S. M., van Haastregt, J. C., Beurskens, A. J., Gielen, A. L.,

Wade, D. T., & Schols, J. M. (2010). Feasibility of a mental prac-

tice intervention in stroke patients in nursing homes: A process

evaluation. BMC Neurology, 10, A74.

Carswell, S. B., Hanlon, T. E., O’Grady, K. E., Watts, A. M., &

Pothong, P. (2009). A preventive intervention program for urban

African American youth attending an alternative education program:

Background, implementation, and feasibility. Education & Treat-

ment of Children, 32, 445-469.

Cohen, A. N., Glynn, S. M., Hamilton, A. B., & Young, A. S. (2010).

Implementation of a family intervention for individuals with

schizophrenia. Journal of General Internal Medicine, 25(Suppl. 1),

S32-S37.

Collaborative for Academic, Social and Emotional Learning (2010).

Implementation guidance. Retrieved February 1, 2011, from

http://www.casel.org/implement/guidance.php

Dane, A. V., & Schneider, B. H. (1998). Program integrity in primary

and early secondary prevention: Are implementation effects out of

control? Clinical Psychology Review, 18, 23-45.

546 Research on Social Work Practice 21(5)

Domitrovich, C. E., & Greenberg, M. T. (2000). The study of imple-

mentation: Current findings from effective programs that prevent

mental disorders in school-aged children. Journal of Educational

and Psychological Consultation, 11, 193-221.

Eisenberg, M. (2009). Integrating a school-based health intervention

in times of high-stakes testing: Lessons learned from full court

press. Health Promotion Practice, 10, 284-292.

Elliot, D. S., & Mihalic, S. (2004). Issues in disseminating and replicating

effective prevention programs. Prevention Science, 5, 47-52.

Fagan, A. A., Hanson, K., Hawkins, J. D., & Arthur, M. W. (2008).

Bridging science to practice: Achieving prevention program imple-

mentation fidelity in the community youth development study.

American Journal of Community Psychology, 41, 235-249.

Fraze, J. L., Uhrig, J. D., Davis, K. C., Taylor, M. K., Lee, N. R., &

Spoeth, S. Robinson, A., Smith, K., Johnston, J., & McElroy, L.

(2009). Applying core principles to the design and evaluation of the

‘‘Take Charge. Take the Test’’ campaign: What worked and lessons

learned. Public Health, 123(Suppl. 1), e23-e30.

Frazen, S., Morrel-Samuels, S., Reischl, T. M., & Zimmerman, M. A.

(2009). Using process evaluation to strengthen intergenerational

partnerships in the Youth Empowerment Solutions Program. Jour-

nal of Prevention & Intervention in the Community, 37, 289-301.

Gnich, W., Sheehy, C., Amos, A., Bitel, M., & Platt, S. (2008).

A Scotland-wide pilot programme of smoking cessation services

for young people: Process and outcome evaluation. British Journal

of Addiction, 103, 1866-1874.

Griffin, S. F., Wilcox, S., Ory, M. G., Lattimore, D., Leviton, L., &

Castro., C. Carpenter, R.A., & Rheaume, C. (2010). Results from

the active for life process evaluation: Program delivery fidelity and

adaptations. Health Education Research, 25, 325-342.

Harachi, T. W., Abbott, R. D., Catalano, R. F., Haggerty, K. P., &

Fleming, C. B. (1999). Opening the black box: Using process eva-

luation measures to assess implementation and theory building.

American Journal of Community Psychology, 27, 711-731.

Hargreaves, J., Hatcher, A., Strange, C., Phela, G., Busza, J., & Kim, J.

Watts, C., Morison, L., Porter, J., Pronyk, P., & Bonell, C. (2009).

Process evaluation of the intervention with microfinance for AIDS

and Gender Equity (IMAGE) in rural South Africa. Health Educa-

tion Research, 25(1), 27-39.

Hart, A. Jr., Bowen, D. J., Christensen, C. L., Mafune, R.,

Campbell, M. K., & Saleeba, A. Kuniyuki, A., Shirley A. A., &

Beresford, S.A.A. (2009). Process evaluation results from the Eat-

ing for a Healthy Life study. American Journal of Health Promo-

tion, 23, 324-327.

Huryk, L. A. (2010). Factors influencing nurses attitudes towards

healthcare information technology. Journal of Nursing Manage-

ment, 18, 606-612.

Johnson, C. C., Lai, Y., Rice, J., Rose, D., & Webber, L. S. (2010).

ACTION live: Using process evaluation to describe implementation

of a worksite wellness program. Journal of Occupational and

Environmental Medicine, 52(Suppl. 1), S14-S21.

Karwalajtys, T., McDonough, B., Hall, H., Guirguis-Younger, M.,

Chambers, L. W., & Kaczorowsk, J. Lohfeld, L., Hutchison, B.

(2009). Development of the volunteer peer educator role in a com-

munity Cardiovascular Health Awareness Program (CHAP): A

process evaluation in two communities. Journal of Community

Health: The Publication for Health Promotion and Disease Pre-

vention, 34, 336-345.

Konle-Parker, D. J., Erien, J. A., & Dubbert, P. M. (2010). Lessons

learned from an HIV adherence pilot study in the Deep South.

Patient Education and Counseling, 78, 91-96.

Kumpfer, K. L., Pinyuchon, M., de Melo, A. T., & Whiteside, H. O.

(2008). Cultural adaptation process for international dissemination

of the Strengthening Families Program. Evaluation & the Health

Professions, 31, 226-239.

Kwong, K., Ferketich, A. K., Wewers, M. E., Shek, A., Tsang, T., &

Tso, A. (2009). Development and evaluation of a physician-led

smoking cessation intervention for low-income Chinese Americans.

Journal of Smoking Cessation, 4, 92-98.

Law, B. M. F., & Shek, D. T. L. (2009). Adolescent volunteerism in

Hong Kong after 1997: A critical review. International Journal

of Adolescent Medicine and Health, 21, 137-149.

Leung, T. K. P., & Chan, R. Y. K. (2003). Face, favour and positioning—

A Chinese power game. European Journal of Marketing, 37,

1575-1598.

Louis, R. M. S., Parow, J. E., Eby, D. W., Bingham, C. R.,

Hockanson, H. M., & Greenspan, A. I. (2008). Evaluation of

community-based programs to increase booster seat use. Accident

Analysis and Prevention, 40, 295-302.

Mair, F. S., Hiscock, J., & Beaton, S. C. (2008). Understanding factors

that inhibit or promote the utilization of telecare in chronic lung

disease. Chronic Illness, 4, 110-117.

Muckelbuer, R., Libuda, L., Clausen, K., & Kersting, M. (2009).

Long-term process evaluation of a school-based programme for

overweight prevention. Child Care, Health & Development, 35,

851-857.

Mukoma, W., Flisher, A., Ahmed, N., Jansen, S., Mathews, C.,

Klepp, K.,, & Schaalma, H. (2009). Process evaluation of a

school-based HIV/AIDS intervention in South Africa. Scandina-

vian Journal of Public Health, 37(Suppl. 2), 37-47.

Nation, M., Crusto, C., Wandersman, A., Kumpfer, K. L., Seybolt, D.,

Morrissey-Kane, E., & Davino, K. (2003). What works in preven-

tion: Principles of effective prevention programs. American Psychol-

ogist, 58, 449-456.

O’Connor, C., Small, S. A., & Cooney, S. M. (2007). Program fidelity

and adaptation: Meeting local needs without compromising pro-

gram effectiveness. What works, Wisconsin – Research to Practice

Series, 4, 1-5.

Painter, J. E., Sales, J. M., Pazol, K., Grimes, T., Wingood, G., &

DiClemente, R. J. (2010). Development, theoretical framework,

and lessons learned from implementation of a school-based

influenza vacation intervention. Health Promotion Practice,

11(Suppl. 3), 425-525.

Quintiliani, L., Yang, M., & Sorensen, G. (2010). A process

evaluation of tobacco-related outcomes from a telephone and

print-delivered intervention for motor freight workers. Addictive

Behaviors, 35, 1036-1039.

Ringwalt, C. L., Ennett, S., Johnson, R., Rohrbach, L. A.,

Simons-Rudolph, A., & Vincus, A. & Thorne, J. (2003). Factors

associated with fidelity to substance use prevention curriculum

guides in the nation’s middle schools. Health Education & Behavior,

30, 375-391.

Law and Shek 547

Salmela, S., Poskiparta, M., Kasila, K., Vahasarja, K., & Vanhala, M.

(2009). Transtheoretical model-based dietary interventions in

primary care: A review of the evidence in diabetes. Health Educa-

tion Research, 24, 237-252.

Saul, J., Wandersman, A., Flaspholer, P., Duffy, J., Lubell, K., &

Noonan, R. (2008). Research and action for bridging science and

practice in prevention. American Journal of Community Psychology,

41, 165-170.

Saunders, R. P., Evans, M. H., & Joshi, P. (2005). Developing a

process-evaluation plan for assessing health promotion program

implementation: A how-to guide. Health Promotion Practice, 6,

134-147.

Scheirer, M. A. (1994). Designing and using process evaluation. In

J. S. Wholey, H. P. Hatry & K. E. Newcomer (Eds.), Handbook

of practical program evaluation. San Francisco, CA: Jossey-Bass.

Shek, D. T. L. (2006a). Adolescent developmental issues in Hong

Kong: Relevance to positive youth development programs in Hong

Kong. International Journal of Adolescent Medicine and Health, 3,

341-354.

Shek, D. T. L. (2006b). Construction of a positive youth development

program in Hong Kong. International Journal of Adolescent

Medicine and Health, 18, 299-302.

Shek, D. T. L., & Lee, T. Y. (2008). Qualitative evaluation of the

Project P.A.T.H.S.: Findings based on focus groups with student

participants. International Journal of Adolescent Medicine and

Health, 20, 449-462.

Shek, D. T. L., Lee, T. Y., & Sun, R. C. F. (2008). Process evaluation

of the implementation of the Secondary 2 Program of Project

P.A.T.H.S. in the experimental implementation phase. Scientific

World Journal, 8, 83-94.

Shek, D. T. L., & Ma, C. M. S. (2011). Impact of the Project

P.A.T.H.S. in the junior secondary school years: Individual growth

curve analyses. The Scientific World JOURNAL: TSW Child

Health & Human Development, 11, 253-266.

Shek, D. T. L., Ma, H. K., Lui, J. H. Y., & Lung, D. W. M. (2006).

Process evaluation of the Tier 1 Program of the Project P.A.T.H.S.

Scientific World Journal, 1, 300-309.

Shek, D. T. L., & Merrick, J. (2009). Promoting positive development

in Chinese adolescents: The Project P.A.T.H.S. in Hong Kong.

International Public Health Journal, 1, 237-241.

Shek, D. T. L., & Sun, R. C. F. (2007). Subjective outcome evaluation

of the Project P.A.T.H.S.: Qualitative findings based on the

experiences of program implementers. Scientific World Journal,

7, 1024-1035.

Shek, D. T. L., & Sun, R. C. F. (2008). Implementation of a positive

youth development program in a Chinese context: The role of

policy, program, people, process, and place. Scientific World Journal,

8, 960-996.

Shek, D. T. L., Sun, R. C. F., Lam, C. M., Lung, D. W. M., & Lo, S. C.

(2008). Evaluation of Project P.A.T.H.S. in Hong Kong: Utiliza-

tion of student weekly dairy. Scientific World Journal, 8, 13-21.

Shek, D. T. L., Sun, R. C. F., & Siu, A. M. H. (2008). Interim evalua-

tion of the Secondary 2 Program of Project P.A.T.H.S.: Insights

based on the experimental implementation phase. Scientific World

Journal, 8, 61-72.

Shek, D. T. L., & Yu, L. (2011). Prevention of adolescent problem

behavior: Longitudinal impact of the Project P.A.T.H.S. in Hong

Kong. Scientific World Journal, 11, 546-567.

Shevil, E., & Finlanyson, M. (2009). Process evaluation of a

self-management cognitive program for persons with multiple

sclerosis. Patient Education and Counseling, 76, 77-83.

Stewart, D. (2008). Implementing mental health promotion in schools:

A process evaluation. The International Journal of Mental Health

Promotion, 10, 32-41.

Tobler, N. S., Lessard, T., Marshall, D., Ochshorn, P., &

Roona, M. (1999). Effectiveness of school-based drug preven-

tion programs for marijuana use. School Psychology Interna-

tional, 20, 105-137.

Wandersman, A., Duffy, J., Flaspholer, P., Noonan, R., Lubell, K., &

Stillman, L. Blachman, M., Dunville, R., & Sual, J. (2008). Brid-

ging the gap between prevention research and practice: The inter-

active systems framework for dissemination and implementation.

American Journal of Community Psychology, 41, 171-181.

Wegner, L., Flisher, A., Caldwell, L. L., Vergnani, T., & Smith, E. A.

(2008). Health wise South Africa: Cultural adaptation of a school-

based risk programme. Health Education Reserach, 23, 1085-

1096.

Wilson, D. K., Griffin, S., Saunders, R. P., Kitzman-Ulrich, H.,

Meyers, D. C., & Mansard, L. (2009). Using process evaluation for

program improvement in dose, fidelity and reach: The ACT trial

experience. The International Journal of Behavioral Nutrition and

Physical Activity, 6, Art79.

Yamada, J., Stevens, B., Sidani, S., Watt-Watson, J., & de Silva, N.

(2009). Content validity of a process evaluation checklist to mea-

sure intervention implementation fidelity of the EPIC intervention.

Worldviews on Evidence-Based Nursing, 7, 158-164.

Zakrzewski, S., Steven, C., & Ricketts, C. (2009). Evaluating

computer-based assessment in a risk-based model. Assessment &

Evaluation in Higher Education, 34, 439-454.

Zani, B., & Cicognani, E. (2010). Evaluating the participatory process

in a community-based health promotion project. Journal of

Prevention & Intervention in the Community, 38, 55-69.

548 Research on Social Work Practice 21(5)

<< /ASCII85EncodePages false /AllowTransparency false /AutoPositionEPSFiles true /AutoRotatePages /None /Binding /Left /CalGrayProfile (Gray Gamma 2.2) /CalRGBProfile (sRGB IEC61966-2.1) /CalCMYKProfile (U.S. Web Coated \050SWOP\051 v2) /sRGBProfile (sRGB IEC61966-2.1) /CannotEmbedFontPolicy /Warning /CompatibilityLevel 1.3 /CompressObjects /Off /CompressPages true /ConvertImagesToIndexed true /PassThroughJPEGImages false /CreateJDFFile false /CreateJobTicket false /DefaultRenderingIntent /Default /DetectBlends true /DetectCurves 0.1000 /ColorConversionStrategy /LeaveColorUnchanged /DoThumbnails false /EmbedAllFonts true /EmbedOpenType false /ParseICCProfilesInComments true /EmbedJobOptions true /DSCReportingLevel 0 /EmitDSCWarnings false /EndPage -1 /ImageMemory 1048576 /LockDistillerParams true /MaxSubsetPct 100 /Optimize true /OPM 1 /ParseDSCComments true /ParseDSCCommentsForDocInfo true /PreserveCopyPage true /PreserveDICMYKValues true /PreserveEPSInfo true /PreserveFlatness false /PreserveHalftoneInfo false /PreserveOPIComments false /PreserveOverprintSettings true /StartPage 1 /SubsetFonts true /TransferFunctionInfo /Apply /UCRandBGInfo /Remove /UsePrologue false /ColorSettingsFile () /AlwaysEmbed [ true ] /NeverEmbed [ true ] /AntiAliasColorImages false /CropColorImages false /ColorImageMinResolution 266 /ColorImageMinResolutionPolicy /OK /DownsampleColorImages true /ColorImageDownsampleType /Bicubic /ColorImageResolution 200 /ColorImageDepth -1 /ColorImageMinDownsampleDepth 1 /ColorImageDownsampleThreshold 1.00000 /EncodeColorImages true /ColorImageFilter /DCTEncode /AutoFilterColorImages false /ColorImageAutoFilterStrategy /JPEG /ColorACSImageDict << /QFactor 0.15 /HSamples [1 1 1 1] /VSamples [1 1 1 1] >> /ColorImageDict << /QFactor 0.76 /HSamples [2 1 1 2] /VSamples [2 1 1 2] >> /JPEG2000ColorACSImageDict << /TileWidth 256 /TileHeight 256 /Quality 30 >> /JPEG2000ColorImageDict << /TileWidth 256 /TileHeight 256 /Quality 30 >> /AntiAliasGrayImages false /CropGrayImages false /GrayImageMinResolution 266 /GrayImageMinResolutionPolicy /OK /DownsampleGrayImages true /GrayImageDownsampleType /Bicubic /GrayImageResolution 200 /GrayImageDepth -1 /GrayImageMinDownsampleDepth 2 /GrayImageDownsampleThreshold 1.00000 /EncodeGrayImages true /GrayImageFilter /DCTEncode /AutoFilterGrayImages false /GrayImageAutoFilterStrategy /JPEG /GrayACSImageDict << /QFactor 0.15 /HSamples [1 1 1 1] /VSamples [1 1 1 1] >> /GrayImageDict << /QFactor 0.76 /HSamples [2 1 1 2] /VSamples [2 1 1 2] >> /JPEG2000GrayACSImageDict << /TileWidth 256 /TileHeight 256 /Quality 30 >> /JPEG2000GrayImageDict << /TileWidth 256 /TileHeight 256 /Quality 30 >> /AntiAliasMonoImages false /CropMonoImages false /MonoImageMinResolution 900 /MonoImageMinResolutionPolicy /OK /DownsampleMonoImages true /MonoImageDownsampleType /Average /MonoImageResolution 600 /MonoImageDepth -1 /MonoImageDownsampleThreshold 1.00000 /EncodeMonoImages true /MonoImageFilter /CCITTFaxEncode /MonoImageDict << /K -1 >> /AllowPSXObjects false /CheckCompliance [ /None ] /PDFX1aCheck false /PDFX3Check false /PDFXCompliantPDFOnly false /PDFXNoTrimBoxError true /PDFXTrimBoxToMediaBoxOffset [ 0.00000 0.00000 0.00000 0.00000 ] /PDFXSetBleedBoxToMediaBox false /PDFXBleedBoxToTrimBoxOffset [ 0.00000 0.00000 0.00000 0.00000 ] /PDFXOutputIntentProfile (U.S. Web Coated \050SWOP\051 v2) /PDFXOutputConditionIdentifier (CGATS TR 001) /PDFXOutputCondition () /PDFXRegistryName (http://www.color.org) /PDFXTrapped /Unknown /Description << /ENU <FEFF005500730065002000740068006500730065002000530061006700650020007300740061006e0064006100720064002000730065007400740069006e0067007300200066006f00720020006300720065006100740069006e006700200077006500620020005000440046002000660069006c00650073002e002000540068006500730065002000730065007400740069006e0067007300200063006f006e006600690067007500720065006400200066006f00720020004100630072006f006200610074002000760037002e0030002e00200043007200650061007400650064002000620079002000540072006f00790020004f00740073002000610074002000530061006700650020005500530020006f006e002000310031002f00310030002f0032003000300036002e000d000d003200300030005000500049002f003600300030005000500049002f004a0050004500470020004d0065006400690075006d002f00430043004900540054002000470072006f0075007000200034> >> /Namespace [ (Adobe) (Common) (1.0) ] /OtherNamespaces [ << /AsReaderSpreads false /CropImagesToFrames true /ErrorControl /WarnAndContinue /FlattenerIgnoreSpreadOverrides false /IncludeGuidesGrids false /IncludeNonPrinting false /IncludeSlug false /Namespace [ (Adobe) (InDesign) (4.0) ] /OmitPlacedBitmaps false /OmitPlacedEPS false /OmitPlacedPDF false /SimulateOverprint /Legacy >> << /AllowImageBreaks true /AllowTableBreaks true /ExpandPage false /HonorBaseURL true /HonorRolloverEffect false /IgnoreHTMLPageBreaks false /IncludeHeaderFooter false /MarginOffset [ 0 0 0 0 ] /MetadataAuthor () /MetadataKeywords () /MetadataSubject () /MetadataTitle () /MetricPageSize [ 0 0 ] /MetricUnit /inch /MobileCompatible 0 /Namespace [ (Adobe) (GoLive) (8.0) ] /OpenZoomToHTMLFontSize false /PageOrientation /Portrait /RemoveBackground false /ShrinkContent true /TreatColorsAs /MainMonitorColors /UseEmbeddedProfiles false /UseHTMLTitleAsMetadata true >> << /AddBleedMarks false /AddColorBars false /AddCropMarks false /AddPageInfo false /AddRegMarks false /BleedOffset [ 9 9 9 9 ] /ConvertColors /ConvertToRGB /DestinationProfileName (sRGB IEC61966-2.1) /DestinationProfileSelector /UseName /Downsample16BitImages true /FlattenerPreset << /ClipComplexRegions true /ConvertStrokesToOutlines false /ConvertTextToOutlines false /GradientResolution 300 /LineArtTextResolution 1200 /PresetName ([High Resolution]) /PresetSelector /HighResolution /RasterVectorBalance 1 >> /FormElements true /GenerateStructure false /IncludeBookmarks false /IncludeHyperlinks false /IncludeInteractive false /IncludeLayers false /IncludeProfiles true /MarksOffset 9 /MarksWeight 0.125000 /MultimediaHandling /UseObjectSettings /Namespace [ (Adobe) (CreativeSuite) (2.0) ] /PDFXOutputIntentProfileSelector /DocumentCMYK /PageMarksFile /RomanDefault /PreserveEditing true /UntaggedCMYKHandling /UseDocumentProfile /UntaggedRGBHandling /UseDocumentProfile /UseDocumentBleed false >> ] /SyntheticBoldness 1.000000 >> setdistillerparams << /HWResolution [288 288] /PageSize [612.000 792.000] >> setpagedevice