S-L Outcomes, Reflection, and Assessment

This section describes a process that faculty can use for structuring the reflection process. The approach described in the following section can be used regardless of specific S-L educational outcomes. However, before following the process outlined in this section, faculty must carefully consider the links between S-L outcomes, reflection, and assessment.

For reflection to be effective:

1. Outcomes must be specified precisely. If outcomes are specified too broadly (e.g. communication skills) it may be difficult to devise appropriate reflection activities and to develop appropriate assessment techniques. Faculty can use a wide range of S-L outcomes (such as those discussed in previous sections) as a starting point for establishing specific S-L outcomes.

2. Before designing reflection, faculty must select appropriate service activities. Unless service activities are structured in a way that supports the acquisition and reinforcement of specific disciplinary content and desired educational outcomes, it will not be feasible to incorporate reflection to support those outcomes.

3. After designing outcomes and service activities, faculty must consider the question: How can reflection be used to enhance a particular outcome? A key issue to consider is the timing of the reflection. Reflection activities before, during, and after the service activity can contribute to an outcome in different ways.

4. Finally, faculty must consider how the outcomes will be assessed. Several resources are available on S-L assessment (see Roufs 2000 for a bibliography of S-L assessment resources). A detailed discussion of S-L assessment is beyond the scope of this web site; however, one key issue to consider in designing S-L is the link between reflection and assessment. Assessment can have multiple purposes. One important purpose is to provide feedback to students on what is expected of them, what they have done well, what they need to improve on and how (cf. Learn and Serve America National Service-learning and Assessment Study Group 1999). Reflection is a key part of such assessment. Structured reflection activities conducted on an ongoing basis provide a means for assessing student progress towards S-L goals and for providing feedback. Since faculty may not be able to directly observe the service activities of students in detail, reflection may also be the key to the assessment of S-L for grading purposes. For example, S-L may be graded on the basis of a presentation and/or integrative term paper.

Example
The example below shows how the design of reflection is facilitated by (1) establishing clear outcomes, (2) structuring service activities appropriately, and (3) by considering the linkage between reflection and assessment. The project management outcome category was selected as it illustrates many of the ideas presented in this section. The specific outcomes listed in the table below have been adapted from the American Institute of Certified Public Accountants (AICPA) Core Competency Framework.

Specification of Outcomes

A precise specification of outcomes is the key to designing reflection. The AICPA Core Competency Framework provides a detailed list of specific outcomes in the project management category. As shown in the table below, related elements of project management competencies have been grouped into four categories: (1) establishing community/educational outcomes, (2) utilization of resources, (3) project execution and assessment, and continuity.

Project Management Outcome Category Specific Outcomes (Adapted from AICPA 2001). S-L Project Proposal
Assessment: content analysis of S-L proposal
Final Report

Assessment: content analysis of final report
Project goals Determines community outcomes and learning goals - Intended community outcomes
- Intended learning outcomes
- Community impact
- Lessons learned
Resource use - Realistically estimates time and resource requirements.
- Allocates project resources to maximize results.
- Effectively manages human resources that are committed to the project
- External resources available (other organizations, professionals)
- Student interests, time and skills
- Utilization of external resources and human resources
-Ways to improve resource utilization in future S-L
Project execution and assessment - Effectively facilitates and controls the project process. - Measures project progress.
- Takes corrective action as needed.
- Planned process for communication with stakeholders (community, partners, faculty, students, professionals)
- Assessment measures and approach
- Actual communication and possible improvements
- Assessment results
Project continuity Sees projects through to completion or orderly transition - Plan for continuation
-Communicating prior successes and corrective measures based on lessons learned
 



Designing Service Activities
For S-L to be effective in accomplishing specific outcomes, service activities must be appropriately structured. For developing project management skills through S-L, students must have an opportunity to practice each of the four categories of skills listed above. Thus students must be given an opportunity to participate in developing community/learning outcomes, planning resource use, executing and managing projects in the community, and in planning for the continuity of projects.

Obviously, different S-L goals might be more appropriate in different settings. For example, project management goals may be more feasible after students have some prior S-L experience. Also, goals such as project management may be more appropriate for a capstone course where S-L constitutes a significant proportion of the course grade. Alternatively, students can be provided an opportunity to practice project management skills in other settings (e.g. student organizations) to complement S-L projects in courses. Stott et al. (2000) provide an example of this approach. In their implementation, students in a Pi Tau Sigma Chapter (National Mechanical Engineering Honors Society) provide support for S-L in a capstone design course. Pi Tau Sigma students are responsible for the following project management activities: (1) conveying program intent to community partners, (2) inviting proposals, (3) sending out mailings and making telephone calls to develop tractable problems, (4) screening proposals, (5) communicating with course leadership, and (6) day-to-day running of the program (establishing meetings, setting deadlines).

Reflection
Once the outcomes have been established and service activities have been appropriately structured, faculty can design reflection. Reflection activities can be incorporated before, during, and after the service experience. The table above shows two examples of reflection activities. The first activity requires students to write a proposal (prior to service) and the second activity involves an integrative report (after service). Other activities can be used during the service (e.g. structured journal).

A key point to note is that reflection activities at different stages of the S-L experience may contribute to the intended learning outcomes in different ways. In the above example, reflection activities prior to the service can focus on helping students learn how to plan and organize service activities in the community. Reflection during/after the experience can focus on helping students reflect on the actual outcomes in relation to S-L plans.

Assessment
The table above also lists possible ways in which the two reflection activities can be assessed. Faculty can use content analysis to evaluate student responses and to provide feedback. Again, note the critical importance of clear objectives to assessment. If we specify the outcome broadly (project management), it may be difficult to perform content analysis. Further, it would be difficult to communicate expectations to students. It is not enough to tell students that they will be assessed on project management. Students need to understand what we mean by project management skills and how these skills will be assessed.

A more detailed specification of objectives, as in our table, facilitates this process. For example, students can learn that one of the components of project management involves planning the use of resources available for a project. Further, they can understand that their project management skills will be assessed on the basis of how they use external resources (other organizations, professionals) and internal resources (e.g. student interests, time and skills) in their service activities.

Faculty can also explain to students how the outcome will be assessed at different stages of S-L. For example, prior to the experience students can be assessed on how they plan to communicate with various stakeholders during S-L. During the project they could be assessed on actual efforts to communicate with and coordinate their activities with others.

References

American Institute of Certified Public Accountants (AICPA). 2001. AICPA Core Competency Framework for Entry into the Accounting Profession. New York, NY: AICPA http://www.aicpa.org/edu/corecomp.htm.


Gelman, S., B. Holland, et. al. 2001. Assessing Service-Learning and Civic Engagement: Principles and Techniques. Providence, RI: Campus Compact.


Learn and Serve America National Service-learning and Assessment Study Group. 1999. Service-learning and Assessment: A Field Guide for Teachers. http://www.umn.edu/~serve/guide1.pdf


Roufs A. L. 2000. Assessment and Evaluation in Service-learning Topic Bibliography. http://www.umn.edu/~serve/res/bibs/assessment.pdf


Stott N. W., W. W. Schultz, Brei D., Hoffman D. M., Markus G. 2000. ProCEED: A Program for Civic Engagement in Engineering Design. American Society for Engineering Education Annual Conference Proceedings. http://www.asee.org/conferences/search/