Academic Senate Resolution

Program Review

(Approved 2/24/06) 

 

  1. Whereas, The UHCC policy on PR (UHCCP #5.202, Review of Established Programs, October 2005) allows for each college to develop its own program review (PR) procedures; and
  2. Whereas, Hawai‘i Community College (HawCC) developed an initial PR process under the guidance of the Vice Chancellor for Academic Affairs (VCAA) who established the College Effectiveness Review Committee (CERC) to assist in this endeavor; and  
  3. Whereas, HawCC implemented a new process for program review (PR) during the 2005/06 academic year; and
  4. Whereas, Many individuals including the PR initiators, PR data providers, and PR assessors (College Effectiveness Review Committee, College Council and Academic Senate Executive Committee members) have contributed and are continuing to contribute countless hours to this new PR process; and
  5. Whereas, Through the PR process, 12 PR’s were submitted in time for the November 2005 interim site visit by ACCJC; and
  6. Whereas, Dr. Barbara Beno, Executive Director of the Accrediting Commission for Community and Junior Colleges (ACCJC), wrote the article, “What Accreditors Expect from College Program Review,” (Accreditation Notes, ACCJC, April 2003, www.accjc.org/documents/April%202003.pdf), in which she outlines the meaning of a program, and summarizes what the PR cycle should include:  a current description; evaluation of sufficiency or “good enough” to meet “institution’s pursuit of excellence;” planning for improvement; implementation of plans; and evaluation of the effectiveness of actions to achieve desired results; and
  7. Whereas, Subsections of Standard II.A.2 of the ACCJC guidelines for Self-study emphasize the central role of the faculty in ensuring and assessing effectiveness and quality of programs; and
  8. Whereas, Other institutions have streamlined their PR processes by:  having a longer PR cycle (4 or 5 years instead of 3); reducing levels of subsequent campus assessment of PR’s after they are filed by including the use external evaluators; allowing self-studies completed within a specified window of time for outside accreditation to be submitted in place of a PR; specifying maximum page lengths for an entire PR and its individual sections; and utilizing on-line preparation (e.g., software to generate charts, etc.), submission, and assessment of PR’s; now therefore be it

 

Resolved, That the HawCC Academic Senate work with the VCAA to evaluate the current PR process to see if the criteria used to evaluate a program culminate in a determination of program effectiveness and to determine if it is efficient, effective and emphasizes the central role of the faculty by:

 

1.ensuring that elements included in the PR are aligned with ACCJC Standards as identified in Whereas clause #6

2.reducing administrative assessment in the initial stages

3.reviewing measures to streamline the PR process such as those identified in Whereas clause #8

 

 

2.

Recommendations to the Executive Committee

of the HawCC Academic Senate

Regarding Program Review

February 13, 2006

 

Background on HawCC’s Program Review (PR) process

Much of the current 3-year cycle of the PR process was developed by the UHCC’s via the VCAA/DOI’s in response to questions raised by the Accrediting Commission for Community and Junior Colleges (ACCJC) on how assessment of student learning was being done at each campus. As a result, a list of data elements was developed at the UHCC system level to attempt to evaluate program efficiency explicitly with quantitative data. At HawCC, 13 data elements were chosen and a 10-part template was developed for instructional programs to use while writing up the reviews. In November, the 12 programs and units that were selected for the first cycle each submitted a program or unit review by the deadline that also coincided with the campus visit made by the ACCJC mid-term evaluation visiting team.  

 

At a February 2, 2006 meeting with the College Effectiveness Research Committee (CERC) and program initiators to review this first round of the new process, Shawn Flood, HawCC Institutional Researcher, provided additional background as a framework for evaluating the process. Shawn’s document should be reviewed by the Senate Executive Committee before a final PR model is forwarded to the Academic Senate.  

 

ACCJC recommendations for program review

Dr. Barbara Beno, Executive Director of ACCJC, wrote an article (“What Accreditors Expect from College Program Review,” Accreditation Notes, April 2003; www.accjc.org/documents/April%202003.pdf), in which she outlines the meaning of program review and how one should be conducted. The following summarizes her article:

 

What is program review? Program review is a look at “the effectiveness of an ‘educational program’” over time and from different perspectives. Dr. Beno says a “program” is what “accreditors mean [to be] a certificate or degree program, a coherent educational experience such as a tutoring or orientation program, a co-curricular learning program, or even an academic discipline (e.g., the social science ‘program’).” (p.1) The PR “cycle” has “several distinct conceptual steps: a precise and accurate description of things as they exist, evaluation of whether those things are sufficient or appropriate or ‘good enough’ to satisfy the institution’s pursuit of excellence, planning for needed improvement; implementation of those plans, and evaluation of the effectiveness of the actions taken in achieving the desired results.” (pp.1-2) The actions for improvement need to be evaluated for effectiveness and should become part of the next program reviews. This process of evaluation and re-evaluation establishes whether there is improvement over time.

 

How should program review be conducted? Dr. Beno further clarifies the PR process by giving a list of questions that can be used by an institution as the basis for evaluation. “An institution can start by examining the stated mission, purpose, or goals of a program, and what a program is doing to achieve that mission.” (p.2) Once the mission is described and evaluated, “results or program effectiveness” will then need to be examined. “An effective program is one that achieves its goals, but the notion of efficiency is also inherent in the idea of effectiveness. Hence, an effective program also uses its resources as efficiently as possible.” (p.2) The development of a planning agenda to address needed improvements is the next step in the PR process. Whether plans have been implemented is then evaluated. Finally, evaluation of the impact of changes is done to determine if the planned changes have resulted in improvements.

 

Summary of concerns about HawCC’s current PR process:

 

Suggestions to involve faculty more in HawCC’s PR process and to streamline it:

·        Work with VCAA to modify process so that it continues to follow guidelines of UHCCP #5.202 but is more faculty-driven, adapted to HawCC’s needs and utilizes other recognized documents for evaluating program effectiveness such as the ACCJC Standards for Accreditation (UHCCP #5.202 Section 9 allows for UHCC’s to supplement “common set of program review data elements” with other elements, such as those identified in Dr. Beno’s article; for example, “program effectiveness”) 

·        Lengthen the cycle from 3 years to 4 or 5 years (UHCCP #5.202 Section 5 allows for a 5 year review cycle)

·        Decrease the layers and time it takes for faculty and/or administrative assessment after the initial PR is filed by:

o Specifying a maximum page length for the entire PR and for each section. For example, UHH’s PR has 8 parts with page lengths specified for each: executive summary (1-2 paragraphs), background (1 page), program organization and performance narrative (2 pages including 11 data elements), student learning outcomes (2 pages), current resources (1 page), department chair’s evaluation (1 page), broad statement of future goals (1 page), and appendices including external evaluation

o Having initiators fil PR on HawCC website using software funded by HawCC (KMR’s software/template for example)

1.      PR is password protected

2.      Assessment of PR

·  Utilize external evaluators (UHH VCAA hires an external evaluator who is a recognized expert in the field and has been selected from a list of 3 ranked names with CV submitted by the program’s department chair)

o After receiving and reading the PR, external evaluator schedules site visit meetings with department faculty and students in program, appropriate support staff (i.e, library, counseling, etc.) and for HawCC CTE, advisory councils

o External evaluator submits written report within 1 month that is appended to PR; Department Chair or Unit Head may respond to consultant’s report

·  And/or submit PR to Academic Senate for initial (or simultaneous with external) assessment

·  Department Chair or Unit Head may respond to Academic Senate’s or External Evaluator’s assessment

3.      PR with assessment by Academic Senate or External Evaluator (and department response, if any) are forwarded to VCAA (may want to include this: VCAA schedules an open meeting for those interested to address issues about a specific program review; note-taker will append notes from this meeting to PR) 

4.      VCAA forwards PR with Academic Senate assessment and/or External Evaluator assessment (and, if included, notes from open meeting) to Administrative Team for strategic planning, biennium budget or other planning for funding priorities

 

Conclusion

Any new policy and process as comprehensive as the PR will require collegial planning by faculty, staff and administration in the initial implementation stages. More importantly, a collaborative spirit that encourages flexibility and adaptability will allow that process to be refined as needed.

 

Submitted by Ellen Okuma