State of California

Task Force on Institutional Accountability and Learning Assessment

21 November , 2000

• Executive Summary
• Introduction
• Guiding Principles and Definitions
• Purpose
• Process Summary
• Roles and Responsiblities
• Elements of the Self-Study

EXECUTIVE SUMMARY

After an extensive study of academic program review processes and practices statewide and nationwide, the Task Force on Institutional Accountability and Learning Assessment proposes a revised academic program review process for Cal Poly. Some of the key features include:

• a mission-centric focus of program reviews
• a discipline-based program review that recognizes program distinctiveness and different disciplinary approaches to student learning
• a self-study that is defined, designed and conducted by the program faculty and encourages serious reflection on issues of interest and significance that is more conducive to program improvement
• the combination of internal and external reviews (peer review and/or specialized accreditation/recognition)
• the involvement of program faculty, students, community, campus administrators, and external experts in the discipline
• the involvement of College Deans in helping to design the review
• a program review team composed of (at least) four members who are knowledgeable in the discipline/field of the program under review
• a 1-2 day site visit conducted by the program review team and
• a feedback loop that includes the development of an action plan for improvement, jointly written by the program, the Dean and the Provost
• a six-year cycle for periodic reviews of all academic programs, including General Education, and centers and institutes
• the alignment of academic program review with planning, budgeting, and Cal Poly's accountability process for the CSU

Back to top

INTRODUCTION

In 1971, the California State University (CSU) Board of Trustees established an academic planning and program review policy (AP 71-32) requiring each campus to establish criteria and procedures for planning and developing new programs and conduct regular reviews of existing programs. CSU Executive Order No. 595 (PDF) calls for "regular periodic reviews of general education policies and practices in a manner comparable to those of major programs. The review should include an off-campus component." CSU Executive Order No. 729 (PDF) also calls for periodic reviews of centers, institutes, and similar organizations. These policies have been reaffirmed in The Cornerstones Report and in the Cornerstones Implementation Plan. In 1992 Cal Poly adopted the Academic Program Review and Improvement Guidelines establishing procedures for the conduct of academic program reviews. These procedures and recommendations for external reviews of programs have since been modified. Currently, the information requested from programs that undergo internal review includes descriptions of educational goals, instructional designs and methods, assessment methods and the data so collected, and the procedures for utilizing the collected information. Thus, there is an increasing interest toward incorporating principles that make individual courses and the general programs in which they reside more accountable for student learning.

The Task Force on Institutional Accountability and Learning Assessment was appointed and charged by the Provost "to propose a systematic and coordinated approach to addressing academic (and larger institutional) accountability and assessment issues" consistent with our institutional mission and values. We have used as guiding principles the need to build upon, integrate and implement the perspective and approaches contained in existing (Cal Poly and CSU) documents, and the desire to keep these approaches clear, concise and simple. Establishing consistency, while maintaining flexibility, in internal accountability, external accountability and reporting is crucial. The Task Force has applied this approach in preparing this document, Report on Institutional Accountability: Academic Program Review, and used the following documents as resources:

Cal Poly Mission Statement
Cal Poly Strategic Plan
Commitment to Visionary Pragmatism
Academic Program Reviews (Senate Resolution AS-383-92) (PDF)
Academic Program Review and Improvement Guidelines Change (Senate Resolution AS-425-94) (PDF)
External Review (Senate Resolution AS-496-98) (PDF)
Procedures for External Review (Senate Resolution AS-497-98) (PDF)
Program Efficiency and Flexibility (Senate Resolution AS-502-98) (PDF)
Program Review and Improvement Committee Bylaws Change (Senate Resolution AS-523-99)
Cal Poly Plan
Cal Poly's General Education Program
Cal Poly as a Center of Learning (WASC Self-Study)
Review of the Baccalaureate in the California State University
The Cornerstones Report
Cornerstones Implementation Plan
The CSU Accountability Process
Cal Poly's Response to the CSU Accountability Process

Back to top

GUIDING PRINCIPLES AND DEFINITIONS

Academic program review (APR) is a comprehensive and periodic review of academic programs, General Education, and centers and institutes. APR is a function of the Provost, in conjunction with the College Deans and the Academic Senate, and is coordinated by the Vice-Provost for Academic Programs and Undergraduate Education (VP-APUE).

Academic program review has as its primary goal, enhancing the quality of academic programs. Hence, it is an essential component of academic planning, budgeting, and accountability to internal and external audiences. APR is not a review of academic departments or other such administrative units. Each program, department (administrative unit) and college is responsible for their curricular decisions and programmatic offerings within existing resources. All such decisions shall be the purview of the faculty of the program, department (administrative unit) and/or college. Interdisciplinary programs, centers, and institutes also fall within the purview of this policy.

Academic program review of programs subject to professional or specialized accreditation/recognition will be coordinated to coincide with the accreditation/recognition or accreditation/recognition review, whenever possible. The document(s) developed for professional or specialized accreditation/recognition reviews may already provide the essential requirements of APR and thus, may also be used for this purpose. Although some programs may choose to use the self-study developed for their professional accreditation/recognition as one of the elements of the APR, it is important to note that accreditation/recognition reviews serve a different purpose than that of institutional academic program reviews.

The following definitions should help in distinguishing terms used throughout this document:

• Academic program is a structured grouping of course work designed to meet an educational objective leading to a baccalaureate or post-baccalaureate degree, or to a teaching credential.
• Centers, institutes and similar organizations are entities under the aegis of an administrative unit that "offer non-credit instruction, information, or other services beyond the campus community, to public or private agencies or individuals."
• Department is an administrative unit which may manage one or more academic program, center, institute or similar organization.
• The term program is used to mean an academic degree program, General Education program, center, institute or similar organizations subject to institutional review.
• The Program Administrator is the individual responsible for administrative authority of the Program, and is usually referred to as the Program Head, Chair, or Director.
• The self-study is to be designed and prepared by the Program Administrator and representative Program faculty, referred to in this document as the Program Representative(s).
• The (time) schedule for every academic program review is based on business, not calendar, days.

Back to top

PURPOSE

The goal of academic program review is to improve the quality and viability of each academic program. Academic program review serves to encourage self-study and planning within programs and to strengthen connections among the strategic plans of the program, the College and the University. Academic program reviews provide information for curricular and budgetary planning decisions at every administrative level.

PROCESS SUMMARY

The academic program review process is intended to close the circle of self-inquiry, review and improvement. The basic components of APR are:

• a self-study completed by the faculty associated with the Program,
• a review and site-visit conducted by a Program Review Team chosen to evaluate the Program, and
• a response to the Program Review Team's report, prepared by the Program Representative(s), the Program Administrator, the College Dean and the Provost.

Although details are contained throughout this document, the process can be summarized as follows:

1. The Provost and College Dean select and announce the programs to be reviewed at least one year prior to the review.
2. For each program under review, a Program Review Team (Team) is appointed and a schedule is established for the review. Willingness and availability of the Team members for the entire review process should be secured well in advance. Procedures and charge to the Team must also be communicated and acknowledged by each member of the Team prior to the review.
3. The Program representative(s), Program Administrator, College Dean and Provost negotiate the content or theme of the self-study and establish a schedule for completion of the review. An essential element of the self-study must address student learning.
4. The Provost, in consultation with the College Dean, the Program Administrator, and the Chair of the Academic Senate (or designee) will determine whether the accreditation/recognition review process covers the essential elements of APR in accordance with any CSU or Cal Poly mandated requirements.
5. The Program representative(s) conducts the self-study and submits copies to the VP-APUE for distribution to the Team, College Dean and Provost at least 45 days prior to the scheduled site-visit.
6. The Team reviews the self-study, requesting additional materials as needed, and conducts a 1-2 day site-visit of the Program. The site-visit is coordinated by the VP-APUE and should include meetings with the Program faculty, staff, students and administrators.
7. The Team submits a draft report to the VP-APUE within 21 days of the site-visit for distribution to the Program. The Program representative(s) reviews the draft for accuracy and facts of omission.
8. The Team submits the final report (consisting of findings and recommendations) to the VP-APUE for distribution to the Program, College Dean and Provost within 45 days of the site-visit.
9. The Program representative(s) prepares a formal response to the Team report within 21 days and submits it to the VP-APUE for distribution to the College Dean and Provost.
10. The Program representative(s), the Program Administrator, the College Dean and the Provost hold a "follow-up" meeting to discuss final APR report (the Program's self-study, program review Team report, and program response).
11. The College Dean, in collaboration with the Program Administrator, submits to the Provost an action plan consistent with the recommendations of the APR report and how the program fits into the College mission and strategic plan.
12. A copy of the APR report and the action plan is forwarded to the Academic Senate.

Back to top

ROLES AND RESPONSIBILITIES

Academic program review is a function of the Provost, in conjunction with the College Dean and the Academic Senate, and is coordinated by the VP-APUE. As required by the CSU Board of Trustees, academic programs "should be reviewed periodically at intervals of from five to ten years." While past campus practice required that program reviews be undertaken at five year intervals, the inclusion of reviews of centers and institutes suggests that the review cycle be modified. Therefore, all academic programs, including General Education, centers, and institutes will be reviewed on a six-year cycle. This schedule may be accelerated in individual cases either at the discretion of the Provost or College Dean or in compliance with recommendations from prior program reviews. In addition to the selection of reviewers, the Academic Senate will have the opportunity to suggest programs or programmatic areas for review. Wherever possible, APR's will coincide with specialized accreditation/recognition, other mandated reviews, or with reviews for new degree programs. For example, engineering programs are subject to accreditation/recognition by ABET on a six-year cycle, whereas business programs are subject to accreditation/recognition on a ten-year cycle. Hence, it is appropriate to consider that engineering programs be reviewed every six years, and that business programs be reviewed every five years. Programs in related disciplines or with similar missions should also be reviewed concurrently.

Each academic program review is conducted by a singular Program Review Team. It is expected most reviewers be knowledgeable in the discipline/field of the program under review. The Team will normally be composed of (at least) four members to be selected using the following guidelines:

• One member chosen by the Dean of the college whose program is under review. This person may be either a current Cal Poly faculty member (from a College different than that of the program under review) or an external reviewer.
• One or two current Cal Poly faculty members (from a College different than that of the program under review) chosen by the Academic Senate Executive Committee.
• Two external members representing the discipline of the program under review chosen by the President.

The composition of the Team may change when the academic program review coincides with a specialized accreditation/recognition review. In this case, it is incumbent on the individual(s) chosen by the Academic Senate Executive Committee to provide the necessary institutional review.

The VP-APUE will appoint one of the Team members to be Chair and will coordinate all reviews, in accordance with the established schedule, to ensure that the process is both efficient and fair.

The academic program review process can be summarized in three parts: the self-study, the review and site-visit, and the response (follow-up).

Back to top

ELEMENTS OF THE SELF-STUDY

In preparation for the review, the Program will undertake a thorough self-study that is defined and designed by the Program faculty in conjuction with the College Dean and Provost. It establishes the program's responsibility for its own mission, purpose and curricular planning within the context of the College and University missions. To accomplish this objective the report should consist of two parts:

Part I - A inquiry-based, self-study, the content or theme of which is to be proposed by the Program and negotiated with the College Dean and Provost. An important element of the content or theme chosen for the self-study must address student learning. To accomplish this, the self-study should include the following points as appropriate or relevant to the Program mission.

• Statement of purpose, quality, centrality, currency, and uniqueness (where appropriate)
• Principles and processes for student learning outcomes and assessment methods
• Strategic plan for program development, planning and improvement

Part II - General information that consists of data appropriate and relevant to the Program mission. (Most of this data is part of that already required for Cal Poly's Response to the CSU Accountability Process and may be obtained with assistance from the office of Institutional Planning and Analysis.)

• Faculty, staff and students engaged in faculty research, scholarship and creative achievement, active learning experiences and academically-related community service or service learning
• Integration of technology in curriculum and instruction
• Evidence of success of graduates (e.g., graduates qualifying for professional licenses & certificates, graduates engaged in teaching, government, or public-service careers)
• Description of adequacy, maintenance and upkeep of facilities (including space and equipment) and other support services (library, and technology infrastructure)
• Alumni satisfaction; employer satisfaction with graduates

When requested by a program, the Provost, in consultation with the College Dean, the Program Administrator, and the Chair of the Academic Senate (or designee will determine whether an accreditation/recognition review process covers the essential elements of APR in accordance with any CSU or Cal Poly mandated requirements.

The Program will provide copies of the two-part, self-study to the VP-APUE for distribution to the Team, College Dean and Provost.

Back to top

THE PROGRAM REVIEW TEAM SITE-VISIT AND REPORT

The Team will receive a copy of the Program's self-study document at least 45 days prior to a proposed site-visit. All members of the Team should read the self-study and are encouraged to request additional materials as needed. A 1-2 day site-visit will be coordinated by the VP-APUE, but travel arrangements and expenses for external reviewers are the responsibility of the College Dean whose program is under review. These might include travel, lodging, meals, and honorarium, etc.

The Team should also be provided with sufficient time to discuss among themselves how to proceed with the visit. This would preferably occur at the beginning of the site-visit. It is expected that during the site-visit, the Team will have access to faculty, staff, students and administrators, and any additional documentation or appointments deemed necessary for the completion of the review.

The Team should also be given the opportunity to meet with the Program representative(s), the Program Administrator, the College Dean and/or Provost to discuss possible outcomes of the review at the end of the site-visit. It is the responsibility of the chair of the Team to ensure that all members of the Team work together throughout the review and that the final report reflects the recommendations of all reviewers.

Within 21 days of the site-visit, the Team will provide a draft of the report to the VP-APUE for distribution to the Program. The report should address the major issues facing the program and the program's discipline within the larger context of the College and University mission and strategic plan, and should suggest specific strategies for improvement. The Program representative(s) will then review the draft report solely for accuracy and facts of omission. The final Team report (consisting of findings and recommendations) should be completed within 45 days of the site-visit and forwarded to the VP-APUE for distribution to the Program, the College Dean and the Provost.

Back to top

RESPONSE (FOLLOW-UP) TO ACADEMIC PROGRAM REVIEW

The effectiveness of academic program review depends on the implementation of the appropriate recommendations contained in the APR report. Hence, a follow-up meeting will be scheduled by the VP-APUE, to include the Provost, the Program Administrator, the Program Representative(s),and the College Dean. The purpose of this meeting is to discuss the recommendations of the Team report, the Program's response, and to develop an action plan for achieving compliance and improvement by the program. The results of this meeting will be summarized in a written document to be prepared by the College Dean and distributed to the Program and the Provost. This document will inform planning and budgeting decisions regarding the Program. A copy of the APR report and the action plan will be forwarded to the Academic Senate. The Provost will prepare a narrative summary of Cal Poly's academic program review activity for the CSU Chancellor's Office as part of the annual reporting for the CSU Accountability Process, with a copy to the Academic Senate.

PROCESS FLOWCHART FOR ACADEMIC PROGRAM REVIEW

®

College Deans and Provost select/announce the programs to be reviewed (at least one year prior to the review) and a timetable is set.

­

College Deans, Academic Senate Executive Committee and President appoint Program Review Team.

­

The Program representative(s), College Dean and Provost negotiate the content or theme of the self-study.

­

The Program representative(s) conducts the self-study. The self-study is distributed to the Program Review Team, College Dean and Provost at least 45 days prior to the scheduled site-visit.

­

The Program Review Team conducts a 1-2 day site-visit. The Team is provided access to the Program faculty, staff, students and administrators.

­

The Program representative(s) reviews a draft report from the Program Review Team for accuracy and facts of omission. The Team submits the final program review report for distribution to the Program, College Dean and Provost.

­

The Program representative(s) prepares a formal response to the Team report for distribution to the College Dean and Provost.

­

Program Administrator, College Dean, Provost and VP-APUE hold a "follow-up" meeting to discuss the APR report and program response.

­

Program Administrator and College Dean submit to the Provost an action plan for Program improvement. A copy of the APR report and action plan are forwarded to the Academic Senate.

  ­

The VP-APUE maintains a record of all academic program reviews.

A CHECKLIST FOR ACADEMIC PROGRAM REVIEW

A sample timetable and checklist for the academic program review process is presented here. Some of these events may occur concurrently.

TARGET DATE

ACTIVITY

RESPONSIBILITY

October

Programs scheduled for review are selected and announced one year prior to the review, and a timetable is set.

College Deans and Provost

Prior to site-visit

Program Review Team is appointed.

College Deans, Academic Senate Executive Committee, President

Prior to site-visit

Participation of Team members is confirmed, Chair of Team is appointed

VP-APUE

Prior to site-visit

Content/theme of self-study is proposed and negotiated.

Program representative(s), College Dean and Provost

Prior to site-visit

Program representative(s) conducts the self-study.

Program

At least 45 days prior to site-visit

Self-study document is provided to VP-APUE for distribution to Team, College Dean and Provost.

Program and VP-APUE

At least 45 days prior to site-visit

Team reviews the Program’s self-study.

Team

Site-visit

The Team conducts a 1-2 day site-visit and is provided access to the Program faculty, staff, students and administrators.

Team, Program, College Dean, Provost and VP-APUE

At most 21 days after the site-visit

Team’s draft report is submitted to VP-APUE for distribution to the Program.

VP-APUE

At most 45 days after the site-visit

Program representative(s) reviews the Team draft report for accuracy and facts of omission.

Program

At most 45 days after the site-visit

Team submits final program review report to VP-APUE for distribution to Program, College Dean and Provost.

Team and VP-APUE

At most 60 days after the site visit

Program representative(s) prepares response to the Team Report and submits the response to VP-APUE for distribution to College Dean and Provost.

Program and VP-APUE

Within 90 days after site-visit

Follow-up meeting to discuss academic program review report.

Program Administrator, College Dean, Provost and VP-APUE

Within 120 days after site-visit

Action plan for Program improvement is submitted to the Provost and forwarded to the Academic Senate.

Program Administrator and College Dean

October (of following year)

Programs scheduled for review are selected and announced.

College Deans and Provost

Anny Morrobel-Sosa, Chair (Special Assistant to the Provost, Materials Engineering)
Denise Campbell (Special Assistant to the Provost)
W. David Conn (Vice Provost for Academic Programs and Undergraduate Education)
Susan Currier (Associate College Dean, College of Liberal Arts)
James Daly (Statistics)
Myron Hood (Mathematics, Academic Senate Chair)
Steven Kane (Disability Resource Center)
Roxy Peck (Associate College Dean, College of Science and Mathematics)
Thomas Ruehr (Soil Science)