AQUAD
The Academic Quality Assessment and Development (AQUAD) is an external review process for assessing the core academic functions of each department or program at the university, including teaching and learning; research, professional, and creative activity; and public service and academic outreach. The AQUAD review provides a rigorous quality assessment, identifying strengths and targeting areas for growth and development. In this section, one will find the AQUAD 1999 Report, the revised AQUAD guidelines system, and the planning and self-study templates.
AQUAD 1999 Report
Procedures for Review of Academic Programs
University of Massachusetts Boston (submitted April 27, 1999)
Committee Membership, Charge, and Working Process
On October 6, 1998, Provost Lyons charged an advisory committee with responsibility to recommend campus-level procedures for implementing the Board of Trustee's AQUAD component of the University Performance Measurement System . The University AQUAD guidelines require each campus to develop on ongoing system of quality assessment at the department or program level . The primary purpose of such assessment is described as "diagnostic", and is intended to be used to enhance academic quality. The Provost asked the committee to do three things: "to make the guidelines provided by the President's office for the AQUAD procedure specific," "to recommend a review cycle for all campus programs," and "to rationalize the AQUAD and APR process in such a way so as to meet the mandate of AQUAD, to accomplish the goals of the APR process and to do both with a reasonable expenditure of campus time and energy."
The committee has generally met twice a month since receiving our charge . Throughout our deliberations, we have had the regular involvement of the Office of Institutional Research and Policy Studies, which has been particularly useful in the development of the data needed to complete any review. We have also consulted with academic deans, some department chairs, and others in the university, who have provided feedback about our plans and suggestions for improvements.
There are two general principles underlying both our deliberations and our recommendations; that any procedure be as flexible as possible and that it make as much use of existing mechanisms as possible. The committee recommends a strategic planning and assessment procedure with built-in flexibility of two kinds: colleges and programs are able to tailor the procedure to serve particular needs they may have, and the campus may revise the procedure over time so that what we learn from our initial efforts can inform later efforts. The committee also made significant use of existing review structures at the campus, particularly the Individual Graduate Program Review.
In January, 1999, we delivered our Interim Report to the Provost. This provided an outline and summary of the proposed implementation procedure as well as some discussion of the benefits of instituting such a planning and assessment mechanism and of the costs of the mandated periodic external reviews. Since that time the committee has worked to make the planning, assessment and evaluation procedure as detailed as possible. In order to complete this task we have had to inform ourselves about how to develop a student outcomes assessment program and how to go about setting appropriate program goals and objectives. Two committee members attended a Project Kaleidoscope conference on assessment in March, 1999, where we learned a great deal about how to go about these tasks. As a result of our efforts, we have developed several documents: a set of recommendations regarding the detailed procedures to be used; a description of how to start the planning process by
developing a mission statement, goals and objectives; a table indicating the review cycle for the campus; guidelines to be used for individual program reviews; a template to be used by programs in completing their procedures. We are also undertaking a pilot review of the Sociology Department this Spring, and we recommend using this review to improve on the procedures suggested here.
One part of our charge remains incomplete; we have not completed our deliberations on the integration of the AQUAD and the APR process. The Provost has agreed to the postponement of that item until the next academic year.
Overview of the AQUAD Review Process
Planning and assessment are complementary processes in the analytic review of academic programs. Faculty in all departments engage in some form of planning and assessment when they reflect on and deliberate about their work. This document sets forth a process for making these activities explicit, consistent across units, and open to periodic examination by others. Planning and assessment are undertaken at all levels within a university: department, college, and campus. This document focuses on departments or programs, including the generation of information by departments for use at college or campus levels.
The terms department and program are used interchangeably in this document. They refer to the primary administrative unit into which faculty are organized. They are meant to include all graduate and undergraduate degree programs administered within the unit, whenever this is meaningful and appropriate for academic planning and assessment. Departments have the organizational structure for carrying out a review process, and most faculty see the department as a place to coordinate diverse graduate and undergraduate programmatic activities. Reviews conducted at this level will likely minimize the expenditure of time and money while maximizing informational gain. Nevertheless, it will sometimes be appropriate to review programs that do not fit the departmental model. Some interdisciplinary programs may not be well served by departmentally housed reviews; some graduate programs may warrant separate review, even when housed within a department; general education programs are by their nature extra-departmental and require separate review; and some professional colleges may best be reviewed as a single unit. Furthermore, there is some degree of flux to programmatic structures, including recent growth of intercollegiate programs. Because of these complexities, we recognize that the programs listed for review in this document are subject to redefinition as a result of consultation among the relevant Deans, Chairs, and Directors, and the approval of the Provost.
The process for reviewing academic programs will occur in repeated cycles of planning, formative assessment, and modification that culminate every seven years in summative self-assessment and review by administrators and external experts. The recommended chronology for a full cycle is listed below. It is recognized that the initial cycle will be truncated to varying degrees for specific programs as this new review process is phased in on campus. This will necessarily result in some degree of compromise in the initial stage. For example, programs reviewed in the first years will have time to develop some assessment strategies but insufficient time to collect information with these strategies. These problems should disappear once the process is fully regularized.
The AQUAD process is seen as an integral part of the annual work of a department. It has overlapping but discrete stages of planning, assessment, and evaluation which are described below. Recommended timing for each stage is listed, although it is recognized that few departments will be able to use this timing during their first review. Departments and deans should work together with the Provost to meet this schedule or to plan a way to approximate it during the initial cycle. Accredited programs will follow the same chronological sequence, although the cycle length may differ slightly.
All parties are encouraged to exercise some flexibility as we begin this new review process. Nevertheless, it is extremely important that departments and their deans begin the process of planning and annual data collection as early as possible. Particular attention should be given to the development of assessment strategies for student learning outcomes and other program goals.
The review cycle includes the following chronological steps [recommended timing in brackets]:
2. Department reviews its plan each year, revising it if desired. Department and Office of Institutional Research and Policy Studies (OIRP) engage in annual data collection [Yrs 1-7].
3. Department engages in mid-period review of plan and annual data with Dean(s) and adjusts plan as needed [Yr 3 or 4].
4. Department prepares a self-study (see attached template); Dean(s) consult with Department to select external reviewers [late Yr 6-early Yr 7].
5. Team of external reviewers assesses the program [Yr 7].
6. Department (program), Dean(s), and Provost evaluate program based on self-study and external review; Provost sends executive summary to President [before June Yr 7].
7. Department, Dean(s), and Provost follow through on evaluation; Department returns to step 1 for second iteration of process, including responses to recommendations arising from the review [end of Yr 7; Yr 1 of round 2].
Planning, Assessment, and Evaluation
Planning and assessment have assumed increased importance throughout academia and have consequently received attention at the national level. National organizations concerned with higher education have fostered this discussion in various conferences and workshops. Accrediting agencies are rewriting their guidelines to call for more information in these areas. The following sections describe in a general way what is included in academic planning and assessment and summarizes some practices that seem to work fairly well at other institutions. The bibliography attached to this document lists print and internet sources for those wishing more detail.
Planning
Planning at the departmental level reflects the identity or purpose of the department with respect to three different domains of action: the discipline or profession; the students and other individuals served by the department; and the institution to which the department belongs. Departmental planning includes the active selection of some dimensions of a discipline or profession for focus in scholarship, service, and teaching. Departmental planning takes account of the nature of students enrolling in the courses taught by the department. Are they primarily majors? graduate students? students seeking a general education? Departmental planning is informed by planning in the college and campus, and must be consistent with the mission of the campus. Departments are urged to consult appropriate planning documents. The departmental mission statement gives a brief synopsis of how a department defines its major purposes within these domains.
The mission statement is an essential starting point for a coherent plan. It is the umbrella under which more particular goals and objectives can be fit. In current terminology, a goal refers to a desired end stated in a relatively general and abstract way whereas an objective refers to something more specific and concrete. A goal can spawn any number of objectives. This distinction is paralleled in assessment. It is possible to assess the degree to which an objective is being met by taking specific measurements, but assessment of whether a goal is being met requires a relatively abstract analysis of information gained from the measurements and an interpretation of the relationship between concrete objectives and the goal they are meant to represent.
A departmental plan will identify a set of goals deemed worthy of significant attention. These goals, which can range over the full spectrum of faculty work, should be appropriate for the department. They should derive from the departmental mission statement and fit with the goals of higher units (e.g., college, campus). Selected goals should be realistic. It should be possible to articulate one or more specific objectives that follow from each goal; to imagine the steps needed to meet that objective; and to foresee some ways of gaining feedback to determine whether either these steps or the objective itself should be maintained or changed as experience is gained.
A goal is a desired outcome. Therefore, an outcome perspective should be used to frame goals. In the area of teaching and learning, for example, goals should be stated in terms of what students in a particular category (e.g, majors, general education nonmajors) should know and know how to do after completing a course of study. Focusing on student learning outcomes in such an explicit way requires a change in language from that usually used to describe course and curriculum content. This change is not mere fashion, however; it prepares the way for effective assessment.
Departmental planning takes place in a dynamic context. Disciplines evolve, faculty often change interests as their careers develop, some faculty leave and others arrive, student populations change, the regional and global context changes, the institution changes, etc. To be useful, a departmental plan must be examined regularly and modified as needed to meet these changes.
A successful departmental plan requires the support of the full department. All departmental members will engage in the initial preparation of the plan and in its annual reconsideration.
Departmental planning takes place within a hierarchically ordered system. At a lower level of the system, individual faculty engage in planning. Departmental planning is more than a collection of these individual plans, but individuals certainly contribute to the whole. There is also planning at each higher level in the institutional hierarchy that is led by administrators and faculty governance. Departments will find it useful to establish connections with plans at these higher systemic levels. Finally, departments will often be helped by looking laterally in the system to other departments, perhaps finding ideas on how to proceed from those with successful track records in one or more selected goals.
Although assessment is often seen as the major tool in program review, effective planning may well be more difficult and more important. Assessment cannot be done without clearly articulated goals that can be further specified as objectives. Once this is done, assessment becomes a well-defined problem to be solved. A strategy for assessing progress should be worked out concurrently with establishing goals and objectives.
Assessment
Assessment is the systematic use of feedback to improve work. It is a way to measure progress toward defined goals. An effective assessment process will determine whether changes are needed and will also suggest the nature of these changes. Those who do assessment find it useful to distinguish formative from summative assessment. Formative assessment is done along the way; it can be used to change how goals and objectives are conceived as well as to modify courses of action designed to meet them. It is thinking analytically about one's work using whatever reliable and valid evidence is available. Summative assessment is done when something is finished, or done for a sufficiently long time to warrant stopping and taking stock. It is used to determine degree of success, and may point to significant changes in direction for a program. Assessment applied to specific objectives is often called evaluation, and the devices used to do the measuring are called evaluation instruments.
Faculty are usually more familiar with summative assessment (student evaluations done at the end of a course, tenure review, manuscript review, etc.). Because these assessments are relatively formal and usually include quantitative evaluation against set benchmarks, it is sometimes assumed that all assessments must have these characteristics. Although summative assessment has an important place in program review, formative assessment to guide the planning and execution of a program is exceedingly important. It can, for example, be used to change direction in teaching a course, to change prerequisite structure in a major, or to change recruitment strategies for graduate students.
An assessment process requires careful analysis and must obey rules of evidence, but useful assessments can be done without special instruments or technical expertise in assessment methodology. The most difficult part of assessment is deciding what to measure, and this is best done during planning because of the close relation to goals. Some measurements that are generally useful across departments are specified in this AQUAD process (see template). There will be institutional support for developing additional assessment strategies for departmental goals (for example, workshops, web pages, reference shelves, and consultation with campus experts) that will be organized by the Provost and Deans.
Because objectives and measurements of their attainment are both relatively concrete, there is a gap between any measurement and the more abstract goal that a department is interested in assessing. This gap must be filled with interpretive analysis. To increase the likelihood that the interpretation is accurate, a good assessment plan should seek multiple measurements for each goal. Converging evidence from disparate sources is more convincing than any single piece of evidence, regardless of its technical merit.
Systematic collection of data is another important component of an assessment plan. Background information on starting conditions is critical for measuring change that results from some action or program. A department setting a goal of increased faculty scholarship would be well served to know the level of publication during several years before instituting a plan designed to increase the scholarship. Similarly, a department setting a goal of having students learn how to work in teams to solve real-world problems would do well to determine the level of skill in this area before instituting courses or activities designed to increase it. The importance of background data against which change can be measured is another argument in favor of establishing an explicit plan in the early stages of a review cycle. This will determine the data that are collected.
Assessment as conceived in this report is primarily an everyday tool that has many uses within a department. It can be used to guide all areas of departmental work so that it becomes both more effective and more efficient. There is in addition an important role for a more formal summative review from time to time. In the process described in this document, a formal review will occur in the seventh year of a review cycle or in the year specified by an accreditation review. It is anticipated that this review will improve departmental planning and assessment by providing an external perspective. It will also provide administrators at the college and campus levels with the information they need for their planning.
Assessment can and should be applied to all institutional activities, including the process for assessing the quality of academic programs that is described in this document. Therefore, this assessment process will itself be assessed and modified as needed to meet its stated goals.
Final Review and Evaluation
The review that takes place in the final year of a cycle is meant to provide an evaluation of a department in three steps: a self-evaluation undertaken by the faculty in the department or program; an evaluation by a review team of experts from outside the department and campus; and an evaluation by administrators responsible for oversight of the program. This review has required elements that will be applied to all programs, and it will result in written reports by the department, the review team, and the administrators. These reports will remain on campus, with the exception of a summary that is filed with the President's office.
Because the review that occurs in the final year of the cycle must follow a set process and include set elements, the remainder of this report is devoted primarily to spelling out this review in detail. However, the information contained in these descriptions and the attached self-study template should be used to guide planning and assessment throughout the seven-year cycle.
Specific recommendations
A. Review cycles and program definition
1. Unless there is an external accreditation cycle of a different length, each program will be reviewed on a seven year cycle.2. External accreditation cycles will be used for all accredited programs.
b. Accredited programs with reviews that take more than one year to complete will, in consultation with their Dean(s) and the Provost, name which year of the multi-year process will serve as the AQUAD year. This year must produce a report from external reviewers by May 1 to permit time for campus reaction and inclusion in a June 30 report to the President's office.
3. The scheduled timing for completing a review cycle may be moved forward or backward under sufficiently compelling circumstances. A petition for such a change, ordinarily limited to one year, may be initiated at any administrative level but is subject to final approval by the Provost.
4. Academic departments are the standard unit (program) under review. Unless otherwise specified, departmental reviews are understood to include all academic degree programs administered within that department.
5. Subject to the approval of the Provost, the Deans in consultation with relevant departments or other units, determine the scope of a program for review purposes. A request for a change in the scope of a program can be initiated at any administrative level but is subject to final approval by the Provost.
6. All interdisciplinary and intercollegiate programs will be reviewed, within departmental reviews or by themselves.
7. Whenever a program review involves more than one dean or more than one department, the relevant administrators should consult together on all aspects of the review process.
8. Table 1 (attached) presents a recommended schedule for reviewing all academic programs on this campus within the next seven years. The cycle will then be repeated, beginning in 2006. Accreditation reviews usually take two years to complete. The AQUAD date for these reviews was selected based on a projection of final report from this process.
B. Review Process
1. The review process is designed to be consistent with the AQUAD (Academic Quality Assessment and Development) guidelines described in Trustee Document T98-033.
2. All departments (programs) undergoing accreditation review should use that review process as the AQUAD review. To assure that all AQUAD criteria are met, the departmental chair or program director in these cases has the responsibility of preparing a brief overview to identify points of correspondence between an AQUAD review and the accreditation review. Should the Dean(s) and Provost determine that the accreditation review fails to satisfy all required elements, a supplementary review to address the missing elements must be conducted.
3. Each department will prepare a brief planning statement that sets forth its mission, goals, and objectives; that indicates how these plans fit within the mission and goals of the campus; and thatdescribes an assessment strategy. The planning statement will be submitted to the Dean(s) and will serve as a guide to departments and as a prospective context for the review. There will be a mid cycle review of the plan by the program and dean(s) at least 3 years before a scheduled AQUAD review. Dean(s) may recommend changes or specify additional elements for inclusion in the planning document. When more than one dean is responsible for a program, they should consult on responses to departmental plans at both stages. Programs that prepare a planning document as part of the accreditation process will be able to substitute that document, supplemented when necessary to meet the requirements described in this paragraph. Further details are provided on the attached template for planning documents.
4. Each department is responsible for assembling annual data relevant to its review (see attached template for self-study and assessment). When data are available from the Office of Institutional Research and Policy Studies (OIRP), they will be recognized as the official data for AQUAD purposes. OIRP data are now available for several measures listed on the template and required in this review process. OIRP may be able to provide additional measures on a routine basis in the future. Data not routinely collected by OIRP may be collected by departments or other administrative units. OIRP may be able to provide some consulting or other assistance with these measures. Deans or others responsible for the review of particular programs (e.g., accredited programs; general education; graduate studies) may specify additional data needed for those reviews.
5. Near the end of each review cycle (year 6 or 7 for most programs), each program or department will prepare a self-study document that describes the program; (b) provides an assessment in the light of specified criteria (see next section); and outlines steps that can be taken to effect improvement. Details for this document are provided in the attached template and instructions. Accredited programs may substitute the template provided by accrediting agencies, with appropriate addenda as needed to include all AQUAD elements.
6. Program faculty will participate in all aspects of the assessment: the planning, the self-assessment, the review process, and responses to reviews and comments. Program faculty will have access to all documents.
7. Each program will be reviewed by a team that includes at least two reviewers external to campus (with at least one external to the University system) and at least two reviewers selected from UMass Boston faculty outside the program under review. Reviewers will be selected by the Dean(s) in consultation with the program Chair, Director, and faculty. When graduate programs are included as part of an integrated departmental review , the Dean of Graduate Studies and the faculty Dean will consult with one another and with the Chair and graduate program Director. If there is no consensus, each Dean will select at least one of the outside (and/or inside) reviewers. Programs to be reviewed are expected to submit to the Dean(s) a list of possible reviewers, with a rationale for each. The Dean(s) will provide a written rationale for any reviewers who are not on the Program's list, and the Program will be given an opportunity to comment. Written rationales to support the selection of reviewers and any associated comments will be part of the review file.
8. The review team will receive copies of the self-study and any appended documents, the criteria for program assessment, and any specific questions prepared by Deans. The Dean of Graduate Studies and the faculty/college Dean may prepare different questions. The team will be provided with written documents in advance of its visit, and it will meet with program faculty, students, and administrators. The review team will be chaired by one of the outside reviewers unless the Provost grants an exception. The review team chair will prepare a written report.
9. The Program will have an opportunity to respond to the committee report. The Dean(s) will then review the committee report, program responses, and all associated documents and prepare written comments on the review and recommendations for improvement which will be circulated to the program faculty for their response.
10. The Dean(s) will forward all documents to the Provost who will meet with the Program Chair (Director) and the Dean(s). If appropriate, the Dean(s) will draft an action plan which will be revisited by the Dean(s) and the Program Chair and/or Graduate Program Director one year after the review, and as long as necessary to meet the goals of the plan.
11. On or before June 30, the Provost will make an annual report to the President containing executive summaries of all academic program reviews completed during the year.
C. Criteria
The review criteria listed here are consistent with the AQUAD (Academic Quality Assessment and Development) guidelines described in Trustee Document T98-033. This document and the attached self-study template can be consulted for more details.
1. Planning. The program should demonstrate effective planning within the context provided by the mission, goals, objectives, and resources of the academic units housing the program and the campus as a whole.
2. Curriculum quality. Assessment of the curriculum should include attention to four dimensions of quality: coherence of the educational program; rigor of the educational experience; relevance of the learning to further education, quality of life, employment, or other use by students; and currency of the educational experience in terms of the broader disciplinary or professional context represented by the program.
3. Faculty quality and productivity. Assessment of faculty quality and productivity should include information about faculty expertise and productivity in teaching; research, professional, or creative activity; and public service or outreach.
4. Student learning. Assessment of student learning outcomes should address how the program facilitates student success in learning. It should address articulated goals for student learning outcomes and procedures for measuring the outcomes. It should also discuss how evidence of student learning is used in reviewing its curriculum and its faculty.
5. Use of resources. Assessment of resource use should address the efficiency and effectiveness of resource allocation and use. Information about the allocation of both personnel and material resources (space, equipment, and operating budget) to meet curricular and other objectives, including any intra- and inter campus collaboration or other innovations, should be included in the assessment.
D. Revision of the Process
The AQUAD review process should be seen as an on-going, evolving process which should be reviewed and revised as necessary. Feedback will be gathered from programs and administrators as reviews are completed. In the second year of the second cycle for the campus, a thorough review of the AQUAD policy, procedures, and implementation guidelines should occur, and any appropriate changes made.
Committee members
Kristine Alster, College of Nursing Celia Moore, CAS Faculty of Sciences (Chair)
Jennifer Brown, OIRP (Spring) Ron Polito, Graduate Studies
Robin Burr, OIRP (Fall) Russell Schutt, Sociology Department
Virginia Harvey, Graduate College of Education Gary Siperstein, Human Services Program, CPCS
Catherine Lynde, Academic Affairs Gordon Wallace, Environmental, Coastal and
Peter McClure, Management and Marketing, CM Ocean Sciences Department
Attachments
1. Schedule 4. Template and instructions for self-study document
2. Trustee Document T98-033 5. Sources on planning and assessment
3. Template and instructions for planning document
Revised AQUAD Guidelines System
Board of Trustees Document T98-033
Academic Quality Assessment and Development Guidelines
Academic Quality Assessment and Development is a component of the University Performance Measurement System. The primary purpose of this component is to assess and improve the core academic functions of teaching and learning, research/professional/creative activity, and public service/academic outreach through an ongoing system of quality control/program assessment at the unit level (i.e., department or program).
Each campus will establish, in consultation with the President's office and in accordance with these guidelines, procedures for implementing the Academic Quality Assessment and Development. These procedures will include a list of units (programs/departments), hereinafter called "Programs", to be reviewed; the cycle of reviews; and a list of core data to be used in Program reviews on that campus. The "Program Chair" shall be defined as the person with administrative oversight of the Program. Campus procedures shall be approved by the President.
All Programs will address the same core evaluation criteria, although these criteria will have varying degrees of relevance and applicability across the campus. Programs undergoing reviews for other purposes (e.g., accreditation) may submit these reviews in lieu of the Academic Quality Assessment and Development review, provided the review addresses the core criteria. If not, the Program shall prepare and submit a supplemental document addressing the core criteria.
Process
1.Each Program shall be reviewed on a regular cycle. Ordinarily, the length of time between reviews may be no more than five to seven years, but campus procedures may establish the circumstances under which exceptions to this time frame may be granted.
2. Each Program review shall be conducted with the participation of Program faculty members.
3. Each Program review shall consist, minimally, of the following written documents: a Program self-assessment, the external reviewers' report, a Program response to the reviewers' report, and an action plan based on the review. These written documents will be submitted to the Dean. Faculty within the Program will receive copies of all documents.
4. Each Program shall be reviewed by a team of no fewer than two external reviewers from outside the campus (one reviewer may be from another campus within the University system). The Dean, in consultation with the Program chair and faculty members, will choose the external reviewers. The campus procedures shall stipulate the questions to be addressed by the external reviewers.
5. The Dean will review and comment on the written reports. The Dean's comments will be distributed to all Program faculty, who may prepare a written response to his/her comments.
6. The Dean will forward the Program self-assessment, external reviewers' report, Program response to the reviewers' report, and his/her comments with faculty responses, if any, to the Provost. Prior to accepting the review, the Provost will meet with the Program chair and the Dean to discuss the review and action plan.
7. The Provost of each campus shall forward to the President's office annually an executive summary of the Program reviews conducted. The summaries shall be submitted no later than June 30th. The documents comprising a review shall not, as a general matter, be circulated beyond the campus.
Core criteria and related questions
1. Programs shall ensure that their goals and objectives are linked to the campus mission and strategic priorities.
The Program should evaluate its purpose and planning in light of the campus mission and strategic priorities. The review should answer the following questions:
- What is the Program's mission and is it clearly aligned with the campus mission and direction?
- How does the Program's mission relate to curriculum; enrollments; faculty teaching, research/professional/creative activity, and outreach? Is it aligned with the campus strategic priorities?
2. Programs shall ensure that curriculum is relevant, rigorous, current and coherent.
The need to provide a high quality education for students should be the primary consideration when evaluating the relevancy, currency, and coherence of curricula. Evaluation of the curriculum should reflect an awareness of changing knowledge, trends in the discipline, and the professional context for curriculum. The review should answer the following questions:
- How does the Program determine curricular content? How does the curriculum relate to current existing standards, if any, of the discipline?
- What internal or external measures of review are employed to ensure that the curriculum is relevant and up-to-date?
- Are the curricular offerings structured in a logical, sequential and coherent manner? Is there an appropriate balance between breadth and depth?
- If consistent with the Program mission, does the curriculum adequately prepare students for further study or employment?
- In what way does the Program contribute to the education of students in terms of general knowledge, critical thinking capacity and other essential cognitive skills?
3. Programs shall ensure faculty quality and productivity.
Programs shall ensure that faculty possess the expertise to assure effective curriculum development, instructional design and delivery, and evaluation of outcomes. Faculty should exhibit awareness of trends in the discipline and the professional field as appropriate. Collectively, faculty should be involved in teaching, research/ professional/creative activity, and public service/academic outreach as appropriate to the mission and regional context of the campus. The review should answer the following questions:
- Do faculty possess the appropriate background, experience and credentials?
- Are faculty current in relation to the knowledge base and content of the discipline and curricular offerings?
- Are the Program expectations for faculty involvement in teaching, research/ professional/creative activity, and public service/academic outreach activities appropriate; and how are these expectations met? Are these expectations consistent with program policies regarding teaching assignments, merit allocations, and other aspects of faculty roles and rewards?
- In what ways does the Program foster professional development and growth of faculty?
- In what ways does the Program faculty lend its professional expertise-- as expressed through teaching and research, scholarly and creative activity--to off-campus constituencies?
4. Programs shall ensure teaching/learning environments that facilitate student success.
Programs shall provide learning environments that promote student success. Students are expected to learn both content and skills appropriate to the discipline. The program should indicate clear expectations for student learning outcomes. The teaching/learning environment should be accessible to all students, should include a variety of instructional methodologies, and should provide timely feedback to students. The review should answer the following questions:
- To what extent does the Program have articulated learning outcomes (content and skills) for students? By what means are these outcomes measured? Are they achieved by most students?
- How is assessment of student learning outcomes used in reviewing or evaluating Program curriculum and faculty?
- In what ways does the Program evaluate student success following graduation and the Programs's contribution to that success?
- What is the role of the core faculty in teaching lower division, upper division and graduate courses? What is the rationale for these assignments?
5. Programs shall ensure that resources are used wisely.
Programs shall ensure that the resources available are used to meet Program goals and objectives, and as appropriate, engage in use of innovation to enhance resources; should engage in both intra- and inter-campus collaboration; and should demonstrate a commitment to effective and efficient use of resources. The review should answer the following questions:
- What process does the Program use to allocate resources?
- In what ways does the Program maximize the use of its human resources?
- In what ways does the Program maximize the use of material resources such as space, equipment, operating funds, etc.?
Planning Template
Template for planning at the departmental or program level
Departments should prepare a written plan that includes a mission statement, a set of goals and objectives, a rationale for the plan, and a strategy for regular programmatic assessment. Plans are best conceived as "living" documents that are up for review and revision on an annual basis. The plan should include the following elements:
I. Mission statement
(The mission of an academic department (program) can be defined with respect to its discipline or field of study; with respect to the students and other constituents it serves; and with respect to the institution to which it belongs. With this in mind, what is the central mission of the department? How does it fit into the broader mission of any applicable higher units, including the campus as a whole? This statement should be brief, focusing on central purposes stated at the broad, conceptual level.)
Further notes: Such a statement would go beyond renaming a discipline to identify general approaches or specializations within a discipline/field or other features that define the program; would identify areas selected for clusters of strength in faculty research, outreach, and other professional activities; would identify nature of the students served and their educational goals--primarily majors? graduate? nonmajors in general education? service courses to other departments in an area? professional training? etc.
II. Goals and objectives
(What are the long-range goals that guide planning in the department (program)? What specific objectives does the department seek to meet during the next review period (7 years)? The goals and objectives selected should be consistent with its mission statement and with the planning of higher units encompassing the program, including the campus as a whole. The spectrum of goals and objectives that can be listed include those for student learning outcomes, outreach, faculty development, research, creative activities, professional service, grant development, distribution of human and other resources, etc.)
III. Brief prose rationale in support of selected goals
(Goals and objectives should be realistic and should have a supporting rationale. State why the goals are most important and most appropriate for department at this time.)
IV. Strategy for assessing progress toward goals
(An assessment plan should be developed as part of departmental planning so that progress can be monitored and changes instituted as experience is gained. What kinds of information will be most useful for determining whether goals are being met? What steps will be taken to gather the information?)
Examples of goals and objectives a department might select: (These are meant only to provoke thinking and to illustrate the kind of detail that is sought. )
Goal: To provide majors with a thorough grounding in contemporary principles and methods of the field.
Objectives
1. to change the structure of the major so that students will take courses in X (part of discipline previously underrepresented in enrollment.
2. to develop new courses that reflect the growth of X specialty in discipline.
Goal: To contribute more effectively to the general education of students.
Objectives
1. to increase the number of faculty who participate in faculty development programs for general education pedagogy.
2. to develop X new courses for nonmajors.
Goal: To add strength to the department in specialty X.
Objectives
1. to search for expertise in specialty X with the next vacated line to be returned to the department.
2. to encourage retraining of xx members of the faculty in specialty X.
3. to develop interdisciplinary collaborations in specialty X.
Goal: To develop an outreach program in X to build on professional strengths in the department.
Objectives
1. to strengthen connections with faculty in department X that has a related outreach effort.
2. to establish working relationships with community groups in X.
Goal: To increase external grant support in the department.
Objectives
1. to increase the number of faculty seeking external grant support to X%.
2. to institute grantsmanship workshops for unfunded faculty.
Goal: To increase the ability of undergraduate majors to reason analytically using significant concepts in the discipline.
Objectives
1. to increase the amount of analytic writing required of students in courses for majors.
2. to develop a phased sequence of writing assignments for use in introductory, intermediate, and advanced majors courses.
3. to develop a capstone senior thesis.
AQUAD Self-Study Template
I. Description
A. Description of program
(Subheadings may be introduced by departments for separate descriptions of major components--e.g., graduate and undergraduate programs--if this will aid description. Note that items 2 and 3 should be taken from a planning document completed well in advance of the review. See template for planning document.)
1. Program identity
(Name of department or program and all of its undergraduate and graduate degrees and certificates, including significant contributions to joint and interdisciplinary programs.)
2. Mission
(The mission of a program can be defined with respect to the discipline or field of study it represents; with respect to the students and other constituents it serves; and with respect to the institution to which it belongs. With this in mind, what is the central mission of the program? Explain how this fits into the broader mission of any applicable higher units, including the campus as a whole? See template for planning document.)
3. Goals and objectives
(What goals and specific objectives with respect to professional service, outreach, research, scholarship, creative activity, or student learning outcomes have guided the program during the period of review? These should be consistent with the program mission statement and the campus mission and goals. See template for planning document. An overview may be presented here with the details presented in Section II.)
4. Curriculum
(Describe the structure, requirements, special features, and availability of the curriculum for each degree program that is offered.)
a. Structure (Describe the major elements comprised by the curriculum, including options, sequences, etc.)
b. Requirements (List required courses and other learning experiences offered within and outside the department.)
c. Special features (List such features as internships, field work, honors programs, etc.)
d. Availability (Can extended day students complete the program? List any distance learning or off site availability.)
5. Advising system
(Describe the academic advising provided to students, including assignment of advisors, tracking of students, group advising sessions, etc.)
6. Co-curricular activities
(Describe any co-curricular activities housed in the program, such as student clubs, honors societies, etc.)
7. Scheduling
(What are major guiding principles for scheduling courses? Describe a typical schedule.)
8. Appendices
(Append a sample schedule, catalog copy, fact sheet, and/or handbook that provides additional details on program structure, availability, and content; advising; co-curricular activities; etc.)
B. Description of human and material resources
1. Full time faculty
a. List of full time faculty with degrees and specializations and information on how the faculty are distributed among the fields of study represented in the program.
b. In % FTE, how is the workload of full time faculty distributed to the major and other programs, graduate and undergraduate, activities other than teaching?
c. A set of current CVs listing publications, professional service, grant support and other creative and scholarly activities should be assembled and made available to reviewers.
Data on numbers of faculty over the past five years and data on student FTE by course section level and type are available through OIRP.
2. Part time faculty and students with instructional roles
a. List of part time faculty with degrees and specializations and information on how the faculty are distributed among the fields of study represented in the program.
b. In % FTE, how is the workload of part time faculty distributed to the major and other programs, graduate and undergraduate, general education and/or 'service' courses?
c. A set of current CVs listing publications, professional service, grant support and other creative and scholarly activities should be assembled and made available to reviewers.
d. What is the nature and frequency of teaching by graduate students TAs?
e. What student tutoring resources are available to the program?
Data on numbers of faculty over the past five years and data on student FTE by course section level and type are available through OIRP.
3. Professional and classified staff
List and describe the functions of professional and classified staff assigned to the department.
Grant supported personnel should be listed separately.
4. Material resources
Describe the physical resources available for use by the program. Include separate descriptions of central resources and resources assigned to departments:
University spaces, equipment, labs., etc.;
Departmental spaces, equipment, labs. , etc
Operating budget. Describe the operating budget for a typical academic year.
5. Library use
Provide a general description of library holdings and other resources that support the program. Some data will be available from the library.
6. Student support (Graduate only)
Describe stipends available for support of graduate student assistants. State and grant support of student assistants should be listed separately.
C. Student profile
1. Enrollment (annual data for at least 5 years)
a. Number of student majors by department and by academic degree program.
b. Student FTEs by level of courses (lower, upper and graduate division).
c. Class sizes (average, range and median).
d. Number of degrees and certificates awarded.
Data are available from the OIRP.
2. Survey results (as available)
a. Satisfaction with program.
b. Post graduation employment or enrollment in graduate or professional school. The OIRP can assist with the planning of surveys, focus groups etc.
3. Descriptive data on students in the program
a. Age, gender, ethnic mix of students enrolled as majors by department and academic degree program. Data are available from the OIRP.
b. Other admissions data and performance measures, subject to availability. Some data may be available from the OIRP.
II. Assessment
This section should provide a reflective analysis of the program in light of the missions, goals, and objectives of the University and the program. Strengths and weaknesses, accomplishments and shortcomings should be analyzed with an eye toward program improvement. The following outline may be used to present the assessment:
A. Teaching and learning
1. Curriculum delivery to students (scheduling, advising, enrollment, student recruitment)
2. Curriculum quality (coherence, rigor, relevance, currency)
3. Student learning outcomes (for overall program; within each course)
B. Professional service and outreach
C. Scholarly representation of and contributions to the discipline or profession
D. Resource use (efficient, effective)
The following criteria are to be used in the assessment:
1. Planning. The program should demonstrate effective planning within the context provided by the mission, goals, objectives, and resources of the academic units housing the program and the campus as a whole.
2. Curriculum quality. Assessment of the curriculum should include attention to four dimensions of quality: coherence of the educational program; rigor of the educational experience; relevance of the learning to further education, quality of life, employment, or other use by students; and currency of the educational experience in terms of the broader disciplinary or professional context represented by the program.
3. Faculty quality and productivity. Assessment of faculty quality and productivity should include information about faculty expertise and productivity in teaching; research, professional, or creative activity; and public service or outreach.
4. Student learning. Assessment of student learning outcomes should address how the program facilitates student success in learning. It should address articulated goals for student learning outcomes and procedures for measuring the outcomes. It should also discuss how evidence of student learning is used in reviewing its curriculum and its faculty.
5. Use of resources. Assessment of resource use should address the efficiency and effectiveness of resource allocation and use. Information about the allocation of both personnel and material resources (space, equipment, and operating budget) to meet curricular and other objectives, including any intra- and inter campus collaboration or other innovations, should be included in the assessment.
III. Plans for the future
1. Priorities for planning in next review period.
2. Strategies for addressing identified weaknesses.
3. Strategies for building on strengths to meet identified needs.