Workshops on Verification: a) SSR and Display Materials, b) Evidence via Interviews – PU

  • Date:March 16th, 2023  
    09:00 am
    04:30 pm

Instructor: Dr. Scott Danielson - Arizona State University, Mr. Thai Tran 
Attendees: PU Quality Assurance Staff and Interested Program Accreditation Team Members

Background: Most accrediting/evaluation agencies and organizations use a peer-review process to assure programs meet standards of compliance with evaluation criteria. Usually, this process utilizes a two-step process in which assessors first evaluate a program’s self-study report, SSR, or self-assessment report, SAR—sometimes called a desktop assessment. The second step usually involves a site visit, although in current times a site visit may be done via distance technology, e.g., Zoom. The first step, the initial review of the SSR forms a preliminary assessment of the program compliance with criteria. Thus, the SSR/SAR is a very critical document as it sets the stage for the rest of the evaluation. The SSR serves as the basis for an initial quantitative and qualitative assessment of program strengths and limitations. As the second step, the site visit allows the assessment team to evaluate factors that cannot be adequately described in written form and to confirm, via additional evidence, the statements made in the SSR/SAR. It is upon this more complete evaluation that the team makes its recommendations about program compliance with the criteria.

Expected Outcomes:  The workshops seek to provide information and “best practices” for PU leaders, quality assurance staff and program accreditation teams related to helping the evaluators verify SSR contents/statements during the onsite/Zoom evaluation phase. The first workshop focuses on techniques used by evaluators to verify key information and documents mentioned in the SSR/SAR using documents provided during the on-site visit (either hardcopy or electronic materials if a virtual visit). Selected evaluation criteria/standards will be used to provide context.  

The second workshop focuses on interviews—individual and group—with students, faculty members, administrators and staff. These interviews may be one-on-one for high-level administrators and program leaders. Lecturers may have individual interviews or as groups of lecturers. Students and laboratory staff may also be group interviews. Different techniques and questions are often used with these groups—yet the questions are based on a common objective—providing evidence related to compliance with criteria.  

Part 1: Morning – Focus on SSR and Display Materials
8:45 – 9:00 am Participant Registration & Sign-in
9:00 – 9:05 am Welcome to Workshop
9:05 – 9:30 am Verification Process—Its Goals and Basic Approach
The SSR and a related on-site verification of aspects of the SSR are fundamental to peer review. A quick review of the process goals and evaluator’s basic approach to verification will be explored.
9:30 – 10:00 am Verification Process— Focus on Facilities (related to criteria AUN-QA Criterion 9.5 and ABET Criterion 7)
Participants will be led on a guided discovery of why and how verification of stakeholder input to the program is accomplished.
10:00 – 10:15 am Verification Process—Focus on Student Assessment (related to criteria AUN-QA Criterion 5.1 and ABET Criterion 4)
Participants will explore how selection and verification of different aspects of a program’s processes for assessment of student learning and attainment of program-level student outcomes is accomplished.
10:15 – 10:30 am Break
10:30 – 11:15 am Continuation of Verification Process—Focus on Student Assessment
11:15 am – 12:00 pm Verification Process— Focus on Stakeholder Input (related to criteria AUN-QA Criterion 2.1 and ABET Criterion 2)
Participants will explore how selection and verification of different aspects of the program’s facilities and their use by students is accomplished.
Part 2: Afternoon – Focus on Evidence via Interviews
1:30 – 2:00 pm Verification Process—Use of Interviews
Meetings and related discussions with individuals and different groups during the site visit are very important verification tools. Basic evaluator approaches to configuration of interviews and selection of those interviewed are reviewed.
2:00 – 2:40 pm Individual Interviews
Once individuals are selected for one-on-one interviews, conduct of the interview by the assessor depends on two primary factors. These factors are interpersonal approach of the evaluator and questions asked. While the latter may vary based on interviewee’s position, the former should not. Suggestions will be made and practice sessions held.
2:40 – 2:55 am Break
2:55 – 4:15 am Group Interviews
Usually, students and industry representatives are interviewed in a group, at least in an ABET evaluation (all interviews done by AUN-QA evaluators are group interviews). If the lecturers supporting a program are large in number, smaller groups of faculties may be interviewed in groups instead of as individuals. While the specific focus/questions asked may vary depending on the group, the approach can be consistent. Typical questions are reviewed for different types of groups and practice sessions held.
4:15 – 4:30 am Workshop Wrap-up