Leveraging Tech for Program Reviews

Alyncia Bowen, Dean; Kelly Evans-Wilson, Executive Director of Accreditation and Authorization; and Yuerong Sweetland, Director of Assessment at Franklin University

Alyncia Bowen, Dean; Kelly Evans-Wilson, Executive Director of Accreditation and Authorization; and Yuerong Sweetland, Director of Assessment at Franklin University

Higher education institutions (HEIs) conduct program reviews to assess the quality and viability of academic programs. Like many peer institutions, Franklin University annually conducts program reviews. However, our existing system was cumbersome and labor-intensive, leading to confusion and frustration among faculty and administrators. Key challenges included:

• Unreliable accessibility of data, scattered across disparate sources, making essential information hard to locate.

• Inconsistent analyses and interpretations due to diverse data sources and variations in data collection methods lead to data variability.

• The absence of a centralized system hinders access to historical documents for understanding contexts and identifying trends.

• Lack of a sustainable feedback loop, impeding a consistent provision of meaningful feedback to academic programs for growth.

While multiple commercially available technology solutions have been designed to address the program review process at HEIs, Franklin University made the decision to develop an in-house solution called the Health of Program (HoP) hub. This web-based platform consolidates all related information into a convenient one-stop platform for faculty and administrators. This tool streamlines the program review process and provides structured feedback between faculty, program chairs, and College Deans. The hub is also available to the senior executive team.

The HoP hub hosts data in five categories: student experiences, curriculum, faculty and instruction, market demand, and competition. Data are seamlessly integrated into the hub from multiple sources, including the learning management system (MS), student information system (SIS), curriculum management system (CMS), and university surveys that provide insights into experiences and perspectives on learning and teaching, respectively, from students and instructors.

Five years of data are integrated into the HoP hub to help detect trends and make informed decisions, including benchmarks of university and degree level averages in student experience ratings, GPAs, retention rates, and graduation rates. This extensive and complex data is presented in visually appealing and understandable formats, further aiding analyses and interpretations.

"Leadership support and commitment are crucial for the success of any new initiatives."

This internally created technological tool also enhances Franklin University’s effectiveness and efficiency by creating the ideal space for faculty, program chairs, and academic leaders to collaboratively review data analysis findings and provide feedback.

Lessons Learned

With two years of program reviews completed, HoP continues to receive positive feedback from faculty, administrators, and executive leadership. Franklin University’s experience developing a technology tool to facilitate program reviews and promote a data-informed decision-making culture comes with six valuable lessons.

1. Establish Clear Goals and Objectives

Throughout the entire process of developing, implementing and continually enhancing the HoP hub, we ensure clear goals and objectives, which are established based on discussions and collaborations with our project partners on campus: the information technology and institutional effectiveness offices, along with all faculty and leadership. We also ensure alignment with the university's mission and strategic priorities.

2. Ensure Data Quality and Accuracy

Data quality and accuracy consistently rank as our top priority for the HoP hub project. We implemented data validation processes, which were facilitated by clearly defined roles and responsibilities for data-related tasks to avoid confusion and promote data accountability. We also emphasize the importance of regularly auditing data sources to make sure the hub data is up-to-date.

3. Promote Substantive Data Analysis and Interpretation

It is crucial that the HoP hub users understand how to interpret and use data accurately. We conduct multiple training programs and facilitate discussion forums to help promote substantive data analyses and interpretations.

4. Encourage a Culture of Collaboration

The development and continuous improvement of the hub are collaborative efforts involving academic affairs, including faculty and administrators, along with information technology and institutional effectiveness offices. Breaking down silos and facilitating collaboration among departments and stakeholders is critical for the success of the HoP hub project.

5. Cultivate a Growth Mindset through Iterative Implementation and Improvement

We started with a pilot of our HoP hub, gathered feedback, and made necessary adjustments before scaling up. At the end of the initial program review, we conducted a study that revealed a high level of user satisfaction regarding the hub’s ease of use and support provided. We continue to engage with stakeholders to enhance the hub based on their suggestions, fostering a growth mindset and a culture of continuous improvement and adaptation.

6. Leadership Support and Commitment

Leadership support and commitment are crucial for the success of any new initiatives. Throughout the process of the hub development and implementation, we ensured that the project stayed aligned with the university's mission and strategic priorities. Regular communication with and continuous feedback from the executive leadership team regarding specific goals and objectives helps ensure ongoing support.

Fostering a data-informed program review process at an HEI requires faculty collaboration, quality data, substantive data analysis, and executive leadership support. Through the creation of a home-grown technology tool, the Health of Program, Franklin University was able to provide a platform that focused on ensuring academic quality and creating a culture of continuous improvement.