Social Impact and Artificial Intelligence: Understanding Indirect Measures

Kent Seaver, Director of Academic Operations, the University of Texas, Dallas

I am an experienced professional with a strong background in the higher education industry. I possess skills in educational assessment, educational technology, instructional design, compliance, presentation, accreditation, and career assessment. With extensive experience collaborating with faculty, I focus on learning outcomes, student success, and curriculum development.

In an exclusive interview with Education Technology Insights he shared invaluable insights on integrating AI into both curriculum and assessment, preparing students for real-world challenges with a data-driven, future-ready approach.

Shaping the Future of Business Education with AI at UT Dallas

Artificial Intelligence (AI) is one of the most talked about phrases in higher education. But from the stand point of course and program assessment, what can be achieved by implementing AI into assessment? What can be gained (or lost?) by implementing AI into assessment techniques? As AI continues to gain acceptance in colleges across the world, how that technology can be used to improve assessments has been written about, discussed, and debated. But one area that has not been heavily discussed is in what manner can AI best assist program assessment.

Institutional Profile

As part of the University of Texas at Dallas, The Naveen Jindal School of Management offers over fifty degree and certificate programs across three levels: undergraduate, graduate (including PhD), and Executive Education. The Jindal School has an enrollment of over 10,000 students, and has numerous nationally ranked academic programs. The Jindal School is ranked #1 for online MBA programs in the U.S. by Poets&Quants. In addition, the graduate program in supply chain management has attained the No. 4 ranking in the Gartner Supply Chain University Top 25 among North American universities, as well as the Professional MBA program ranking No. 8 (tied) among public university programs and No. 13 (tied) overall in its 2024 Best Part-Time MBA Programs.

AI, Assessment and Indirect Measures

accreditation visit or report, you have likely been exposed to program and course assessment. Meant as means of quality assurance and source of accountability, assessment can occur inside or outside the classroom. It can also be done thought direct, or indirect measures. Direct measures refer to any method of collecting assessment data that requires students to demonstrate a knowledge, skill, or behavior. The most common examples are standardized tests, cumulative exams, and research projects requiring a grade.

“It is our responsibility to provide the tools necessary for students to succeed, and AI has become a very important tool in higher education”

Indirect measures refer to any method of collecting data that requires reflection on student learning, skills, or behaviors, rather than a demonstration of it.

These methods have existed as long direct measures, but have not always been required to conduct assessment. However, with changes in students, institutions, and technology, more and more schools are being asked to use indirect measures to gauge student and program success. In the Jindal School, some of the indirect methods of assessment used include advisory board feedback, alumni and exit surveys, career fair evaluations, case competitions, and the work of student organizations. It is through these indirect measures of assessment where the Jindal School has started to incorporate AI, at both the course and program level.

Beginning in the fall of 2022, all graduate and undergraduate programs in the Jindal School of Management at UT Dallas began using a modified electronic template for capturing assessment data that stresses not only societal impact strategies, but how those strategies capture external examples of lifelong learning. The creation of this template modification was needed by the Jindal School to illustrate what we already knew but needed to document. That being: our programs were already contributing to the creation of a stronger society both in and outside of the classroom, but needed a better technological method to document the data. This documentation and use of data ultimately leads to better course mapping for both faculty and students. The added template, modeled after the 2020 Standards set forth by the Association to Advance Collegiate Schools of Business (AACSB), allows all programs to describe the specific activity used in or outside the classroom by entering data into specific columns.

Like other institutions, UT Dallas (and the Jindal School in particular) are addressing the AI revolution in higher education from a positive perspective. All faculty and staff know the AI tools are out there, and students will be using them regardless of warning, rules, or established guidelines. But one role of educators is to guide and provide the necessary decision-making skills to harness such tools, and a specific way the Jindal School is doing this is incorporating AI in the classroom…and in many cases the class itself.

Beginning in 2024, The Jindal School has made a concentrated effort to embrace AI as part of the curriculum, and that has led to the incorporation of various AI elements into the subject matter of existing classes at both the graduate and undergraduate level. In all, twenty-five existing courses have been updated to incorporate AI as part of the course description, course title, or class goal/outcome. A key example at the graduate level is MIS 6371 SAP Cloud Analytics, where AI integration for practitioners has been blended into the existing curriculum dealing with machine learning and data security. For undergraduates, OPRE 4362 is now Supply Chain Strategies, Processes and AI Applications, with AI enabled information technology being added to existing work with global supply chains, strategic and financial goals, and case studies related to supply chain trends.

But these improvements are not just for existing courses; AI is here to stay and new courses had to be created to properly prepare our students for the works that awaits them after graduation. Our faculty have created 26 new AI-oriented classes that will be offered as of Fall 2025, and the subjects they reach are not just information systems. For example, one key course is HMGT 6337 Applied AI in a Healthcare Setting, designed to assist students by utilizing AI in a larger enterprise strategy, emphasis on AI as a tool to improve costs, access, patient impact, and unmet needs by extracting actionable intelligence and data from healthcare systems. Another popular addition is MKT 3341 AI within the Sales Technology Landscape, which examines how Artificial Intelligence is expanding categories and applications with the existing sales technology landscape.

From as assessment perspective, these AI additions allow for faculty to better understand student learning as it relates to indirect measures. Surveys, interviews and focus groups that are often part of the class requirements can now be more easily understand and quantified using recognized AI methods. In addition, the existing student learning outcomes and goals can be revisited (and if necessary, updated) using the same AI tools utilized in classroom assignments. These changes are part of the world in which we live, and this same world will welcome our students upon graduation. It is our responsibility to provide the tools necessary for those students to succeed, and AI has become a very important tool in the higher education.

Supply Chain

Artificial Intelligence

Healthcare

Information Technology

Financial

Data Security

Educational Technology

Machine Learning