Using the Campus LMS to Capture Assessment Data

Sheri H. Barrett, Director, Johnson County Community College

Even before the COVID-19 challenges that sent many of our colleagues and students into the online world of teaching and learning, many Colleges and Universities had begun exploring and using the campus Learning Management System (LMS) to capture assessment data. Given the many different types of LMS systems that are being used in higher ed (in the 10 years I have been in my current post we have used three different LMS software packages) I can’t give specific instructions, but I can suggest some general guidelines to consider when choosing or using your campus LMS.

I have frequently joked that what most LMS developers need is a good assessment person on staff. All of the systems I have worked with over the years have had some ability to capture assessment data in a way that is beneficial to the faculty member, but not all of them have been geared to the needs of reporting at the department, program or institutional level. It is important if the campus is considering moving to a new or considering using your current LMS capture assessment data it is important to understand its capabilities for collection. Below are a few suggestions when the campus is reviewing a potential LMS software for this purpose:

• Determine if the assessment capabilities of the LMS are standard or require additional modules or costs to implement.

• I have often been told that something is “in-development” but it never seems to appear during the life of the contract.

• Many LMS software have assessment components that are fairly straightforward for faculty use but require rather extensive work on the part of staff to retrieve assessment data that can be aggregated at the program or campus level.

• With our current system the Educational Technology staff must extract the data each semester, and then my office spends several days cleaning up and massaging the data to create usable charts and graphs we can provide to the programs or faculty member.

• Make sure the assessment processes within the LMS software are easy for the faculty to use.

• I personally have never believed that the terms “software” and “intuitive” should appear in the same sentence. I have often been assured by sales consultants that the assessment module is very “intuitive” but have just as often found them anything but.

As I have noted, there are a plethora of companies that provide LMS software to campuses, there are some commonalities with the systems that are potential assessment tools for the campus.

• Most LMS software will allow you to incorporate your campus learning outcomes into the system for assignments by students to be connected to by the faculty.

•This ability is a great first step when you want to think about how you can map, gather data, and analyze your institutional or general education learning outcomes. Be sure to ask for a demonstration of how you can “report out” this data.

• Some systems have multiple levels of this process, allowing mapping to both the campus-wide outcomes and the program level outcomes.

“All of the systems I have worked with over the years have had some ability to capture assessment data in a way that is beneficial to the faculty member, but not all of them have been geared to the needs of reporting at the department, program or institutional level.”

A caution here: although it is an excellent tool, this is not the best way to do a true curriculum map for your general education or institutional level learning outcomes. I am also not a fan of requiring faculty to have every assignment attached to an institutional or general education learning outcome. Doing so moves this process from improving student learning to a compliance based “proving” student learning. It is like the old adage, just because you can do something, doesn’t mean you should do something. As an example, in a recent the assessment cycle on the campus I had a department that mapped every student assignment in the LMS to every one of the College’s general education learning outcomes, and three of the five institutional learning outcomes. This despite the fact that the program doesn’t have any general education curriculum, and not all these student assignments matched with their assessment initiative.

Most LMS software allows for common rubrics to be loaded in for faculty access. If you are a campus that uses a common rubric to assess campus wide learning outcomes this is a great means of providing it to your faculty. However, having it provided through the LMS does not replace the need to train for inter-rater reliability!

Some LMS features will also allow for programs to share within just their courses a common rubric. We currently use that functionality but must do so with the help of the Educational Technology staff, it is not something that can be completed by the faculty.

Training is still the most important key to using the LMS to capture assessment data. Most systems have a very specific set of steps, in a particular sequence, during a specific timeline! We had an entire department decide to move their assessment rubric online for the Fall semester. I worked with the department head and we provided both written instruction and a video of how to accomplish this goal. Fast forward to the end of the term and we received multiple calls from faculty telling us they were ready to “set it up” now. But the LMS required everything to be completed at the BEGINNING of the semester!

Finally, my advice on using your LMS to gather assessment data is to be aware that working with it won’t necessarily lighten the amount of work that needs to be performed. It is a great tool, but like any tool requires ongoing work. This includes training faculty in the appropriate use of the tool, keeping up with upgrades from the LMS and determining what impact changes have on the assessment function, making appropriate updates in the system (like when we changed our general education outcomes), and knowing that someone, in this case my office, is going to have to become an expert in this area of the LMS.