SLO Assessment: Where the Buck Stops (and Starts)
Welcome to the first of an occasional series of offerings on issues I find to be of interest and importance to our membership. The following was prompted by essays by Robert Shireman, Senior Fellow at The Century Foundation, and Peter Ewell, President of the National Center for Higher Education Management Systems, related to Student Learning Outcomes assessment and accreditation.
I welcome your comments at mepetrisko [at] wascsenior.org
Mary Ellen Petrisko, President
WASC Senior College and University Commission
In the April 7 edition of Inside Higher Ed, Robert Shireman (“SLO Madness”) and Peter Ewell (“Improving with Age”) share their views on student learning outcomes (SLOs) and on accreditors’ roles and responsibilities related to quality assurance regarding teaching and learning. Ewell and others have commented on Shireman’s misunderstandings of SLOs and how they are assessed as discussed in “SLO Madness” and in a February 26 Century Foundation essay, “The Real Value of What Students Do in College.”
As president of the WASC Senior College and University Commission (WSCUC), one of the regional accrediting agencies, I would like to comment on the extent to which Shireman’s and Ewell’s comments fairly represent our current efforts and, importantly, the challenges we face in communicating and working with institutions about SLOs and their assessment processes.
Shireman states in “SLO Madness” that “validating colleges’ own quality-assurance systems should become the core of what accreditors do if they want to serve as a gateway to federal funds…With this approach, colleges are responsible for establishing their own systems for the occasional review of their majors and courses by outside experts they identify.
Validating institutions’ own systems is exactly what regional accreditors do.
Accreditors, meanwhile, have the responsibility of auditing those campus review processes, to make sure that they are comprehensive and valid, involving truly independent outsiders and the examination of student work.” In fact, validating institutions’ own systems is exactly what regional accreditors do.
WSCUC’s Standard 4, related to Quality Assurance Processes, requires that “The institution employs a deliberate set of quality-assurance processes in both academic and non-academic areas, including new curriculum and program approval processes, periodic program review, assessment of student learning, and other forms of ongoing evaluation...” It is not accidental that the program review processes of three institutions cited in Shireman’s Century Foundation essay as examples of “shining a light on the work that students do in their classes” and using external experts to help ensure objectivity are three WSCUC-accredited institutions: WSCUC’s Resource Guide for ‘Good Practices’ in Academic Program Review includes among its guiding principles the inclusion of such experts, who are expected to review, among other things, “examples of student work, such as senior papers and theses.”
Dependence on outside experts, however, is in no way sufficient to evaluate institutions’ own quality assurance systems.
Dependence on outside experts, however, is in no way sufficient to evaluate institutions’ own quality assurance systems. While a 2014 report published by the National Institute for Learning Outcomes Assessment (NILOA), “Knowing What Students Know and Can Do: The Current State of Student Learning Outcomes Assessment in US Colleges and Universities” included the statement that “regional and specialized/program accreditation remain the prime drivers of assessment work,” it also noted the central importance of institutional commitment and faculty interest in improving student learning in accomplishing this work. The critical role of this commitment and interest cannot be underestimated: assessment must be seen, encouraged, supported, and protected as the domain of the faculty.
Faculty perceive assessment to be a “bureaucratically imposed workload of dubious value". It is a problem that can stand in the way of quality improvement and that must be solved.
The fact that some faculty perceive assessment to be a “bureaucratically imposed workload of dubious value,” as Shireman describes it, is a problem with which accreditors are very familiar and a challenge that we have been working on for years.
It is a problem that can stand in the way of quality improvement and that can be solved (or dissolved) by emphasizing faculty’s engagement in assessment for the right reasons: i.e., for the purposes of improving teaching and learning.
This is made explicit in WSCUC’s Standard 2: “The institution’s student learning outcomes and standards of performance are developed by faculty and widely shared among faculty, students, staff, and (where appropriate) external stakeholders. The institution’s faculty take collective responsibility for establishing appropriate standards of performance and demonstrating through assessment the achievement of these standards.” And in Standard 4: “The faculty and other educators take responsibility for evaluating the effectiveness of teaching and learning processes and use the results for improvement of student learning and success.”
No one other than faculty can determine whether what they intend their students to learn has actually been learned.
In my discussions with faculty, I have stressed that no one other than they can determine whether what they intend their students to learn has actually been learned. No one other than they can and must determine what “actual outputs” provide the evidence that their students have achieved the course or program’s intended learning outcomes. No one other than they can and must determine, when intended outcomes are not met, what might or should be changed to improve that achievement. Finally, no one other than they have the authority and responsibility to make those changes—and to assess, down the road, whether the intended improvements have occurred.
It is for these reasons that a WSCUC-accredited institution cited in Shireman’s Century Foundation essay was praised for adjusting its faculty reward system to ensure that faculty take on the work of assessment—and that this work be done well. It is also why, as stated in the 2014 NILOA report cited above, “faculty are the key to moving assessment forward.” And it is the reason why Ewell writes, in “Improving with Age,” that “faculty members and institutional leaders need to engage in assessment primarily for purposes of improving their own teaching and learning practices. If they get that right, success with actors like regional accreditors will automatically follow.” Amen.
Accreditors can and must improve how we communicate.
That accreditors can and must improve how we communicate and work with institutions about SLOs and assessment processes is clear. Ewell suggests that this might be done by adopting more common language and notes, correctly, that accreditors tend to resist references to external frameworks like the Degree Qualifications Profile or AAC&U’s Essential Learning Outcomes.
WSCUC actually does mention these and other such tools and resources in our 2013 Handbook of Accreditation with relationship to our expectation that institutions define the meaning of the undergraduate and graduate degrees they confer that they ensure their quality and integrity. At the same time, it must be recognized that what accreditors require of their memberships must take into account those memberships’ autonomy, which presents a significant challenge to adopting common language, let alone shared expectations for learning, across the country.
What is the value of what students do in college?
What is the value of what students do in college? In his Century Foundation essay, Shireman rightly states that even those who believe students do gain something in college find it surprisingly difficult to identify what that something is, and that “the simplistic but wrong answer is that it is all the specific knowledge gained.” It is true that we forget much, perhaps even most, of the information that we are taught and/or learn in college, giving credence to the dictum that “education is what remains after one has forgotten what one earned in school.” But what would that education be without having learned in the first place what is forgotten later?
To what extent might we say that achieving student learning outcomes is a necessary, but perhaps not sufficient, condition for becoming an educated person?
Is it sufficient to involve students “in rich and meaningful educational activities” without identifying first what we hope those activities will accomplish and then assessing the extent to which they not only acquire information but through doing so develop the skills, habits, and attitudes necessary to keep learning, keep thinking, keep analyzing and drawing appropriate conclusions on which to base their actions?
Many of us, including accreditors, would say no.