Assessment After Ten Years: Reflection and Renewal

VAG Conference, November 1996
Margaret A. Miller

      Good afternoon. This is my tenth year standing before you, trying to peer into the future. I consider myself an ultramarathoner who is about to finish her tenth race -- that means, you will recall, that I expect a silver belt buckle when I'm done. On the other hand, an ultramarathoner's success is assessed by her or his capacity to finish a 100-mile race in less than 24 hours. Mine might be by how well I anticipate what's ahead, and I must admit that there's some question about that. My first such speech opened with the line, "Welcome to what the other coordinators of this conference like to refer to as the first annual student assessment conference in Virginia." Little did I suspect that we were going to have a run of annual conferences that would rival am ultramarathon in its length.

      But ten is a such a nice round number that it's hard to resist the temptation to be oracular. For that reason, I've borrowed the conference's very fine title for my paper: "Assessment After Ten Years: Reflection and Renewal." Beginning with reflection, I recall that Ted Marchese asked, at the first annual assessment conference, whether we had the time to do it right. "Does the Council have an adequate attention span?" was his actual question, and I think he had some scepticism about the answer. I do think that any program, after ten years, can be said to have reached a certain maturity. Here are the signs of it that I see in assessment:

  • On some campuses, paying attention to what students learn has become a habit of mind, so that faculty there can't imagine not having that information when they change their programs.
  • Some deans have come to count on assessment results as one source of information when they make budget and hiring decisions.
  • On at least one campus, big decisions -- like program terminations and major reconfigurings -- have been made on the basis of assessment results (or lack thereof).
  • Assessment is increasingly becoming a part of a now-mandated process of program review, as it has of program approval and productivity review. While you may not always welcome the extra work, you may find yourselves becoming the best friends, suddenly, of people who didn't know you existed before you held the keys to their kingdom.
  • On several campuses assessment directors have become the all-purpose answer men and women, evaluating anything that moves.
  • One institution that only several years ago was still resistant is now using information gathered through assessment to demonstrate how satisfied students are to have attended it.
  • Another institution that is facing a major change has learned through assessment exactly what it needs to know to make a success of that change.
  • And finally, the assessment directors have formed a solid professional community who can call on each other and upon whom the Council can call when it needs their expertise.

      So what's ahead? Ten years ago I said that "assessment which is understood as something an institution does apart from its normal activities will, I hope, wither away in time." This, I think, is happening. Where it is happening well, assessment has woven itself into the fabric of institutional life, so that its threads are very hard to pick out, in either sense of the word -- to see, or to remove. In less successful cases, it continues as a sidebar conversation. As a tool for evaluating the effects of change and contributing to improvement on campuses, I think assessment in varying forms will continue for a long time -- there has been a genuine culture shift, so that it's hard to remember, for instance, that once we didn't ask students how successful we have been with them.

      But there are also new challenges. Any human enterprise, like the human species itself, outwits extinction by being opportunistic -- it takes advantage of the new opportunities thrown up by a changing environment to grow, while itself changing. When David Potter reviewed the history of the assessment mandate in Virginia at this meeting in 1991, he said that assessment was one of the Council's pre-emptive strikes, in this case against the possibility that higher education was "in danger of being lumped together with secondary education as an enterprise of questionable or declining quality." What Peter Ewell has described as the unusual robustness of assessment in Virginia, in other words, resulted from its creation to meet a challenge that was seen to be coming. And if I were to try to be prescient now, I would say there are a set of opportunities that assessment programs will need to address if they are to flourish in the next decade. The ones I see, although there may be others, are captured in the "areas of general state interest" that are a part of the Guidelines for the Second Decade.

  • How transfer information is being used to improve programs. From the beginning, we have stressed that all the data in the world doesn't equal information, and information is not an end in itself but a means to improvement.
  • What students learn using the new technologies or new kinds of facilities, such as multimedia classroom, compared to what they learn in more traditional formats or settings. This issue was first mentioned in 1992. To quote myself:

    If the use of technology is not to be a mere cost-saving measure, student learning will have to be carefully assessed to determine what students can learn aided by a computer rather than a by teacher, when the large-lecture format can work and how it can be made to work both more efficiently and better with a judicious use of technology, and when there is no substitute for a student on one end of the log and a teacher on the other.
  • Learning goals for, and ways of measuring the learning that results from, various alternatives to contact-hour models of credit, including alternative scheduling, internships, and other learning experiences outside the classroom. In 1992 this is what I said about this subject:
    In the traditional model, the process by which a student learns is fixed and the outcome varies. The output model, the one on which assessment is based but traditional curriculum construction is not, presumes that what is important is what the student knows at the end of the process. In the output model, the expected outcome is fixed but neither how long it takes nor how the student develops that mastery is important. Some students might take two years to develop what the faculty agree are the capacities a graduate of the program should have, while others might take seven. Some might develop those abilities and master that knowledge in classrooms, while others would do so in front of a television screen or computer.
-- or, I might add, in internships, giving speeches in the community, or wherever.
The De Vrieses of this world are eating an important piece of our lunch by guaranteeing results, and the faster-growing assessment movement in the country is probably industrial assessment. Note the principle "Assessment should yield specific and detailed information, such as the number and percentages of students reaching the desired level of competency and satisfaction." Vague generalities can't compete and won't compute.
  • Using assessment to set and implement clear and rigorous performance standards for academic progression and graduation. The new hot issue is "standards." Concern about standards exists at three levels:
    • Admissions. Many members of the public worry that students unprepared to do collegiate work are being admitted. We need to track at-risk students and demonstrate that we can help them succeed.
    • Content, especially in the general education curriculum. The curricular reforms that followed our inability to assess general education (note: one of our biggest successes came out of failure) puts us ahead here, but we must follow through and assess the effectiveness of the new programs.
    • Qualifications of graduates. If you received some of the letters I have received from recent graduates, your hair would be curling uncontrollably right now. I said ten years ago, "I would strongly encourage institutions to set standards to match their highest aspirations." I've seen little evidence of that in the intervening decade.
          I urge you to remember that one argument against using assessment results as the basis of Funds for Excellence project funding was that assessment is about the past. It is up to you to make it shape the future.

  •