Assessment After Ten Years: Reflection and Renewal
VAG Conference, November 1996
Margaret A. Miller
Good afternoon. This is my tenth year standing before
you, trying to peer into the future. I consider myself an ultramarathoner who is about to finish her
tenth race -- that means, you will recall, that I expect a silver belt buckle when I'm done. On the
other hand, an ultramarathoner's success is assessed by her or his capacity to finish a 100-mile
race in less than 24 hours. Mine might be by how well I anticipate what's ahead, and I must admit
that there's some question about that. My first such speech opened with the line, "Welcome to
what the other coordinators of this conference like to refer to as the first annual student
assessment conference in Virginia." Little did I suspect that we were going to have a run of annual
conferences that would rival am ultramarathon in its length.
But ten is a such a nice round number that it's hard to
resist the temptation to be oracular. For that reason, I've borrowed the conference's very fine title
for my paper: "Assessment After Ten Years: Reflection and Renewal." Beginning with
reflection, I recall that Ted Marchese asked, at the first annual assessment conference, whether we
had the time to do it right. "Does the Council have an adequate attention span?" was his actual
question, and I think he had some scepticism about the answer. I do think that any program, after
ten years, can be said to have reached a certain maturity. Here are the signs of it that I see in
assessment:
- On some campuses, paying attention to what students learn has become a habit of mind, so
that faculty there can't imagine not having that information when they change their programs.
- Some deans have come to count on assessment results as one source of information when
they make budget and hiring decisions.
- On at least one campus, big decisions -- like program terminations and major reconfigurings --
have been made on the basis of assessment results (or lack thereof).
- Assessment is increasingly becoming a part of a now-mandated process of program review, as
it has of program approval and productivity review. While you may not always welcome the extra
work, you may find yourselves becoming the best friends, suddenly, of people who didn't know
you existed before you held the keys to their kingdom.
- On several campuses assessment directors have become the all-purpose answer men and
women, evaluating anything that moves.
- One institution that only several years ago was still resistant is now using information
gathered through assessment to demonstrate how satisfied students are to have attended it.
- Another institution that is facing a major change has learned through assessment exactly what
it needs to know to make a success of that change.
- And finally, the assessment directors have formed a solid professional community who can
call on each other and upon whom the Council can call when it needs their expertise.
So what's ahead? Ten years ago I said that
"assessment which is understood as something an institution does apart from its normal activities
will, I hope, wither away in time." This, I think, is happening. Where it is happening well,
assessment has woven itself into the fabric of institutional life, so that its threads are very hard to
pick out, in either sense of the word -- to see, or to remove. In less successful cases, it continues
as a sidebar conversation. As a tool for evaluating the effects of change and contributing to
improvement on campuses, I think assessment in varying forms will continue for a long time --
there has been a genuine culture shift, so that it's hard to remember, for instance, that once we
didn't ask students how successful we have been with them.
But there are also new challenges. Any human
enterprise, like the human species itself, outwits extinction by being opportunistic -- it takes
advantage of the new opportunities thrown up by a changing environment to grow, while itself
changing. When David Potter reviewed the history of the assessment mandate in Virginia at this
meeting in 1991, he said that assessment was one of the Council's pre-emptive strikes, in this case
against the possibility that higher education was "in danger of being lumped together with
secondary education as an enterprise of questionable or declining quality." What Peter Ewell has
described as the unusual robustness of assessment in Virginia, in other words, resulted from its
creation to meet a challenge that was seen to be coming. And if I were to try to be prescient now,
I would say there are a set of opportunities that assessment programs will need to address if they
are to flourish in the next decade. The ones I see, although there may be others, are captured in
the "areas of general state interest" that are a part of the Guidelines for the Second Decade.
- How transfer information is being used to improve programs. From the beginning, we have
stressed that all the data in the world doesn't equal information, and information is not an end in
itself but a means to improvement.
- What students learn using the new technologies or new kinds of facilities, such as multimedia
classroom, compared to what they learn in more traditional formats or settings. This issue was
first mentioned in 1992. To quote myself:
If the use of technology is not to be a mere cost-saving measure, student learning
will have to be carefully assessed to determine what students can learn aided by a computer rather
than a by teacher, when the large-lecture format can work and how it can be made to work both
more efficiently and better with a judicious use of technology, and when there is no substitute for
a student on one end of the log and a teacher on the other.
- Learning goals for, and ways of measuring the learning that results from, various alternatives
to contact-hour models of credit, including alternative scheduling, internships, and other learning
experiences outside the classroom. In 1992 this is what I said about this subject:
In the traditional model, the process by which a student learns is fixed and the
outcome varies. The output model, the one on which assessment is based but traditional
curriculum construction is not, presumes that what is important is what the student knows at the
end of the process. In the output model, the expected outcome is fixed but neither how long it
takes nor how the student develops that mastery is important. Some students might take two
years to develop what the faculty agree are the capacities a graduate of the program should have,
while others might take seven. Some might develop those abilities and master that knowledge in
classrooms, while others would do so in front of a television screen or computer.
-- or, I might add, in internships, giving speeches in the community, or wherever.
The De Vrieses of this world are eating an important piece of our lunch by guaranteeing results,
and the faster-growing assessment movement in the country is probably industrial assessment.
Note the principle "Assessment should yield specific and detailed information, such as the number
and percentages of students reaching the desired level of competency and satisfaction." Vague
generalities can't compete and won't compute.
Using assessment to set and implement clear and rigorous performance standards for
academic progression and graduation. The new hot issue is "standards." Concern about
standards exists at three levels:
- Admissions. Many members of the public worry that students unprepared to do collegiate
work are being admitted. We need to track at-risk students and demonstrate that we can help
them succeed.
- Content, especially in the general education curriculum. The curricular reforms that followed
our inability to assess general education (note: one of our biggest successes came out of failure)
puts us ahead here, but we must follow through and assess the effectiveness of the new programs.
- Qualifications of graduates. If you received some of the letters I have received from recent
graduates, your hair would be curling uncontrollably right now. I said ten years ago, "I would
strongly encourage institutions to set standards to match their highest aspirations." I've seen little
evidence of that in the intervening decade.
I urge you to remember that one argument against
using assessment results as the basis of Funds for Excellence project funding was that assessment
is about the past. It is up to you to make it shape the future.
|