Closing the loop

One phrase that occurs repeatedly in the language of assessment is "closing the loop." It comes from the practice of illustrating processes using graphs or networks. The idea is that one collects data concerning an activity and then reexamines the activity in the light of that data. Some do not like the phrase and prefer to say "using the data." It all amounts to the same thing.

One cannot over-emphasize the importance of closing the loop/using the data. Without this piece, the entire process is pointless. Regardless of how efficient we are, regardless of how simple we keep the process in toto, it still requires energy, i.e. effort and expense. I believe the Second Law of Thermodynamics is at work. Given the expense of taking data, if that data is not used, then money has been wasted. This is a tautology.

Now, having said that, I need to be careful. Sometimes we must monitor things that are in good shape and don't require our action just because they are important. For example, even people who don't have high cholesterol have their cholesterol monitored regularly; they just don't have it monitored as regularly as those who have issues in that area. But we do keep an eye on it because there are consequences if it creeps out of the safety zone. Similarly, there are areas where a particular department might excel and have done so over a historically lengthy interval, and there is no need for change in that area, but it is so important that it should be monitored. In such cases, one can take measurements at longer intervals, such as every three years or every five years.

The point is that data is taken to be used. This is an important point to make, especially in the initial stages of setting up a system of assessment.

In many cases, assessment is rolled-down from above upon groups which are unevenly prepared. In some cases, the message gets through that "Someone up there wants numbers." The response is then to "Come up with some numbers," irrespective of whether or not those numbers are meaningful. This is, of course, not the best possible response.

We don't take data simply for the great joy of taking data. It is done for a purpose.

When I check the outdoor temperature, it is to determine whether I need to wear a parka or gym shorts when I go outside. Data taken in the act of assessment similarly should have a purpose. Measures should be implemented with the idea that particular results will result in particular actions. When I check the outdoor temperature, it measuring the temperature on Mars for which my actions do not matter. I measure it outside my own house. Similarly, whatever departments measure should have some meaning to their mission at the university.