The importance of writing

As I may have mentioned in earlier posts, my university is in the process of putting together a system of assessing general education. At the foundation of our general education program are the basic skills. I would argue that the cornerstone of that foundation is writing.

Writing, of course, is a basic means of communication. When a student responds to a test question—at least when beyond the survey classes—the response is in writing, regardless of whether the answer is an essay, a sentence, "text", or simply just a word. Its importance is far beyond that. Writing—good writing—requires that the student have established an internal dialog. We all have multiple voices within ourselves, but having an internal dialog requires that we discipline those voices. We must be able to take an idea and attempt to support it while at the same time honestly looking for valid criticism of that idea. Such an internal dialog enables us to be surer of our opinions and to be able to defend them in a reasoned way.

I would also submit that having learned to write enables us to become better readers. The struggle to convey a particular idea using the writer's toolbox builds the ability to better examine another's writing for nuance. As someone who's played football is better able to appreciate the finer points of the game so is someone who has written better able to understand the subtleties there.

One of the reasons we have chosen to tackle writing first is because of its fundamental importance to learning at level and breadth of the university. Because of its importance, we know that it is thing that we must strive to excel at. As a consequence, the assessment tools we put in place to measure that success must be create carefully and well. These tools, and the spirit of the endeavor, will provide a template to follow in the creation of assessment tools for mathematics and communication, and from there for the rest of general education.

Journey to the core

General education at my university is designed so as to have certain basic skills at its core. These are Writing, Mathematics, and Communication. The vast majority of academics would agree on the importance and foundational nature of these subjects though, of course, each would have view of the value of each.

Communication is obviously important for its utility in helping each of us to communicate what we know. Students gain knowledge over the course of their careers and it is important that they be able to convey what they know at the end of the experience. We need also remember that communication is a two-way street. By learning to communicate one's own ideas, a student is better able to understand the methods teachers use in communicating to them. They are better able to analyze what the teacher is saying.

Writing, of course, is obviously important as a means of communication, but, in addition to this, the process of writing well and employing convincing arguments cannot be separated from critical thinking. Creating an effective argument requires having a robust internal dialog. One must not only be able to create arguments but look for holes in those arguments and find ways to patch those holes or to discern they are un-patchable.

Then my area of specialty, mathematics, enters the fray. That mathematics is basic to the physical sciences no one will dispute. Indeed, given the shift in biology from the field to the lab, few would dispute that a knowledge of mathematics is necessary to the life sciences as well. Certainly few would argue against the utility of statistics in the social sciences and business. As one drifts more deeply into the humanities one meets not only doubt as to its value but outright hostility.

As I said, I am a mathematician and we are used to hostility, so perhaps it is not surprising that I believe mathematics is important to the humanities too, but I would also concede that those in the humanities might be better served by a different sort of mathematics than those in the sciences. I can appreciate music without knowing how to play the guitar, but I can get more out of a musical experience by being taught certain things. An artist need not be taught the technical details of complex variables in order to appreciate the hyperbolic plane.

We are on the verge of appointing a task force to study the mathematical needs on this campus. It grew out of our needs in assessing the mathematical component of general education but its scope is greater than this. Ultimately, it may provide mechanisms for the university to maximize the impact of mathematics on our campus.

Value Added

As a member of my university's faculty, I've been a participant in many conversations concerning assessment. Most (and I say most just to simplify exposition) of the faculty I've known have a deep desire to make their students

1. Love the subject they are teaching;
2. Become as proficient at that subject as the student is capable.

But when faced with the prospect of measuring the student's knowledge in a setting larger than the faculty member's office--like that provided by an institutional assessment system--those faculty become nervous. I speak from personal experience when I say the prospect of having the quality of your work examined in not necessarily comfortable.

For the most part, we as teachers do the best we know how but, as the mathematician in me would put it, there are variables that are beyond our control and they make us feel exposed. One of the concerns frequently voiced is the preparation, or lack thereof, of students when they “come in the front door” and being held accountable for how those students perform when they go out the back door.

There are a number of reasonable answers to this concern. One of these is that all faculty at the university are teaching those same students. Ill-prepared students affect all faculty across the board.

Another answer, which is more germane to the practice of assessment, is the concept of added value.

When a diamond is taken from the mine, it is rough and dull. The diamond has to be cut by and expert hand and polished before it is marketable. When potatoes are dug from the ground, they are covered with soil. They have to be washed, pealed, and cooked before they can be eaten. The diamond and the potato have a natural worth, but value is added to them through processing.

We, in the university, attempt to add value to students in our own special way. Students come in the front door of the class, involve themselves in it activities, and should look different when they go out the back. We can measure the value-added to those students by measuring the change.

There are problems with this.

This is still bothersome because those of us who have taught know that not all students take the fullest advantage of their time under our expert guidance. For this, I retreat to my first answer that everyone teaching at the university is teaching those same students. If they don’t work for me, they are not going to work for you, and if they do work for you, perhaps I should learn from you how you get them to do it.

Another issue could be phrased in this way. Not all diamonds are the same. Some are small and are always going to be small. The master’s hand can maximize their value, but that value will have a small upper bound. Some diamonds, while large and beautiful, come in requiring very little cutting or polishing. They might leave the shop with a very high value, but that might not be much more than when they came in.

The answer to this is not as easy and may require putting an offering before the gods of statistics. It also requires us to learn more about our students and to seek better ways to teach the students we have. This includes learning the strengths and weaknesses they bring with them and learning the ways we can use their strengths and eliminate or compensate for their weaknesses.

Assessing General Education

My university has begun the process of extending our system of assessment into the general education program. Before I began this process I thought I knew the purpose of general education and I was happily ignorant of its size.

Before I launch into thoughts concerning the purpose of general education, let me first comment as to its extent. At my university, each student must have at least a total of 124 credit hours to graduate. Of those 124, at least 46 credit hours must be general education. This is slightly over one-third of the total requirement. Somewhere among the remaining two-thirds of those 124 hours, the student must specialize in something and do some exploring on his own.

In the book Excellence without a Soul, Harry Lewis remarks that the idea that students should know one thing very well but also know a little about everything has long been an ideal at Harvard where he taught. This same ideal permeates higher education throughout the United States and informs our attitude toward general education. In addition, there are other functions performed by general education.

In a setting where many of the students are the first generation of their families to attend college, there is often a lack of knowledge among the students and their support groups of pathways open to them through education. General education provide samples of areas in which students might find a career that they might not otherwise have known about. We all have heard the student stories that begin "I'd never even heard of X before, but I took Professor Y's Introduction to X because my adviser made me, and it changed my life."

There are also aspects of general education that are foundational to later learning. At my school, we have decided these are writing, mathematics, and communication. We refer to these as our Basic Skills.

It is with the Basic Skills that we have chosen to begin as we extend our assessment system to general education. This is being done for a number of reasons. The disciplines involved in this area are almost universally (as near as can be expected on a university campus) accepted as being a part of general education. In addition, Basic Skills only includes courses being taught by three departments, and, therefore, the politics involved are more manageable.

Even though we are only beginning, it has been discovered that expectations for student learning outcomes in these areas can vary widely. We are currently looking at writing, and, without being too surprised, we've learned different disciplines expect different styles of writing. Some expect students to write in a literary style, some in a business style. Some expect essays and some expect text. As we continue to examine mathematics and communications, I expect similar results.

Our purpose, however, is not to bludgeon the faculty into saying there is a single correct vision of any of these areas. We must open lines of communication so that the needs of the students are correctly understood by those whose charge it is to teach them. And, as assessment is our charge, we must do that in such a way that we can measure how well these needs are being met.

Closing the loop

One phrase that occurs repeatedly in the language of assessment is "closing the loop." It comes from the practice of illustrating processes using graphs or networks. The idea is that one collects data concerning an activity and then reexamines the activity in the light of that data. Some do not like the phrase and prefer to say "using the data." It all amounts to the same thing.

One cannot over-emphasize the importance of closing the loop/using the data. Without this piece, the entire process is pointless. Regardless of how efficient we are, regardless of how simple we keep the process in toto, it still requires energy, i.e. effort and expense. I believe the Second Law of Thermodynamics is at work. Given the expense of taking data, if that data is not used, then money has been wasted. This is a tautology.

Now, having said that, I need to be careful. Sometimes we must monitor things that are in good shape and don't require our action just because they are important. For example, even people who don't have high cholesterol have their cholesterol monitored regularly; they just don't have it monitored as regularly as those who have issues in that area. But we do keep an eye on it because there are consequences if it creeps out of the safety zone. Similarly, there are areas where a particular department might excel and have done so over a historically lengthy interval, and there is no need for change in that area, but it is so important that it should be monitored. In such cases, one can take measurements at longer intervals, such as every three years or every five years.

The point is that data is taken to be used. This is an important point to make, especially in the initial stages of setting up a system of assessment.

In many cases, assessment is rolled-down from above upon groups which are unevenly prepared. In some cases, the message gets through that "Someone up there wants numbers." The response is then to "Come up with some numbers," irrespective of whether or not those numbers are meaningful. This is, of course, not the best possible response.

We don't take data simply for the great joy of taking data. It is done for a purpose.

When I check the outdoor temperature, it is to determine whether I need to wear a parka or gym shorts when I go outside. Data taken in the act of assessment similarly should have a purpose. Measures should be implemented with the idea that particular results will result in particular actions. When I check the outdoor temperature, it measuring the temperature on Mars for which my actions do not matter. I measure it outside my own house. Similarly, whatever departments measure should have some meaning to their mission at the university.

Creating the environment, a culture of continuous improvement

As I've mentioned earlier, I am a mathematics teacher and have been teaching the subject in one way or another with varying levels of intensity since 1983. As a teacher, I've never been able to open a student's head and just pour knowledge in. Lately, I've had fantasies of surgically implanting a USB port in the skull of each student and just plugging him in so that he can slurp up the knowledge. That is unlikely to happen any time soon.

My method of teaching is to structure my classes with a time-line. I set learning goals and embed those goals in a calendar. I then organize the material and try to personalize the presentation of it to the needs of each class collectively and each individual within that class as much as possible by using language that the class understands, using a suitable idiom.

This is a learning environment. Within this environment, the student must apply his own efforts and abilities if he is to learn. There are students who are better at this than others, and I am not necessarily talking about students who are more intelligent. What I mean, is there are students who have cultivated habits which are conductive to the act of learning.

There are teachers as well who have, in an intentional way, acquired habits which improve their teaching. These people will be better teachers on the day they retire than on the day they started. They will be better teachers next year than they are this year.

Part of the business of a healthy educational institution is to create an environment which fosters that sort of teacher. We must create what is called "a culture of continuous improvement."

I use that phrase even though over the course of my career I've seen it misused extensively. Having been a mathematician, I'm not unfamiliar with folks who interpret words in a direct way, and I've seen that phrase taken to mean that there is an expectation that every class will improve its scores every year. This is, of course, idiotic. Because of random variation within each class, it is unreasonable to expect scores that are continually increasing.

It is not, by way of contrast, unreasonable to expect an environment where teachers are encouraged to improve their teaching and given the means, within the available resources, to do so. Indeed, far from being unreasonable, this is a natural expectation.

The creation of an institutional system of assessment is a part of that environment.

Thoughts on sampling

My academic discipline is mathematics. It is natural for someone to say, "Oh you know a lot about statistics," but the truth is more complicated.

I am a topologist, and I've only ever had one course in statistics which was as an undergraduate and was theoretical. However, at a school like ours, we are not allowed to be to picky as to what we teach, so, as a consequence of this, I started teaching our elementary statistics course about fifteen years ago and discovered that I liked the subject. Now, after having taught it for that length of time, I cannot claim expertise, but I can say that I do understand some of the basic ideas.

One of the basic ideas is that of a sample. I would like to explain that to you.

Statistics is about the search for answers in a particular spirit. We begin with the need to know, but with the realization that we can't know exactly. Given that, we proceed to find the approximate answer with the realization that the approximate answer will, in a certain number of cases, not be good enough. It is very practical.

We might start out desiring to know what the average IQ of the students on our campus is. The obvious way to approach this would be to administer an IQ test to every student on campus. There are problems with this, however: Students don't like taking tests, testing is expensive, and so forth. What we would need to do, in this case, would be to take a sample.

The idea is that a sample serves as a representative of the whole. One of the great mathematical miracles that make statistics work is that most samples do serve as representatives of the whole. One of the dangers is that regardless of that, bias can creep in through sampling techniques.

The gold standard of taking a sample of size 100 would be to choose 100 student identification numbers at random and then to administer the test to those 100 individuals. One problem with this is that some of that 100 will not show up, so we would have to over sample by a certain amount. But a more basic problem is to get any so show up at all. It is possible to offer incentives, but then the risk is biasing the sample by the type of reward you offer. What if you offer an iPod as a prize? Will you bias this by attracting students who like to gamble? Will you bias it toward groups that don't already have iPods?

In a similar project, we've dealt with this by choosing to measure particular classes, not for IQ but for other things. In doing this, there is a risk of bias toward students who choose particular classes, but the low cost of this method makes it very attractive and, being aware of the possibility of bias, we can be on the guard against it.