More on metrics: what about college?

After my post yesterday on metrics, the AVI left a good comment, and then wrote his own follow up post using  sports as an example.  It’s worth a read.

I’ve been thinking more about metrics today, and wondering about other areas where there’s no consensus on outcomes.  Before I get in to the rest of my thoughts, I wanted to mention a quick anecdote I once heard a pastor give.

Back when he was in high school, this man’s class had been handed a poll.  In it, they were asked what they most wanted to be in life:  rich, successful in their field, famous, successful in love, well traveled or happy.  According to him, when the teacher wrote the results on the board, he was the only one who had put “happy”.  As he discussed this with his classmates afterwards, he realized this was because they all had so closely associated happiness with one of the other metrics that it had never occurred to them that checking off “rich” might not be the same thing as checking off “happy”.

This occurs to me as a common mistake with metrics….we start associating two traits so closely that we forget they do not actually have to coexist.

This brings me to college.

In the student loan debates, there’s been much wailing over how much debt undergraduates are taking on, while the ability to obtain salaries that enable repayment has decreased.  In reading these articles, one would be left with the impression that we had some sort of national consensus on what the point of college actually is: to get a good job.

This is wrong.

According to the Pew Research Center:

Just under half of the public (47%) says the main purpose of a college education is to teach work-related skills and knowledge, while 39% say it is to help a student grow personally and intellectually; the remainder volunteer that both missions are equally important. College graduates place more emphasis on intellectual growth; those who are not college graduates place more emphasis on career preparation.

Even college presidents don’t agree on what they’re trying to do:

(College) Presidents are evenly divided about the main role colleges play in students’ lives: Half say it is to help them mature and grow intellectually, while 48% say it is to provide skills, knowledge and training to help them succeed in the working world. Most heads of four-year colleges and universities emphasize the former; most heads of two-year and for-profit schools emphasize the latter.

So half of the people heading up colleges never thought that their primary goal would be to get kids good jobs, and 40% of the public didn’t prioritize getting a good job.  Loans are generally based on an ability to repay, but a good chunk of those taking out the loans weren’t focused on ability to repay when they signed on.

My guess is that this is not what actually went through these people’s heads, at least not in those words.  My guess is that maturing and intellectual growth is so conflated with being qualified for a good job that it’s unfathomable to some people that they’re not the same thing.

Maybe they should start asking this on student loan applications.  I certainly think it should be at least be part of the conversation.

Outcome metrics and the research we do not do

I’ve spent most of last week at work trying to perfect a grant proposal that pretty much everyone in our program has to sign off on.  On Thursday, Friday and today there was a great deal of discussion about what metrics we could use to measure our outcomes, should we get funding.

It’s actually not an easy question, as the project we’re working on is a general good thing (patient education) designed to address a multitude of issues, as opposed to something more targeted.

Watching half a dozen people go back and forth about all this got me thinking about how often it is taken for granted that somewhere out there is a definition for “success” in various topics.

When I took a child development class in grad school, I remember in one of the first classes someone asked what the best parenting methods were.  Our professor replied that there really couldn’t be a consensus, because no one could agree on what would qualify as a success.  He proceeded to use religion as an example:  for parents of strong religious persuasion, a child who grew up a financially successful atheist would not necessarily be what they were going for.  Conversely, secular atheist parents might be distressed at a strong religious conversion.

There are probably scores of good studies that could have been done on parenting methods if we actually had a definition of success we could all agree on.  Too frequently, I think people overlook this point.  The reason so many strange fads in parenting can get going is because it is really really hard to prove anyone right or wrong.  Even if you try, you might just wind up with the dodo bird verdict.

If you can’t agree on where you’re going, you most certainly can’t tell people how to get there.  The studies you don’t do are often as important as the studies you do.