In the most recent Communications of the ACM there was yet another article discussing open online courses. The title of the article was provocative (“Are the Costs of ‘Free’ Too High in Online Education?”), and I couldn’t help reading it. While I appreciated the author’s points and I think my time was well spent reading it, it reminded me of a frustrating phenomenon. Computing educators seem to display an appalling inability to place their work in context, by which I mean discuss previous work that is related to their project/approach/ideas. In this article, the author wrote:
Technology now makes it possible to reach many more students at minimal costs compared to on-campus education. These “distance learning” efforts began years ago, but have gained special prominence recently.
He then went on to describe some of the more recent efforts, including Khan Academy and Coursera. There wasn’t a single mention of the vast online learning literature that existed long before any of the mentioned approaches were a gleam in their creators’ eyes. And, yes, previous online learning was focused more narrowly, with 30 or 300 students, not 30,000, but they did exist. To have an article in the CACM ignore hundreds, if not thousands, of studies done over more than a decade is a bit surprising. And, yes, I know that many, if not most, of those studies may not be relevant. But an acknowledgment that online education didn’t spring into existence in 2002 when MIT started Open Courseware would have been appropriate.
This isn’t the first time I’ve seen this phenomenon. I don’t know how many conference, and even journal, submissions I’ve reviewed where the authors ignored vast bodies of literature. I’ve seen it enough that I’ve begun to speculate on why it happens. I used to think that it was laziness, that the authors couldn’t be bothered to look for previous work, but I’ve since decided that’s uncharitable. In the case I mention above, it may be a bit of arrogance. Something can’t exist until the top-tier universities do it. But I’m not sure that’s accurate either, since researchers in other areas of computing don’t fall victim to that. A theoretician will cite all previous work, not just the work done by researchers at the top-tier universities. I’m beginning to think that’s it’s because many people who work in computing education simply weren’t trained effectively. They did their graduate work in another area of computing, but then drifted into (or dabble with) computing education. The problem there is that computing education is a social science, and a good social science researcher does a broad literature review as one of the initial steps in any research project. Computing education researchers don’t always seem to have this instinct, and it can produce a sort of obliviousness in the research.
Even if I’m right about the cause of the obliviousness, I’m stumped about how to fix it. It will be a long time (if ever) before everyone working in computing education got there because of graduate study. Reviews seem like a possible way to enforce good summaries of existing literature, but lately I’ve lost faith in the ability of reviewers to even read the entire article much less require something as stringent as a good literature review. So there is no clear path out of this. I do think it’s worth trying to find one though, because having articles that don’t place work in context can only damage the credibility of the field in the long term.