To respond to David’s question about folksonomies Aren’t we going to innovate our way out of this? My answer is yes, but only for small values of “out.” A big part of what’s coming is accepting and adapting to the mess, instead of exiting it.
Seeing people defend professional classification as a viable option for large systems is giving me horrible flashbacks to the arguments in 1993 about why gopher was superior to the Web. Gopher was categorized by professionals, and it was hierarchical. Gotta love hierarchy for forcing organization. You can’t just stick things any old place — to be able to add something to a hierarchy, you have to say where it goes, and what it goes with, next to, under, and above. And if you do it right, you can even call it an ontology, which means you get to charge extra. (I loves me some ontologies.)
The Web, meanwhile, was chaos. Chaos! You could link anything to anything else! Melvil Dewey would plotz if he saw such a tuml. How on earth could you organize the Web? The task is plainly impossible.
And you know what? The gopher people were right. The Web is chaos, and instead of getting the well-groomed world of gopher, we’ve adapted to the Web by meeting it half way.
Part of the solution has been that brilliant minds keep finding ways of extracting value from the mess, but another big part has been that we’ve adapted to the chaos, including satisficing search strategies, acceptance of partial results, the addition of post hoc and edge-based filters, and so on.
Anything that operates at really large scale takes on the characteristics of organic systems, including especially degeneracy, the principle that there is not a one-to-one mapping between function and location in the system. (Christopher Alexander got there a long time ago, in A City Is Not a Tree, to which we might only add that the Web is not a tree either.)
If we’re going to let just any old person write whatever they want and then make it available globally, we’re going to have to extend the same freedom to classifying the resulting flood of material. And critics will be able to say, rightly, that such a system will lack the coherence we would have had, had we not gone and let everyone publish willy-nilly. To which the only sensible reply, as to the gopher people, is “Oh well.”
The best we can do is figure out the second-order problems, such as lowest common denominator classification, and try to fix them. And this is the innovator’s lemma — classification is a sub-problem of the global publishing phenomenon. We have folksonomies because we have a world where professional classification and controlled vocabularies are as broken as gopher was in 1994. But getting folksonomic classification right isn’t the main event, which is why there’s much less flexibility than professional classifiers imagine.
You can find lots of people to blame for this problem — George Boole, Claude Shannon, Vint Cerf, Tim Berners-Lee, and so on. What’s certain is that the explosion of content that makes folksonomies a forced move was a done deal a long time ago. The librarians are now in the same situation as the journalists: “What will happen when everyone can do what we do, without having to be professionally trained?” And the answer, as always, is “We’ll see.”