Clay Shirky
( Archive | Home )

Liz Lawley
( Archive | Home )

Ross Mayfield
( Archive | Home )

Sébastien Paquet
( Archive | Home )

David Weinberger
( Archive | Home )

danah boyd
( Archive | Home )

Guest Authors
Recent Comments

Donte on My book. Let me show you it.

telecharger subway surfers on My book. Let me show you it.

Ask Fm Anonymous Finder on My book. Let me show you it.

Ask Fm Anonymous Finder on My book. Let me show you it.

mobile games on My book. Let me show you it. on My book. Let me show you it.

Site Search
Monthly Archives
RSS 1.0
RSS 2.0
In the Pipeline: Don't miss Derek Lowe's excellent commentary on drug discovery and the pharma industry in general at In the Pipeline


February 28, 2008

My book. Let me Amazon show you it.Email This EntryPrint This Article

Posted by Clay Shirky

I’m delighted to say that online bookstores are shipping copies of Here Comes Everybody today, and that it has gotten several terrific notices in the blogosphere:

Cory Doctorow:
Clay’s book makes sense of the way that groups are using the Internet. Really good sense. In a treatise that spans all manner of social activity from vigilantism to terrorism, from Flickr to Howard Dean, from blogs to newspapers, Clay unpicks what has made some “social” Internet media into something utterly transformative, while other attempts have fizzled or fallen to griefers and vandals. Clay picks perfect anecdotes to vividly illustrate his points, then shows the larger truth behind them.
Russell Davies:
Here Comes Everybody goes beyond wild-eyed webby boosterism and points out what seems to be different about web-based communities and organisation and why it’s different; the good and the bad. With useful and interesting examples, good stories and sticky theories. Very good stuff.
Eric Nehrlich:
These newly possible activities are moving us towards the collapse of social structures created by technology limitations. Shirky compares this process to how the invention of the printing press impacted scribes. Suddenly, their expertise in reading and writing went from essential to meaningless. Shirky suggests that those associated with controlling the means to media production are headed for a similar fall.
Philip Young:
Shirky has a piercingly sharp eye for the spotting the illuminating case studies - some familiar, some new - and using them to energise wider themes. His basic thesis is simple: “Everywhere you look groups of people are coming together to share with one another, work together, take some kind of public action.” The difference is that today, unlike even ten years ago, technological change means such groups can be form and act in new and powerful ways. Drawing on a wide range of examples Shirky teases out remarkable contrasts with what has been the expected logic, and shows quite how quickly the dynamics of reputation and relationships have changed.

Comments (30) + TrackBacks (0) | Category: social software

February 7, 2008

My book. Let me show you it.Email This EntryPrint This Article

Posted by Clay Shirky

I’ve written a book, called Here Comes Everybody: The Power of Organizing Without Organizations, which is coming out in a month. It’s coming out first in the US and UK (and in translation later this year in Holland, Portugal and Brazil, Korea, and China.)

Here Comes Everybody is about why new social tools matter for society. It is a non-techie book for the general reader (the letters TCP IP appear nowhere in that order). It is also post-utopian (I assume that the coming changes are both good and bad) and written from the point of view I have adopted from my students, namely that the internet is now boring, and the key question is what we are going to do with it.

One of the great frustrations of writing a book as opposed to blogging is seeing a new story that would have been a perfect illustration, or deepened an argument, and not being able to add it. To remedy that, I’ve just launched a new blog, at, to continue writing about the effects of social tools.

Wow. What a great response — we’ve given out all the copies we can, but many thanks for all the interest. Also, I’ve convinced the good folks at Penguin Press to let me give a few review copies away to people in the kinds of communities the book is about. I’ve got half a dozen copies to give to anyone reading this, with the only quid pro quo being that you blog your reactions to it, good bad or indifferent, some time in the next month or so. Drop me a line if you would like a review copy —

Comments (119) + TrackBacks (0) | Category: social software

August 3, 2007

User-generated neologism: "Indigenous content"Email This EntryPrint This Article

Posted by Clay Shirky

My class in the fall is called “User-generated”, and it looks, among other things, at the tension surrounding that phrase, and in particular its existence as an external and anxiety-ridden label, by traditional media companies, for the way that advertising can be put next to material not created by Trained Professionals™.

All right-thinking individuals (by which I basically mean Anil Dash and Heather Champ) hate that phrase. Now my friend Kio Stark* has come up with what seems like a nice, and more anthropologically correct version: Indigenous Content (which is to say “Created by the natives for themselves.”)

* ObKio: Best. Tagset. Evar.

Comments (12) + TrackBacks (0) | Category:

August 1, 2007

New Freedom Destroys Old Culture: A response to Nick CarrEmail This EntryPrint This Article

Posted by Clay Shirky

I have never understood Nick Carr’s objections to the cultural effects of the internet. He’s much too smart to lump in with nay-sayers like Keen, and when he talks about the effects of the net on business, he sounds more optimistic, even factoring in the wrenching transition, so why aren’t the cultural effects similar cause for optimism, even accepting the wrenching transition in those domains as well?

I think I finally got understood the dichotomy between his reading of business and culture after reading Long Player, his piece on metadata and what he calls “the myth of liberation”, a post spurred in turn by David Weinberger’s Everything Is Miscellaneous.

Carr discusses the ways in which the long-playing album was both conceived of and executed as an aesthetic unit, its length determined by a desire to hold most of the classical canon on a single record, and its possibilities exploited by musicians who created for the form — who created albums, in other words, rather than mere bags of songs. He illustrates this with an exegesis of the Rolling Stones’ Exile on Main Street, showing how the overall construction makes that album itself a work of art.

Carr uses this point to take on what he calls the myth of liberation: “This mythology is founded on a sweeping historical revisionism that conjures up an imaginary predigital world - a world of profound physical and economic constraints - from which the web is now liberating us.” Carr observes, correctly, that the LP was what it was in part for aesthetic reasons, and the album, as a unit, became what it became in the hands of people who knew how to use it.

That is not, however, the neat story Carr wants to it be, and the messiness of the rest of the story is key, I think, to the anxiety about the effects on culture, his and others.

The LP was an aesthetic unit, but one designed within strong technical constraints. When Edward Wallerstein of Columbia Records was trying to figure out how long the long-playing format should be, he settled on 17 minutes a side as something that would “…enable about 90% of all classical music to be put on two sides of a record.” But why only 90%? Because 100% would be impossible — the rest of the canon was too long for the technology of the day. And why should you have to flip the record in the middle? Why not have it play straight through? Impossible again.

Contra Carr, in other words, the pre-digital world was a world of profound physical and economic constraints. The LP could hold 34 minutes of music, which was a bigger number of minutes than some possibilities (33 possibilities, to be precise), but smaller than an infinite number of others. The album as a form provided modest freedom embedded in serious constraints, and the people who worked well with the form accepted those constraints as a way of getting at those freedoms. And now the constraints are gone; there is no necessary link between an amount of music and its playback vehicle.

And what Carr dislikes, I think, is evidence that the freedoms of the album were only as valuable as they were in the context of the constraints. If Exile on Main Street was as good an idea as he thinks it was, it would survive the removal of those constraints.

And it hasn’t.

Here is the iTunes snapshot of Exile, sorted by popularity:

While we can’t get absolute numbers from this, we can get relative ones — many more people want to listen to Tumbling Dice or Happy than Ventilator Blues or Turd on the Run, even though iTunes makes it cheaper per song to buy the whole album. Even with a financial inducement to preserve the album form, the users still say no thanks.

The only way to support the view that Exile is best listened to as an album, in other words, is to dismiss the actual preferences of most of the people who like the Rolling Stones. Carr sets about this task with gusto:
Who would unbundle Exile on Main Street or Blonde on Blonde or Tonight’s the Night - or, for that matter, Dirty Mind or Youth and Young Manhood or (Come On Feel the) Illinoise? Only a fool would.
Only a fool. If you are one of those people who has, say, Happy on your iPod (as I do), then you are a fool (though you have lots of company). And of course this foolishness extends to the recording industry, and to the Stones themselves, who went and put Tumbling Dice on a Greatest Hits collection. (One can only imagine how Carr feels about Greatest Hits collections.)

I think Weinberger’s got it right about liberation, even taking at face value the cartoonish version Carr offers. Prior to unlimited perfect copyability, media was defined by profound physical and economic constraints, and now it’s not. Fewer constraints and better matching of supply and demand are good for business, because business is not concerned with historical continuity. Fewer constraints and better matching of supply and demand are bad for current culture, because culture continually mistakes current exigencies for eternal verities.

This isn’t just Carr of course. As people come to realize that freedom destroys old forms just as surely as it creates new ones, the lament for the long-lost present is going up everywhere. As another example, Sven Birkerts, the literary critic, has a post in the Boston Globe, Lost in the blogosphere, that is almost indescribably self-involved. His two complaints are that newspapers are reducing the space allotted to literary criticism, and too many people on the Web are writing about books. In other words, literary criticism, as practiced during Birkerts’ lifetime, was just right, and having either fewer or more writers are both lamentable situations.

In order that the “Life was better when I was younger” flavor of his complaint not become too obvious, Birkerts frames the changing landscape not as a personal annoyance but as A Threat To Culture Itself. As he puts it “…what we have been calling “culture” at least since the Enlightenment — is the emergent maturity that constrains unbounded freedom in the interest of mattering.”

This is silly. The constraints of print were not a product of “emergent maturity.” They were accidents of physical production. Newspapers published book reviews because their customers read books and because publishers took out ads, the same reason they published pieces about cars or food or vacations. Some newspapers hired critics because they could afford to, others didn’t because they couldn’t. Ordinary citizens didn’t write about books in a global medium because no such medium existed. None of this was an attempt to “constrain unbounded freedom” because there was no such freedom to constrain; it was just how things were back then.

Genres are always created in part by limitations. Albums are as long as they are because that Wallerstein picked a length his engineers could deliver. Novels are as long as they are because Aldus Manutius’s italic letters and octavo bookbinding could hold about that many words. The album is already a marginal form, and the novel will probably become one in the next fifty years, but that also happened to the sonnet and the madrigal.

I’m old enough to remember the dwindling world, but it never meant enough to me to make me a nostalgist. In my students’ work I see hints of a culture that takes both the new freedoms and the new constraints for granted, but the fullest expression of that world will probably come after I’m dead. But despite living in transitional times, I’m not willing to pretend that the erosion of my worldview is a crisis for culture itself. It’s just how things are right now.

Carr fails to note that the LP was created for classical music, but used by rock and roll bands. Creators work within whatever constraints exist at the time they are creating, and when the old constraints give way, new forms arise while old ones dwindle. Some work from the older forms will survive — Shakespeare’s 116th sonnet remains a masterwork — while other work will wane — Exile as an album-length experience is a fading memory. This kind of transition isn’t a threat to Culture Itself, or even much of a tragedy, and we should resist attempts to preserve old constraints in order to defend old forms.

Comments (49) + TrackBacks (0) | Category: social software

July 20, 2007

Spolsky on Blog Comments: Scale mattersEmail This EntryPrint This Article

Posted by Clay Shirky

Joel Spolsky approvingly quotes Dave Winer on the subject of blog-comments:

The cool thing about blogs is that while they may be quiet, and it may be hard to find what you’re looking for, at least you can say what you think without being shouted down. This makes it possible for unpopular ideas to be expressed. And if you know history, the most important ideas often are the unpopular ones…. That’s what’s important about blogs, not that people can comment on your ideas. As long as they can start their own blog, there will be no shortage of places to comment.

Joel then adds his own observations:

When a blog allows comments right below the writer’s post, what you get is a bunch of interesting ideas, carefully constructed, followed by a long spew of noise, filth, and anonymous rubbish that nobody … nobody … would say out loud if they had to take ownership of their words.

This can be true, all true, as any casual read of blog comments can attest. BoingBoing turned off their comments years ago, because they’d long since passed the scale where polite conversation was possible. The Tragedy of the Conversational Commons becomes too persistently tempting when an audience gorws large. At BoingBoing scale, John Gabriel’s Greater Internet Fuckwad Theory cannot be repealed.

But the uselessness of comments it is not the universal truth that Dave or (fixed, per Dave’s comment below) Joel makes it out to be, for two reasons. First, posting and conversation are different kinds of things — same keyboard, same text box, same web page, different modes of expression. Second, the sites that suffer most from anonymous postings and drivel are the ones operating at large scale.

If you are operating below that scale, comments can be quite good, in a way not replicable in any “everyone post to their own blog”. To take but three recent examples, take a look at the comments on my post on Michael Gorman, on danah’s post at Apophenia on fame, narcissism and MySpace and on Kieran Healy’s biological thought experiment on Crooked Timber.

Those three threads contain a hundred or so comments, including some distinctly low-signal bouquets and brickbats. But there is also spirited disputation and emendation, alternate points of view, linky goodness, and a conversational sharpening of the argument on all sides, in a way that doesn’t happen blog to blog. This, I think, is the missing element in Dave and Joel’s points — two blog posts do not make a conversation. The conversation that can be attached to a post is different in style and content, and in intent and effect, than the post itself.

I have long thought that the ‘freedom of speech means no filtering’ argument is dumb where blogs are concerned — it is the blogger’s space, and he or she should feel free to delete, disemvowel, or otherwise dispose of material, for any reason, or no reason. But we’ve long since passed the point where what happens on a blog is mainly influenced by what the software does — the question to ask about comments is not whether they are available, but how a community uses them. The value in in blogs as communities of practice is considerable, and its a mistake to write off comment threads on those kinds of blogs just because, in other environments, comments are lame.

Comments (20) + TrackBacks (0) | Category: social software

July 10, 2007

"The internet's output is data, but its product is freedom"Email This EntryPrint This Article

Posted by Clay Shirky

I said that in Andrew Keen: Rescuing ‘Luddite’ from the Luddites, to which Phil, one of the commenters, replied

There are assertions of verifiable fact and then there are invocations of shared values. Don’t mix them up.

I meant this as an assertion of fact, but re-reading it after Tom’s feedback, it comes off as simple flag-waving, since I’d compressed the technical part of the argument out of existence. So here it is again, in slightly longer form:

The internet’s essential operation is to encode and transmit data from sender to receiver. In 1969, this was not a new capability; we’d had networks that did this in since the telegraph, at the day of the internet’s launch, we had a phone network that was nearly a hundred years old, alongside more specialized networks for things like telexes and wire-services for photographs.

Thus the basics of what the internet did (and does) isn’t enough to explain its spread; what is it for has to be accounted for by looking at the difference between it and the other data-transfer networks of the day.

The principal difference between older networks and the internet (ARPAnet, at its birth) is the end-to-end principle, which says, roughly, “The best way to design a network is to allow the sender and receiver to decide what the data means, without asking the intervening network to interpret the data.” The original expression of this idea is from the Saltzer and Clark paper End-to-End Arguments in System Design; the same argument is explained in other terms in Isenberg’s Stupid Network and Searls and Weinberger’s World of Ends.

What the internet is for, in other words, what made it worth adopting in a world already well provisioned with other networks, was that the sender and receiver didn’t have to ask for either help or permission before inventing a new kind of message. The core virtue of the internet was a huge increase in the technical freedom of all of its participating nodes, a freedom that has been translated into productive and intellectual freedoms for its users.

As Scott Bradner put it, the Internet means you don’t have to convince anyone else that something is a good idea before trying it. The upshot is that the internet’s output is data, but its product is freedom.

Comments (7) + TrackBacks (0) | Category: social software

July 9, 2007

Andrew Keen: Rescuing 'Luddite' from the LudditesEmail This EntryPrint This Article

Posted by Clay Shirky

Last week, while in a conversation with Andrew Keen on the radio show To The Point, he suggested that he was not opposed to the technology of the internet, but rather to how it was being used.

This reminded me of Michael Gorman’s insistence that digital tools are fine, so long as they are shaped to replicate the social (and particularly academic) institutions that have grown up around paper.

There is a similar strand in these two arguments, namely that technology is one thing, but the way it is used is another, and that the two can and should be separated. I think this view is in the main wrong, even Luddite, but to make such an accusation requires a definition of Luddite considerably more grounded than ‘anti-technology’ (a vacuous notion — no one who wears shoes can reasonably be called ‘anti-technology.’) Both Keen and Gorman have said they are not opposed to digital technology. I believe them when they say this, but I still think their views are Luddite, by historical analogy with the real Luddite movement of the early 1800s.

What follows is a long detour into the Luddite rebellion, followed by a reply to Keen about the inseparability of the internet from its basic effects.

Infernal Machines

The historical record is relatively clear. In March of 1811, a group of weavers in Nottinghamshire began destroying mechanical looms. This was not the first such riot — in the late 1700s, when Parliament refused to guarantee the weaver’s control of supply of woven goods, workers in Nottingham destroyed looms as well. The Luddite rebellion, though, was unusual for several reasons: its breadth and sustained character, taking place in many industrializing towns at once; its having a nominal leader, going by the name Ned Ludd, General Ludd, or King Ludd (the pseudonym itself a reference to an apocryphal figure from an earlier loom-breaking riot in the late 1700s); and its written documentation of grievances and rationale. The rebellion, which lasted two years, was ultimately put down by force, and was over in 1813.

Over the last two decades, several historians have re-examined the record of the Luddite movement, and have attempted to replace the simplistic view of Luddites as being opposed to technological change with a more nuanced accounting of their motivations and actions. The common thread of the analysis is that the Luddites didn’t object to mechanized wide-frame looms per se, they objected to the price collapse of woven goods caused by the way industrialists were using the looms. Though the target of the Luddite attacks were the looms themselves, their concerns and goals were not about technology but about economics.

I believe that the nuanced view is wrong, and that the simpler view of Luddites as counter-revolutionaries is in fact the correct one. The romantic view of Luddites as industrial-age Robin Hoods, concerned not to halt progress but to embrace justice, runs aground on both the written record, in which the Luddites outline a program that is against any technology that increases productivity, and on their actions, which were not anti-capitalist but anti-consumer. It also assumes that there was some coherent distinction between technological and economic effects of the looms; there was none.

A Technology is For Whatever Happens When You Use It

The idea that the Luddites were targeting economic rather than technological change is a category fallacy, where the use of two discrete labels (technology and economics, in this case) are wrongly thought to demonstrate two discrete aspects of the thing labeled (here wide-frame looms.) This separation does not exist in this case; the technological effects of the looms were economic. This is because, at the moment of its arrival, what a technology does and what it is for are different.

What any given technology does is fairly obvious: rifles fire bullets, pencils make marks, looms weave cloth, and so on. What a technology is for, on the other hand, what leads people to adopt it, is whatever new thing becomes possible on the day of its arrival. The Winchester repeating rifle was not for firing bullets — that capability already existed. It was for decreasing the wait between bullets. Similarly, pencils were not for writing but for portability, and so on.

And the wide-frame looms, target of the Luddite’s destructive forays? What were they for? They weren’t for making cloth — humankind was making cloth long before looms arrived. They weren’t for making better cloth — in 1811, industrial cloth was inferior to cloth spun by the weavers. Mechanical looms were for making cheap cloth, lots and lots of cheap cloth. The output of a mechanical loom was cloth, but the product of such a loom was savings.

The wide-frame loom was a cost-lowering machine, and as such, it threatened the old inefficiencies on which the Luddite’s revenues depended. Their revolt had the goal of preventing those savings from being passed along to the customer. One of their demands was that Parliament outlaw “all Machinery hurtful to Commonality” — all machines that worked efficiently enough to lower prices.

Perhaps more tellingly, and against recent fables of Luddism as a principled anti-capitalist movement, they refrained from breaking the looms of industrial weavers who didn’t lower their prices. What the Luddites were rioting in favor of was price gouging; they didn’t care how much a wide-frame loom might save in production costs, so long as none of those savings were passed on to their fellow citizens.

Their common cause was not with citizens and against industrialists, it was against citizens and with those industrialists who joined them in a cartel. The effect of their campaign, had it succeeded, would been to have raise, rather than lower, the profits of the wide-frame operators, while producing no benefit for those consumers who used cloth in their daily lives, which is to say the entire population of England. (Tellingly, none of the “Robin Hood” versions of Luddite history make any mention of the effect of high prices on the buyers of cloth, just on the sellers.)

Back to Keen

A Luddite argument is one in which some broadly useful technology is opposed on the grounds that it will discomfit the people who benefit from the inefficiency the technology destroys. An argument is especially Luddite if the discomfort of the newly challenged professionals is presented as a general social crisis, rather than as trouble for a special interest. (“How will we know what to listen to without record store clerks!”) When the music industry suggests that the prices of music should continue to be inflated, to preserve the industry as we have known it, that is a Luddite argument, as is the suggestion that Google pay reparations to newspapers or the phone company’s opposition to VoIP undermining their ability to profit from older ways of making phone calls.

This is what makes Keen’s argument a Luddite one — he doesn’t oppose all uses of technology, just ones that destroy older ways of doing things. In his view, the internet does not need to undermine the primacy of the copy as the anchor for both filtering and profitability.

But Keen is wrong. What the internet does is move data from point A to B, but what it is for is empowerment. Using the internet without putting new capabilities into the hands of its users (who are, by definition, amateurs in most things they can now do) would be like using a mechanical loom and not lowering the cost of buying a coat — possible, but utterly beside the point.

The internet’s output is data, but its product is freedom, lots and lots of freedom. Freedom of speech, freedom of the press, freedom of association, the freedom of an unprecedented number of people to say absolutely anything they like at any time, with the reasonable expectation that those utterances will be globally available, broadly discoverable at no cost, and preserved for far longer than most utterances are, and possibly forever.

Keen is right in understanding that this massive supply-side shock to freedom will destabilize and in some cases destroy a number of older social institutions. He is wrong in believing that there is some third way — lets deploy the internet, but not use it to increase the freedom of amateurs to do as they like.

It is possible to want a society in which new technology doesn’t demolish traditional ways of doing things. It is not possible to hold this view without being a Luddite, however. That view — incumbents should wield veto-power over adoption of tools they dislike, no matter the positive effects for the citizenry — is the core of Luddism, then and now.

Comments (26) + TrackBacks (0) | Category: social software

June 20, 2007

Gorman, redux: The Siren Song of the InternetEmail This EntryPrint This Article

Posted by Clay Shirky

Michael Gorman has his next post up at the Britannica blog: The Siren Song of the Internet. My reply is also up, and posted below. The themes of the historical lessons of Luddism are also being discussed in the comments to last week’s Gorman response, Old Revolutions Good, New Revolutions Bad

Siren Song of the Internet contains a curious omission and a basic misunderstanding. The omission is part of his defense of the Luddites; the misunderstanding is about the value of paper and the nature of e-books.

The omission comes early: Gorman cavils at being called a Luddite, though he then embraces the label, suggesting that they “…had legitimate grievances and that their lives were adversely affected by the mechanization that led to the Industrial Revolution.” No one using the term Luddite disputes the effects on pre-industrial weavers. This is the general case — any technology that fixes a problem (in this case the high cost of homespun goods) threatens the people who profit from the previous inefficiency. However, Gorman omits mentioning the Luddite response: an attempt to halt the spread of mechanical looms which, though beneficial to the general populace, threatened the livelihoods of King Ludd’s band.

By labeling the Luddite program legitimate, Gorman seems to be suggesting that incumbents are right to expect veto power over technological change. Here his stand in favor of printed matter is inconsistent, since printing was itself enormously disruptive, and many people wanted veto power over its spread as well. Indeed, one of the great Luddites of history (if we can apply the label anachronistically) was Johannes Trithemius, who argued in the late 1400s that the printing revolution be contained, in order to shield scribes from adverse effects. This is the same argument Gorman is making, in defense of the very tools Trithemius opposed. His attempt to rescue Luddism looks less like a principled stand than special pleading: the printing press was good, no matter happened to the scribes, but let’s not let that sort of thing happen to my tribe.

Gorman then defends traditional publishing methods, and ends up conflating several separate concepts into one false conclusion, saying “To think that digitization is the answer to all that ails the world is to ignore the uncomfortable fact that most people, young and old, prefer to interact with recorded knowledge and literature in the form of print on paper.”

Dispensing with the obvious straw man of “all that ails the world”, a claim no one has made, we are presented with a fact that is supposed to be uncomfortable — it’s good to read on paper. Well duh, as the kids say; there’s nothing uncomfortable about that. Paper is obviously superior to the screen for both contrast and resolution; Hewlett-Packard would be about half the size it is today if that were not true. But how did we get to talking about paper when we were talking about knowledge a moment ago?

Gorman is relying on metonymy. When he notes a preference for reading on paper he means a preference for traditional printed forms such as books and journals, but this is simply wrong. The uncomfortable fact is that the advantages of paper have become decoupled from the advantages of publishing; a big part of preference for reading on paper is expressed by hitting the print button. As we know from Lyman and Varian’s “How Much Information” study, “…the vast majority of original information on paper is produced by individuals in office documents and postal mail, not in formally published titles such as books, newspapers and journals.”

We see these effects everywhere: well over 90% of new information produced in any year is stored electronically. Use of the physical holdings of libraries are falling, while the use of electronic resources is rising. Scholarly monographs, contra Gorman, are increasingly distributed electronically. Even the physical form of newspapers is shrinking in response to shrinking demand, and so on.

The belief that a preference for paper leads to a preference for traditional publishing is a simple misunderstanding, demonstrated by his introduction of the failed e-book program as evidence that the current revolution is limited to “hobbyists and premature adopters.” The problems with e-books are that they are not radical enough: they dispense with the best aspect of books (paper as a display medium) while simultaneously aiming to disable the best aspects of electronic data (sharability, copyability, searchability, editability.) The failure of e-books is in fact bad news for Gorman’s thesis, as it demonstrates yet again that users have an overwhelming preference for the full range of digital advantages, and are not content with digital tools that are designed to be inefficient in the ways that printed matter is inefficient.

If we gathered every bit of output from traditional publishers, we could line them up in order of vulnerability to digital evanescence. Reference works were the first to go — phone books, dictionaries, and thesauri have largely gone digital; the encyclopedia is going, as are scholarly journals. Last to go will be novels — it will be some time before anyone reads One Hundred Years of Solitude in any format other than a traditionally printed book. Some time, however, is not forever. The old institutions, and especially publishers and libraries, have been forced to use paper not just for display, for which is it well suited, but also for storage, transport, and categorization, things for which paper is completely terrible. We are now able to recover from those disadvantages, though only by transforming the institutions organized around the older assumptions.

The ideal situation, which we are groping our way towards, will be to have all written material, wherever it lies on the ‘information to knowledge’ continuum, in digital form, right up the moment a reader wants it. At that point, the advantages of paper can be made manifest, either by printing on demand, or by using a display that matches paper’s superior readability. Many of the traditional managers of books and journals will suffer from this change, though it will benefit society as a whole. The question Gorman pointedly asks, by invoking Ned Ludd and his company, is whether we want that change to be in the hands of people who would be happy to discomfit society as a whole in order to preserve the inefficiencies that have defined their world.

Comments (6) + TrackBacks (0) | Category: social software

June 18, 2007

Mis-understanding Fred Wilson's 'Age and Entrepreneurship' argumentEmail This EntryPrint This Article

Posted by Clay Shirky

Technorati reports approximately one buptillion disputatious replies to Fred Wilson’s observations about age and tech entrepreneurship. These come in two basic forms: examples from industry (“Youth doesn’t matter because Steve Jobs is still going strong”) and examples from personal experience (“Youth doesn’t matter because my grandmother invented DoS attacks when she was 87!”)

These arguments, not to put too fine a point on it, are stupid.

Fred is not talking about intelligence or even tech chops. He is talking about a specific kind of tech entrepreneurialism: the likelihood of coming up with an idea that is so powerful it will shift the tech landscape. He is then asserting that, statistically, young people have an edge in their ability to come up with these kinds of ideas.

The first counter-argument, whose commonest explanandum is Steve Jobs current success, not only fails, it actually supports Fred’s point. Back In The Day, Jobs’ best decision was to work with Wozniak, and together they brought out usable versions of the GUI and the mouse. These changes were so radical that they didn’t catch on until they were copied in more pedestrian and backwards-compatible forms. And now? What is Apple doing with a seasoned Jobs at the helm? They are polishing the GUI to a fare-thee-well. They are making Diamond’s idea of an MP3 player work better than anyone imagined it could. They are making (brainstorm alert!) a phone! Woz was a mere tinkerer in light of such revolutionary moves, no?

As a Mac user, I love what Jobs is doing for the company, but no way am I willing to confuse the Polecat release of the current OS with what Lisa tried and the Mac achieved in the early 80s.

Then there is the personal attestation to brilliant ideas, coming from the outraged older set. I guess I should feel some sort of pride that my fellow proto-geriatrics are still in there fighting, but instead, I think they kind of prove Fred’s point by demonstrating that they’ve either forgotten how to read, or that they can’t do math so good anymore.

Fred’s basic observation is statistical: In the domain E, with the actors divided into two groups Y and O, there are more Y in E than you’d expect from a random distribution, and many more if you thought there should be an advantage to being a member of O.

This observation cannot be falsified by a single counter-example. Given that Fred’s argument is about the odds of success (he is a VC, after all), the fact that you remember the words to the Pina Colada song and you recently did something useful is meaningless. Fred’s question is about how many grizzled veterans are founding world-changing tech firms, not whether any are.

There are lots of possible counter-arguments to what Fred is saying (and I am echoing): Maybe so many young people start companies that the observation suffers from denominator bias. Or: young people raise money from VCs in disproportionate numbers because they don’t have the contacts to raise money in other ways. Or: the conservatism of the old is social, not mental, and concerns for family and quality of life turn otherwise undiminished imaginations to lower-risk goals. And so on.

It would be good if someone made those arguments — the thesis is provocative and it matters, so it should be scrutinized and, if false, destroyed. But Fred has said something important, something with both internal evidence (the list of successful recent entrepreneurs) and external existence proofs (mathematicians careers are also statistically front-weighted, so the pattern isn’t obviously absurd.) Given this, the argument cannot simply be whined away, robbing many of the current respondents of the weapon with which they are evidently most adept.

Comments (10) + TrackBacks (0) | Category:

June 16, 2007

The Future Belongs to Those Who Take The Present For Granted: A return to Fred Wilson's "age question"Email This EntryPrint This Article

Posted by Clay Shirky

My friend Fred Wilson had a pair of posts a few weeks back, the first arguing that youth was, in and of itself an advantage for tech entrepreneurs, and the second waffling on that question with idea that age is a mindset.

I think Fred got it right the first time, and I said so at the time, in The (Bayesian) Advantages of Youth:

I’m old enough to know a lot of things, just from life experience. I know that music comes from stores. I know that newspapers are where you get your political news and how you look for a job. I know that if you need to take a trip, you visit a travel agent. In the last 15 years or so, I’ve had to unlearn those things and a million others. This makes me a not-bad analyst, because I have to explain new technology to myself first — I’m too old to understand it natively. But it makes me a lousy entrepreneur.
Today, Fred seems to have returned to his original (and in my view correct) idea in The Age Question (continued):
It is incredibly hard to think of new paradigms when you’ve grown up reading the newspaper every morning. When you turn to TV for your entertainment. When you read magazines on the train home from work. But we have a generation coming of age right now that has never relied on newspapers, TV, and magazines for their information and entertainment.[…] The Internet is their medium and they are showing us how it needs to be used.

This is exactly right.

I think the real issue, of which age is a predictor, is this: the future belongs to those who take the present for granted. I had this thought while talking to Robert Cook of Metaweb, who are making Freebase. They need structured metadata, lots of structured metadata, and one of the places they are getting it is from Wikipedia, by spidering the bio boxes (among other things) for things like birthplace and age of people listed Freebase. While Andrew Keen is trying to get a conversation going on whether Wikipedia is a good idea, Metaweb takes it for granted as a stable part of the environment, which lets them see past this hurdle to the next one.

This is not to handicap the success of Freebase itself — it takes a lot more than taking the present for granted to make a successful tool. But one easy way to fail is to assume that the past is more solid than it is, and the present more contingent. And the people least likely to make this mistake — the people best able to take the present for granted — are young people, for whom knowing what the world is really like is as easy as waking up in the morning, since this is the only world they’ve ever known.

Some things improve with age — I wouldn’t re-live my 20s if you paid me — but high-leverage ignorance isn’t one of them.

Comments (20) + TrackBacks (0) | Category: social software

June 13, 2007

Old Revolutions Good, New Revolutions Bad: A Response to GormanEmail This EntryPrint This Article

Posted by Clay Shirky

Encyclopedia Britannica has started a Web 2.0 Forum, where they are hosting a conversation going on around a set of posts by Michael Gorman. The first post, in two parts, is titled Web 2.0: The Sleep of Reason Brings Forth Monsters, and is a defense of the print culture against alteration by digital technologies. This is my response, which will be going up on the Britannica site later this week.

Web 2.0: The Sleep of Reason Brings Forth Monsters starts with a broad list of complaints against the current culture, from biblical literalism to interest in alternatives to Western medicine.

The life of the mind in the age of Web 2.0 suffers, in many ways, from an increase in credulity and an associated flight from expertise. Bloggers are called “citizen journalists”; alternatives to Western medicine are increasingly popular, though we can thank our stars there is no discernable “citizen surgeon” movement; millions of Americans are believers in Biblical inerrancy—the belief that every word in the Bible is both true and the literal word of God, something that, among other things, pits faith against carbon dating; and, scientific truths on such matters as medical research, accepted by all mainstream scientists, are rejected by substantial numbers of citizens and many in politics. Cartoonist Garry Trudeau’s Dr. Nathan Null, “a White House Situational Science Adviser,” tells us that: “Situational science is about respecting both sides of a scientific argument, not just the one supported by facts.”

This is meant to set the argument against a big canvas of social change, but the list is so at odds with the historical record as to be self-defeating.

The percentage of the US population believing in the literal truth of the Bible has remained relatively constant since the 1980s, while the percentage listing themselves as having “no religion” has grown. Interest in alternative medicine dates to at least the patent medicines of the 19th century; the biggest recent boost for that movement came under Reagan, when health supplements, soi-disant, were exempted from FDA scrutiny. Trudeau’s welcome critique of the White House’s assault on reason targets a political minority, not the internet-using population, and so on. If you didn’t know that this litany appeared under the heading Web 2.0, you might suspect Gorman’s target was anti-intellectualism during Republican administrations.

Even the part of the list specific to new technology gets it wrong. Bloggers aren’t called citizen-journalists; bloggers are called bloggers. Citizen-journalist describes people like Alisara Chirapongse, the Thai student who posted photos and observations of the recent coup during a press blackout. If Gorman can think of a better label for times when citizens operate as journalists, he hasn’t shared it with us.

Similarly, lumping Biblical literalism with Web 2.0 misses the mark. Many of the most active social media sites — Slashdot, Digg, Reddit — are rallying points for those committed to scientific truth. Wikipedia users have so successfully defended articles on Evolution, Creationism and so on from the introduction of counter-factual beliefs that frustrated literalists helped found Conservapedia, whose entry on Evolution is a farrago of anti-scientific nonsense.

But wait — if use of social media is bad, and attacks on the scientific method are bad, what are we to make of social media sites that defend the scientific method? Surely Wikipedia is better than Conservapedia on that score, no? Well, it all gets confusing when you start looking at the details, but Gorman is not interested in the details. His grand theory, of the hell-in-a-handbasket variety, avoids any look at specific instantiations of these tools — how do the social models of Digg and Wikipedia differ? does Huffington Post do better or worse than Instapundit on factual accuracy? — in favor of one sweeping theme: defense of incumbent stewards of knowledge against attenuation of their erstwhile roles.

There are two alternate theories of technology on display in Sleep of Reason. The first is that technology is an empty vessel, into which social norms may be poured. This is the theory behind statements like “The difference is not, emphatically not, in the communication technology involved.” (Emphasis his.) The second theory is that intellectual revolutions are shaped in part by the tools that sustain them. This is the theory behind his observation that the virtues of print were “…often absent in the manuscript age that preceded print.”

These two theories cannot both be true, so it’s odd to find them side by side, but Gorman does not seem to be comfortable with either of them as a general case. This leads to a certain schizophrenic quality to the writing. We’re told that print does not necessarily bestow authenticity and that some digital material does, but we’re also told that he consulted “authoritative printed sources” on Goya. If authenticity is an option for both printed and digital material, why does printedness matter? Would the same words on the screen be less scholarly somehow?

Gorman is adopting a historically contingent view: Revolution then was good, revolution now is bad. As a result, according to Gorman, the shift to digital and networked reproduction of information will fail unless it recapitulates the institutions and habits that have grown up around print.

Gorman’s theory about print — its capabilities ushered in an age very different from manuscript culture — is correct, and the same kind of shift is at work today. As with the transition from manuscripts to print, the new technologies offer virtues that did not previously exist, but are now an assumed and permanent part of our intellectual environment. When reproduction, distribution, and findability were all hard, as they were for the last five hundred years, we needed specialists to undertake those jobs, and we properly venerated them for the service they performed. Now those tasks are simpler, and the earlier roles have instead become obstacles to direct access.

Digital and networked production vastly increase three kinds of freedom: freedom of speech, of the press, and of assembly. This perforce increases the freedom of anyone to say anything at any time. This freedom has led to an explosion in novel content, much of it mediocre, but freedom is like that. Critically, this expansion of freedom has not undermined any of the absolute advantages of expertise; the virtues of mastery remain are as they were. What has happened is that the relative advantages of expertise are in precipitous decline. Experts the world over have been shocked to discover that they were consulted not as a direct result of their expertise, but often as a secondary effect — the apparatus of credentialing made finding experts easier than finding amateurs, even when the amateurs knew the same things as the experts.

This improved ability to find both content and people is one of the core virtues of our age. Gorman insists that he was able to find “…the recorded knowledge and information I wanted [about Goya] in seconds.” This is obviously an impossibility for most of the population; if you wanted detailed printed information on Goya and worked in any environment other than a library, it would take you hours at least. This scholars-eye view is the key to Gorman’s lament: so long as scholars are content with their culture, the inability of most people to enjoy similar access is not even a consideration.

Wikipedia is the best known example of improved findability of knowledge. Gorman is correct that an encyclopedia is not the product of a collective mind; this is as true of Wikipedia as of Britannica. Gorman’s unfamiliarity and even distaste for Wikipedia leads him to mistake the dumbest utterances of its most credulous observers for an authentic accounting of its mechanisms; people pushing arguments about digital collectivism, pro or con, known nothing about how Wikipedia actually works. Wikipedia is the product not of collectivism but of unending argumentation; the corpus grows not from harmonious thought but from constant scrutiny and emendation.

The success of Wikipedia forces a profound question on print culture: how is information is to be shared with the majority of the population? This is an especially tough question, as print culture has so manifestly failed at the transition to a world of unlimited perfect copies. Because Wikipedia’s contents are both useful and available, it has eroded the monopoly held by earlier modes of production. Other encyclopedias now have to compete for value to the user, and they are failing because their model mainly commits them to denying access and forbidding sharing. If Gorman wants more people reading Britannica, the choice lies with its management. Were they to allow users unfettered access to read and share Britannica’s content tomorrow, the only interesting question is whether their readership would rise a ten-fold or a hundred-fold.

Britannica will tell you that they don’t want to compete on universality of access or sharability, but this is the lament of the scribe who thinks that writing fast shouldn’t be part of the test. In a world where copies have become cost-free, people who expend their resources to prevent access or sharing are forgoing the principal advantages of the new tools, and this dilemma is common to every institution modeled on the scarcity and fragility of physical copies. Academic libraries, which in earlier days provided a service, have outsourced themselves as bouncers to publishers like Reed-Elsevier; their principal job, in the digital realm, is to prevent interested readers from gaining access to scholarly material.

If Gorman were looking at Web 2.0 and wondering how print culture could aspire to that level of accessibility, he would be doing something to bridge the gap he laments. Instead, he insists that the historical mediators of access “…promote intellectual development by exercising judgment and expertise to make the task of the seeker of knowledge easier.” This is the argument Catholic priests made to the operators of printing presses against publishing translations of the Bible — the laity shouldn’t have direct access to the source material, because they won’t understand it properly without us. Gorman offers no hint as to why direct access was an improvement when created by the printing press then but a degradation when created by the computer. Despite the high-minded tone, Gorman’s ultimate sentiment is no different from that of everyone from music executives to newspaper publishers: Old revolutions good, new revolutions bad.

Comments (48) + TrackBacks (0) | Category: social software

May 24, 2007

What are we going to say about "Cult of the Amateur"?Email This EntryPrint This Article

Posted by Clay Shirky

A month or so ago, Micah Sifry offered me a chance to respond to Andrew Keen, author of the forthcoming Cult of the Amateur, at a panel at last week’s Personal Democracy Forum (PdF). The book is a polemic against the current expansion of freedom of speech, freedom of the press, and freedom of association. Also on the panel were Craig Newmark and Robert Scoble, so I was in good company; my role would, I thought, be easy — be pro-amateur production, pro-distributed creation, pro-collective action, and so on, things that come naturally to me.

What I did not expect was what happened — I ended up defending Keen, and key points from Cult of the Amateur, against a panel of my peers.

I won’t review CotA here, except to say that the book is going to get a harsh reception from the blogosphere. It is, as Keen himself says, largely anecdotal, which makes it more a list of ‘bad things that have happened where the internet is somewhere in the story’ than an account of cause and effect; as a result, internet gambling and click fraud are lumped together with the problems with DRM and epistemological questions about peer-produced material. In addition to this structural weakness, it is both aggressive enough and reckless enough to make people spitting mad. Dan Gillmor was furious about the inaccuracies, including his erroneous (and since corrected) description in the book, Yochai Benkler asked me why I was even deigning to engage Andrew in conversation, and so on. I don’t think I talked to anyone who wasn’t dismissive of the work.

But even if we stipulate that the book doesn’t do much to separate cause from effect, and has the problems of presentation that often accompany polemic, the core point remains: Keen’s sub-title, “How today’s internet is destroying our culture”, has more than a grain of truth to it, and the only thing those of us who care about the network could do wrong would be to dismiss Keen out of hand.

Which is exactly what people were gearing up to do last week. Because Keen is a master of the dismissive phrase — bloggers are monkeys, only people who get paid do good work, and so on — he will engender a reaction from our side that assumes that everything he says in the book is therefore wrong. This is a bad (but probably inevitable) reaction, but I want to do my bit to try to stave it off, both because fairness dictates it — Keen is at least in part right, and we need to admit that — and because a book-burning accompanied by a hanging-in-effigy will be fun for us, but will weaken the pro-freedom position, not strengthen it.

The Panel

The panel at PdF started with Andrew speaking, in some generality, about ways in which amateurs were discomfiting people who actually know what they are doing, while producing sub-standard work on their own.

My response started by acknowledging that many of the negative effects Keen talked about were real, but that the source of these effect was an increase in the freedom of people to say what they want, when they want to, on a global stage; that the advantages of this freedom outweigh the disadvantages; that many of the disadvantages are localized to professions based on pre-internet inefficiencies; and that the effort required to take expressive power away from citizens was not compatible with a free society.

This was, I thought, a pretty harsh critique of the book. I was wrong; I didn’t know from harsh.

Scoble was simply contemptuous. He had circled offending passages which he would read, and then offer an aphoristic riposte that was more scorn than critique. For instance, in taking on Andrew’s point that talent is unevenly distributed, Scoble’s only comment was, roughly, “Yeah, Britney must be talented…”

Now you know and I know what Scoble meant — traditional media gives outsize rewards to people on characteristics other than pure talent. This is true, but because he was so dismissive of Keen, it’s not the point that Scoble actually got across. Instead, he seemed to be denying either that talent is unevenly distributed, or that Britney is talented.

But Britney is talented. She’s not Yo-Yo Ma, and you don’t have to like her music (back when she made music rather than just headlines), but what she does is hard, and she does it well. Furthermore, deriding the music business’s concern with looks isn’t much of a criticism. It escaped no one’s notice that Amanda Congdon and lonelygirl15 were easy on the eyes, and that that was part of their appeal. So cheap shots at mainstream talent or presumptions of the internet’s high-mindedness are both non-starters.

More importantly, talent is unevenly distributed, and everyone knows it. Indeed, one of the many great things about the net is that talent can now express itself outside traditional frameworks; this extends to blogging, of course, but also to music, as Clive Thompson described in his great NY Times piece, or to software, as with Linus’ talent as an OS developer, and so on. The price of this, however, is that the amount of poorly written or produced material has expanded a million-fold. Increased failure is an inevitable byproduct of increased experimentation, and finding new filtering methods for dealing with an astonishingly adverse signal-to-noise ratio is the great engineering challenge of our age (c.f. Google.) Whatever we think of Keen or CotA, it would be insane to deny that.

Similarly, Scoble scoffed at the idea that there is a war on copyright, but there is a war on copyright, at least as it is currently practiced. As new capabilities go, infinite perfect copyability is a lulu, and it breaks a lot of previously stable systems. In the transition from encoding on atoms to encoding with bits, information goes from having the characteristics of chattel to those of a public good. For the pro-freedom camp to deny that there is a war on copyright puts Keen in the position of truth-teller, and makes us look like employees of the Ministry of Doublespeak.

It will be objected that engaging Keen and discussing a flawed book will give him attention he neither needs nor deserves. This is fantasy. CotA will get an enthusiastic reception no matter what, and whatever we think of it or him, we will be called to account for the issues he raises. This is not right, fair, or just, but it is inevitable, and if we dismiss the book based on its errors or a-causal attributions, we will not be regarded as people who have high standards, but rather as defensive cult members who don’t like to explain ourselves to outsiders.

What We Should Say

Here’s my response to the core of Keen’s argument.

Keen is correct in seeing that the internet is not an improvement to modern society; it is a challenge to it. New technology makes new things possible, or, put another way, when new technology appears, previously impossible things start occurring. If enough of those impossible things are significantly important, and happen in a bundle, quickly, the change becomes a revolution.

The hallmark of revolution is that the goals of the revolutionaries cannot be contained by the institutional structure of the society they live in. As a result, either the revolutionaries are put down, or some of those institutions are transmogrified, replaced, or simply destroyed. We are plainly witnessing a restructuring of the music and newspaper businesses, but their suffering isn’t unique, it’s prophetic. All businesses are media businesses, because whatever else they do, all businesses rely on the managing of information for two audiences — employees and the world. The increase in the power of both individuals and groups, outside traditional organizational structures, is epochal. Many institutions we rely on today will not survive this change without radical alteration.

This change will create three kinds of loss.

First, people whose jobs relied on solving a hard problem will lose those jobs when the hard problems disappear. Creating is hard, filtering is hard, but the basic fact of making acceptable copies of information, previously the basis of the aforementioned music and newspaper industries, is a solved problem, and we should regard with suspicion anyone who tries to return copying to its previously difficult state.

Similarly, Andrew describes a firm running a $50K campaign soliciting user-generated ads, and notes that some professional advertising agency therefore missed out on something like $300,000 dollars of fees. Its possible to regard this as a hardship for the ad guys, but its also possible to wonder whether they were really worth the $300K in the first place if an amateur, working in their spare time with consumer-grade equipment, can create something the client is satisfied with. This loss is real, but it is not general. Video tools are sad for ad guys in the same way movable type was sad for scribes, but as they say in show biz, the world doesn’t owe you a living.

The second kind of loss will come from institutional structures that we like as a society, but which are becoming unsupportable. Online ads offer better value for money, but as a result, they are not going to generate enough cash to stand up the equivalent of the NY Times’ 15-person Baghdad bureau. Josh Wolf has argued that journalistic privilege should be extended to bloggers, but the irony is that Wolf’s very position as a videoblogger makes that view untenable — journalistic privilege is a special exemption to a general requirement for citizens to aid the police. We can’t have a general exception to that case.

The old model of defining a journalist by tying their professional identity to employment by people who own a media outlet is broken. Wolf himself has helped transform journalism from a profession to an activity; now we need a litmus test for when to offer source confidentiality for acts of journalism. This will in some ways be a worse compromise than the one we have now, not least because it will take a long time to unfold, but we can’t have mass amateurization of journalism and keep the social mechanisms that regard journalists as a special minority.

The third kind of loss is the serious kind. Some of these Andrew mentions in his book: the rise of spam, the dramatically enlarged market for identity theft. Other examples he doesn’t: terrorist organizations being more resilient as a result of better communications tools, pro-anorexic girls forming self-help groups to help them remain anorexic. These things are not side-effects of the current increase in freedom, they are effects of that increase. Spam is not just a plague in open, low-entry-cost systems; it is a result of those systems. We can no longer limit things like who gets to form self-help groups through social controls (the church will rent its basement to AA but not to the pro-ana kids), because no one needs help or permission to form such a group anymore.

The hard question contained in Cult of the Amateur is “What are we going to do about the negative effects of freedom?” Our side has generally advocated having as few limits as possible (when we even admit that there are downsides), but we’ve been short on particular cases. It’s easy to tell the newspaper people to quit whining, because the writing has been on the wall since Brad Templeton founded Clarinet. It’s harder to say what we should be doing about the pro-ana kids, or the newly robust terror networks.

Those cases are going to shift us from prevention to reaction (a shift that parallels the current model of publishing first, then filtering later), but so much of the conversation about the social effects of the internet has been so upbeat that even when there is an obvious catastrophe (as with the essjay crisis on Wikipedia), we talk about it amongst ourselves, but not in public.

What Wikipedia (and Digg and eBay and craigslist) have shown us is that mature systems have more controls than immature ones, as the number of bad cases is identified and dealt with, and as these systems become more critical and more populous, the number of bad cases (and therefore the granularity and sophistication of the controls) will continue to increase.

We are creating a governance model for the world that will coalesce after the pre-internet institutions suffer whatever damage or decay they are going to suffer. The conversation about those governance models, what they look like and why we need them, is going to move out into the general public with CotA, and we should be ready for it. My fear, though, is that we will instead get a game of “Did not!”, “Did so!”, and miss the opportunity to say something much more important.

Comments (19) + TrackBacks (0) | Category: social software

May 19, 2007

The (Bayesian) Advantage of YouthEmail This EntryPrint This Article

Posted by Clay Shirky

A couple of weeks ago, Fred Wilson wrote, in The Mid Life Entrepreneur Crisis “…prime time entrepreneurship is 30s. And its possibly getting younger as web technology meets youth culture.” After some followup from Valleywag, he addressed the question at greater length in The Age Question (continued), saying “I don’t totally buy that age matters. I think, as I said in my original post, that age is a mind set.”

This is a relief for people like me — you’re as young as you feel, and all that — or rather it would be a relief but for one little problem: Fred was right before, and he’s wrong now. Young entrepreneurs have an advantage over older ones (and by older I mean over 30), and contra Fred’s second post, age isn’t in fact a mindset. Young people have an advantage that older people don’t have and can’t fake, and it isn’t about vigor or hunger — it’s a mental advantage. The principal asset a young tech entrepreneur has is that they don’t know a lot of things.

In almost every other circumstance, this would be a disadvantage, but not here, and not now. The reason this is so (and the reason smart old people can’t fake their way into this asset) has everything to do with our innate ability to cement past experience into knowledge.

Probability and the Crisis of Novelty

The classic illustration for learning outcomes based on probability uses a bag of colored balls. Imagine that you can take out one ball, record its color, put it back, and draw again. How long does it take you to form an opinion about the contents of the bag, and how correct is that opinion?

Imagine a bag of black and white balls, with a slight majority of white. Drawing out a single ball would provide little information beyond “There is at least one white (or black) ball in this bag.” If you drew out ten balls in a row, you might guess that there are a similar number of black and white balls. A hundred would make you relatively certain of that, and might give you an inkling that white slightly outnumbers black. By a thousand draws, you could put a rough percentage on that imbalance, and by ten thousand draws, you could say something like “53% white to 47% black” with some confidence.

This is the world most of us live in, most of the time; the people with the most experience know the most.

But what would happen if the contents of the bag changed overnight? What if the bag suddenly started yielding balls of all colors and patterns — black and white but also green and blue, striped and spotted? The next day, when the expert draws a striped ball, he might well regard it as a mere anomaly. After all, his considerable experience has revealed a predictable and stable distribution over tens of thousands of draws, so no need to throw out the old theory because of just one anomaly. (To put it in Bayesian terms, the prior beliefs of the expert are valuable precisely because they have been strengthened through repetition, which repetition makes the expert confident in them even in the face of a small number of challenging cases.)

But the expert keeps drawing odd colors, and so after a while, he is forced to throw out the ‘this is an anomaly, and the bag is otherwise as it was’ theory, and start on a new one, which is that some novel variability has indeed entered the system. Now, the expert thinks, we have a world of mostly black and white, but with some new colors as well.

But the expert is still wrong. The bag changed overnight, and the new degree of variation is huge compared to the older black-and-white world. Critically, any attempt to rescue the older theory will cause the expert to misunderstand the world, and the more carefully the expert relies on the very knowledge that constitutes his expertise, the worse his misunderstanding will be.

Meanwhile, on the morning after the contents of the bag turn technicolor, someone who just showed up five minutes ago would say “Hey, this bag has lots of colors and patterns in it.” While the expert is still trying to explain away or minimize the change as a fluke, or as a slight adjustment to an otherwise stable situation, the novice, who has no prior theory to throw out, understands exactly what’s going on.

What our expert should have done, the minute he saw the first odd ball, is to say “I must abandon everything I have ever thought about how this bag works, and start from scratch.” He should, in other words, start behaving like a novice.

Which is exactly the thing he — we — cannot do. We are wired to learn from experience. This is, in almost all cases, absolutely the right strategy, because most things in life benefit from mental continuity. Again, today, gravity pulls things downwards. Again, today, I get hungry and need to eat something in the middle of the day. Again, today, my wife will be happier if I put my socks in the hamper than on the floor. We don’t need to re-learn things like this; once we get the pattern, we can internalize it and move on.

A Lot of Knowledge Is A Dangerous Thing

This is where Fred’s earlier argument comes in. In 999,999 cases, learning from experience is a good idea, but what entrepreneurs do is look for the one in a million shot. When the world really has changed overnight, when wild new things are possible if you don’t have any sense of how things used to be, then it is the people who got here five minutes ago who understand that new possibility, and they understand it precisely because, to them, it isn’t new.

These cases, let it be said, are rare. The mistakes novices make come from a lack of experience. They overestimate mere fads, seeing revolution everywhere, and they make this kind of mistake a thousand times before they learn better. But the experts make the opposite mistake, so that when a real once-in-a-lifetime change comes along, they are at risk of regarding it as a fad. As a result of this asymmetry, the novice makes their one good call during an actual revolution, at exactly the same time the expert makes their one big mistake, but at that moment, that’s all that is needed to give the newcomer a considerable edge.

Here’s a tech history question: Which went mainstream first, the PC or the VCR?

People over 35 have a hard time even understanding why you’d even ask — VCRs obviously pre-date PCs for general adoption.

Here’s another: Which went mainstream first, the radio or the telephone?

The same people often have to think about this question, even though the practical demonstration of radio came almost two decades after the practical demonstration of the telephone. We have to think about that second question because, to us, radio and the telephone arrived at the same time, which is to say the day we were born. And for college students today, that is true of the VCR and the PC.

People who think of the VCR as old and stable, and the PC as a newer invention, are not the kind of people who think up Tivo. It’s people who are presented with two storage choices, tape or disk, without historical bias making tape seem more normal and disk more provisional, who do that kind of work, and those people are, overwhelmingly, young.

This is sad for a lot of us, but its also true, and Fred’s kind lies about age being a mind set won’t reverse that.

The Uses of Experience

I’m old enough to know a lot of things, just from life experience. I know that music comes from stores. I know that you have to try on pants before you buy them. I know that newspapers are where you get your political news and how you look for a job. I know that if you want to have a conversation with someone, you call them on the phone. I know that the library is the most important building on a college campus. I know that if you need to take a trip, you visit a travel agent.

In the last 15 years or so, I’ve had to unlearn every one of those things and a million others. This makes me a not-bad analyst, because I have to explain new technology to myself first — I’m too old to understand it natively. But it makes me a lousy entrepreneur.

Ten years ago, I was the CTO of a web company we built and sold in what seemed like an eon but what was in retrospect an eyeblink. Looking back, I’m embarrassed at how little I knew, but I was a better entrepreneur because of it.

I can take some comfort in the fact that people much more successful than I succumb to the same fate. IBM learned, from decades of experience, that competitive advantage lay in the hardware; Bill Gates had never had those experiences, and didn’t have to unlearn them. Jerry and David at Yahoo learned, after a few short years, that search was a commodity. Sergey and Larry never knew that. Mark Cuban learned that the infrastructure required for online video made the economics of web video look a lot like TV. That memo was never circulated at YouTube.

So what can you do when you get kicked out of the club? My answer has been to do the things older and wiser people do. I teach, I write, I consult, and when I work with startups, it’s as an advisor, not as a founder.

And the hardest discipline, whether talking to my students or the companies I work with, is to hold back from offering too much advice, too definitively. When I see students or startups thinking up something crazy, and I want to explain why that won’t work, couldn’t possibly work, why this recapitulates the very argument that led to RFC 939 back in the day, I have to remind myself to shut up for a minute and just watch, because it may be me who will be surprised when I see what color comes out of the bag next.

Comments (42) + TrackBacks (0) | Category: social software

April 25, 2007

Sorry, Wrong Number: McCloud Abandons MicropaymentsEmail This EntryPrint This Article

Posted by Clay Shirky

Four years ago, I wrote a piece called Fame vs Fortune: Micropayments and Free Content. The piece was sparked by the founding of a company called BitPass and its adoption by the comic artist Scott McCloud (author of the seminal Understanding Comics, among other things.) McCloud created a graphic work called “The Right Number”, which you had to buy using BitPass.

It didn’t work. BitPass went out of business in January of this year. I didn’t write about it at the time because its failure was a foregone conclusion. This isn’t just retrospective certainty, either; here’s what I said about BitPass in 2003:
BitPass will fail, as FirstVirtual, Cybercoin, Millicent, Digicash, Internet Dollar, Pay2See, and many others have in the decade since Digital Silk Road, the paper that helped launch interest in micropayments. These systems didn’t fail because of poor implementation; they failed because the trend towards freely offered content is an epochal change, to which micropayments are a pointless response.

I’d love to take credit for having made a brave prediction there, but in fact Nick Szabo wrote a dispositive critique of micropayments back in 1996. The BitPass model never made a lick of sense, so predicting its demise was mere throat-clearing on the way to the bigger argument. The conclusion I drew in 2003 (and which I still believe) was that the vanishingly low cost of making unlimited perfect copies would put creators in the position of having to decide between going for audience size (fame) or restricting and charging for access (fortune), and that the desire for fame, no longer tempered by reproduction costs, would generally win out.

Creators are not publishers, and putting the power to publish directly into their hands does not make them publishers. It makes them artists with printing presses. This matters because creative people crave attention in a way publishers do not. […] with the power to publish directly in their hands, many creative people face a dilemma they’ve never had before: fame vs fortune.

Scott McCloud, who was also an advisor to BitPass, took strong issue with this idea in Misunderstanding Micropayments, a reply to the Fame vs. Fortune argument:

In many cases, it’s no longer a choice between getting it for a price or getting it for free. It’s the choice between getting it for price or not getting it at all. Fortunately, the price doesn’t have to be high.

McCloud was arguing that the creator’s natural monopoly — only Scott McCloud can produce another Scott McCloud work — would provide the artist the leverage needed to insist on micropayments (true), and that this leverage would create throngs of two-bit users (false).

What’s really interesting is that, after the failure of BitPass, McCloud has now released The Right Number absolutely free of charge. Nothing. Nada. Kein Preis. After the micropayment barrier had proved too high for his potential audience (as predicted), McCloud had to choose between keeping his work obscure, in order to preserve the possibility of charging for it, or going for attention. His actual choice in 2007, upends his argument of four years ago: he went for the fame, at the expense of the fortune. (This recapitulates Tim O’Reilly’s formulation: “Obscurity is a far greater threat to authors and creative artists than piracy.” [ thanks, Cory, for the pointer ])

Everyone who imagines a working micropayment system either misunderstands user preferences, or imagines preventing users from expressing those preferences. The working micropayments systems that people hold up as existence proofs — ringtones, iTunes — are businesses that have escaped from market dynamics through a monopoly or cartel (music labels, carriers, etc.) Indeed, the very appeal of micropayments to content producers (the only people who like them — they offer no feature a user has ever requested) is to re-establish the leverage of the creator over the users. This isn’t going to happen, because the leverage wasn’t based on the valuing of content, but of packaging and distribution.

I’ll let my 2003 self finish the argument:
People want to believe in things like micropayments because without a magic bullet to believe in, they would be left with the uncomfortable conclusion that what seems to be happening — free content is growing in both amount and quality — is what’s actually happening.

The economics of content creation are in fact fairly simple. The two critical questions are “Does the support come from the reader, or from an advertiser, patron, or the creator?” and “Is the support mandatory or voluntary?”

The internet adds no new possibilities. Instead, it simply shifts both answers strongly to the right. It makes all user-supported schemes harder, and all subsidized schemes easier. It likewise makes collecting fees harder, and soliciting donations easier. And these effects are multiplicative. The internet makes collecting mandatory user fees much harder, and makes voluntarily subsidy much easier.

The only interesting footnote, in 2007, is that these forces have now reversed even McCloud’s behavior.

Comments (11) + TrackBacks (0) | Category: social software

March 7, 2007

Spam that knows you: anyone else getting this?Email This EntryPrint This Article

Posted by Clay Shirky

So a few weeks ago, I started getting spam referencing O’Reilly books in the subject line, and I thought that the spammers had just gotten lucky, and that the universe of possible offensive measures for spammers now included generating so many different subject lines that at least some of them got through to my inbox, but recently I’ve started to get more of this kind of spam, as with:

  • Subject: definition of what “free software” means. Outgrowing its
  • Subject: What makes it particularly interesting to private users is that there has been much activity to bring free UNIXoid operating systems to the PC,
  • Subject: and so have been long-haul links using public telephone lines. A rapidly growing conglomerate of world-wide networks has, however, made joining the global

(All are phrases drawn from

Can it be that spammers are starting to associate context with individual email addresses, in an effort to evade Bayesian filters? (If you wanted to make sure a message got to my inbox, references to free software, open source, and telecom networks would be a pretty good way to do it. I mean, what are the chances?) Some of this stuff is so close to my interests that I thought I’d written some of the subject lines and was receiving this as a reply. Or is this just general Bayes-busting that happens to overlap with my interests?

If it’s the former, then Teilhard de Chardin is laughing it up in some odd corner of the noosphere, as our public expressions are being reflected back to us as a come-on. History repeats itself, first as self-expression, then as ad copy…

Comments (15) + TrackBacks (0) | Category: social software

February 6, 2007

Second Life: A response to Henry JenkinsEmail This EntryPrint This Article

Posted by Clay Shirky

Introduction: Last week, Henry Jenkins, Beth Coleman and I all published pieces on Second Life and virtual worlds. (Those pieces are here: Henry, Beth, Clay.)

We also agreed we would each post reaction pieces this week. Henry’s second week post is here, Beth’s is here. (I have not read either yet, so the reactions to those pieces will come next week.) My reaction to Henry’s first piece is below; my reaction to Beth’s first piece will appear later in the week.


I hope you’re a betting man, because at the end of this post, I’m going to propose a bet (or rather, as befits a conversation between academics, a framework for a bet.)

Before I do, though, I want to react to your earlier post on Second Life. Reading it, it struck me that we agree about many of the basic facts, and that most of our variance is about their relative importance. So as to prevent the softness of false consensus from settling over some sharp but interesting disagreements, let me start with a list of assertions I think we could both agree with. If I succeed, we can concentrate on our smaller but more interesting set of differences.

I think you and I agree that:

1. Linden has embraced participatory culture, including, inter alia, providing user tools, using CC licenses, and open sourcing the client.
2. Users of Second Life have created interesting effects by taking advantage of those opportunities.

and also

3. Most people who try Second Life do not like it. As a result, SL is is not going to be a mass movement in any meaningful sense of the term, to use your phrase.
4. Reporters and marketers ought not discuss Second Life using phony numbers.

The core difference between our respective views of the current situation is that you place more emphasis on the first two items on that list, and I on the second two.

With that having been said (and assuming you roughly agree with that analysis), I’ll respond to three points in your post that I either don’t understand or do understand but disagree with. First, I want to push back on one of your historical comparisons. Next, I want to try to convince you that giving bad actors a pass when they embrace participatory culture is short-sighted. Finally, and most importantly, I want to propose a bet on the future utility or inutility of virtual worlds.

One: Second Life != The Renaissance

You compare Second Life with the Renaissance and the Age of Reason. This is approximately insane, and your disclaimer that Second Life may not reach this rarefied plateau doesn’t do much to make it less insane. Using the Renaissance as a reference point links the two in the reader’s mind, even in the face of subsequent denial. (We are all Lakoffians now.)

I disagree with this comparison, not because I think its wrong, but because I think it’s a category error. Your example assumes I was writing about Second Life as shorthand for broader cultural change. I wasn’t. I was writing about Second Life as a client/server program that creates a visual rendering of 3-dimensional space. Not only is it not the Renaissance, it isn’t even the same kind of thing as the Renaissance. Your comparison elides the differences between general movements and particular tools.

Participatory culture is one of the essential movements of our age. It creates many different kinds of artifacts, however, and it is possible to be skeptical about Second Life as an artifact without being skeptical of participatory culture generally. Let me re-write the sentiment you reacted to, to make that distinction clear: Second Life, a piece of software developed by Linden Labs, is unlikely to become widely adopted in the future, because it is not now and has never been widely adopted, measured either in retention of new users or in the number of current return users.

If you believe, separate from the fortunes of Linden Lab’s current offering, that virtual worlds will someday become popular, we have a different, and far more interesting disagreement, one I’ll write about in the last section below.

Two: Participatoriness Should Not Be A ‘Get Out of Jail Free’ Card

You say:
I care only a little bit about the future of virtual worlds. I care a great deal about the future of participatory culture. And for the moment, the debate about and the hype surrounding SL is keeping alive the idea that we might design and inhabit our own worlds and construct our own culture. That’s something worth defending.

Reading this, I get the feeling that if I opened Clay’s Kitten Mulching Hut™, so long as it involved participatory kitten mulching, it would get the Jenkins seal of approval.

Of everything you wrote, I think this is likeliest to turn out to be a tactical mistake, for two reasons. First, for every Second Life user, there are at least 5 former users who quickly bailed. (And that’s in the short term; more have bailed the long term. We don’t know how many current return users there are, except that the number is smaller than any published figure.) Giving a pass to laudatory Second Life stories that use false numbers, simply because they are “keeping alive” an idea you like, risks bootstrapping Second Life’s failure to retain users into unwarranted skepticism about peer production generally.

More importantly, though, lowering your scrutiny of people using bogus Linden numbers, just because they are on your team, is a bad idea. To put this in Brooklynese, everyone touting the popularity of Second Life is either a schlemiel or a schuyster. The schlemiels are simply people who were fooled by Linden. The schuysters, though, are people who know the numbers are junk but use them anyway; making participatory culture a Get Out of Jail Free card for that kind of deceit is an invitation to corruption.

And I mean corruption in the most literal sense of the word:

- “As I write this, there are just over 1.4 million residents in Second Life, with over $704,000 spent in the last 24 hours.” Ian Shafer, at Clikz Experts

- “…one million users and growing by hundreds of thousands of users each month, that may well care about that virtual fashion crisis. They’re members of the virtual world Second Life, and when you consider all the places you need to reach consumers, this is by far one of the strangest.”David Berkowitz of 360i.

- “A new market emerging from virtual reality into the real-world global economy is growing by leaps and bounds […] such as the hugely popular Second Life - where the lines between online and offline commerce are rapidly blurring. Rodney Nelsestuen of TowerGroup.

- “At the rate Second Life is growing — from 100,000 registered users a year ago to one million in October, and now all the way up to two million — it may be over thirty million a year from now. At thirty million users Second Life is no longer a sideshow, but is something everyone has heard of and many people are experiencing for themselves.” Catherine Winters at SocialSignal

Shafer, Berkowitz, Nelsestuen and Winters all know that the figures they are touting are inaccurate, if inaccurate is a strong enough word for claims that would make a stock spammer blush. (30 million users at the end of 2007? Please. Linden will be lucky to have a million return users in any month of this year.)

And why are these people saying these things? Because they are selling their services, and they are likelier to get clients if people wrongly believe that Second Life is hugely popular, or that millions of people currently use it, or that that tens of millions will use it in a year. If, on the other hand, those potential clients were to understand that both the size and growth of the active population was considerably more modest than the Hamburger Helper figures Linden publishes, well that wouldn’t be as good for business.

Perhaps, so long as it is about user-generated experiences, this sort of lying doesn’t bug you. It bugs me. More to the point, I think it should bug you, if only for reasons of self-interest; accepting Second Life hyperbole now, on the grounds that people will later remember the participatory nature but not the falsified popularity, is whistling past a pretty big graveyard.

Three: ‘Virtual Worlds’ is a failed composite

Aside from historical comparisons and concerns about abuse of population numbers, there is one area where I think we understand one another, and come to opposite conclusions. Even after we’ve agreed that Second Life will not become a mass movement, and that 3D worlds will not replace existing forms of communication, there’s still this:

Most of us will find uses for virtual worlds one of these days; most of us will not “live” there nor will we conduct most of our business there.

(When you say ‘most of us will find uses for virtual worlds…one of these days’, I assume you mean that something like ‘half of users will use virtual worlds at some future date amenable to prediction’, something like half a dozen years, say.)

We both seem to have written off any short-term realization of a Snow Crash-ish general purpose, visually immersive social world (though probably for different reasons.) Here’s my read of your theory of virtual worlds — you tell me where I’m wrong:

1. The case of immersive 3D representation creates enough similarity among various platforms that they can be reasonably analyzed as a group.
2. Though we’re not headed to a wholesale replacement of current tools, or a general purpose “we live there” cyberspace, the number of special cases for virtual worlds will continue to grow, until most of us will use them someday.

If I have this right, or right enough, then our disagreement seems potentially crisp enough to make a bet on. Let me lay out where I think we disagree most clearly:

First, I think the label ‘virtual worlds’ makes no sense as a category. (I’ve already covered why elsewhere, so I won’t repeat that here.)

Second, I think that 3D game worlds will keep going gangbusters, and 3D non-game worlds will keep not. (An argument covered in the same place.)

Third, even with games driving almost all uses of 3D immersive worlds, I do not believe the portmanteau category of virtual worlds will reach anything like half of internet users (or even half of broadband users) in any predictable time.

#1 is just a thesis statement, but #2 and #3 are potential bets. So, if you’d care to make a prediction about either the percentage of game to non-game uses of virtual worlds, or overall population figures in virtual worlds, expressed either as an absolute number or as a percentage, I think we could take opposite side of a bet for, say, dinner anyplace in either New York or Boston, set for the year of your prediction. (If the date is too far out, we can also pro-rate it via compound average growth, to make the term of the bet shorter.)

Game? Lemme know.

Comments (21) + TrackBacks (0) | Category:

January 29, 2007

Second Life, Games, and Virtual WorldsEmail This EntryPrint This Article

Posted by Clay Shirky

Introduction: This post is an experiment in synchronization. Since Henry Jenkins, Beth Coleman, and I are all writing about Second Life and because we like each other’s work, even when (or especially when) we disagree, we’ve decided to all post something on Second Life today. Beth’s post will appear at, and Henry’s is at

Let me start with some background. Because of the number of themes involved in discussions of Second Life, it’s easy to end up talking at different levels of abstraction, so let me start with two core assertions, things that I take as background to my part of the larger conversation:

  • First, Linden’s Residents figures are methodologically worthless. Any claim about Second Life derived from a count of Residents is not to be taken seriously, and anyone making claims about Second Life based on those figures is to be regarded with skepticism. (Explanation here and here.)
  • Second, there are many interesting things going on in Second Life. As I have said in other forums, and will repeat here, passionate users are a law unto themselves, and rightly so. Nothing I could say about their experience in Second Life, pro or con, would matter to those users. My concerns are demographic.

With those assertions covered, I am asking myself two things: will Second Life become a platform for a significant online population? And, second, what can Second Life tell us about the future of virtual worlds generally?

Concerning popularity, I predict that Second Life will remain a niche application, which is to say an application that will be of considerable interest to a small percentage of the people who try it. Such niches can be profitable (an argument I made in the Meganiche article), but they won’t, by definition, appeal to a broad cross-section of users.

The logic behind this belief is simple: most people who try Second Life don’t like it. Something like five out of six new users abandon it before a month is up. The three month abandonment figure seems to be closer to nine out of ten. (This figure is less firm, as it has only been reported colloquially, with no absolute numbers behind it.)

More importantly, the current active population is still an unknown. (Call this metric something like “How many users in the last 30 days have accounts more than 30 days old?”) We know the highest that figure could be is in the low hundreds of thousands, but no one other than the Lindens (and, presumably, their bigger marketing clients) knows how much lower it is than this theoretical maximum.

The poor adoption rate is a form of aggregate judgment. Anything bruited for wide adoption would have trouble with 85%+ abandonment, whether software or toothpaste. One possible explanation for this considerable user defection might be a technological gap. I do not doubt that improvements to the client and server would decrease the abandonment rate. I do doubt the improvement would be anything other than incremental, given 5 years and tens of millions in effort already.

Note too that abandonment is not a problem that all visually traversable spaces suffer from. Both Doom and Cyworld serve as counter-examples; in those cases, the rendering is cartoonish, yet both platforms achieved huge popularity in a short period. If the non-visual experience is good, the rendering does not need to be, but the converse does not seem to be true, on present evidence.

Two Objections

There have been two broad responses to skepticism occasioned by the Linden population numbers. (Three, if you count ad hominem, but Chris Lott has already covered that.)

The first response is not specific to Second Life. Many people have recalled earlier instances of misguided skepticism about new technologies, but the logical end-case of that thought is that skepticism about technology is never appropriate. (Disconfirmation of this thesis is left as an exercise for the reader.) Given that most new technologies fail, the challenge is to figure out which ones won’t. No one has noted examples of software with 85% abandonment rates, after five years of development, that went on to become widespread. Such examples may exist, but I can’t think of any.

The second objection is a conviction that demographics are irrelevant, and that the interesting goings-on in Second Life are what matters, no matter how few users are engaged in those activities.

I have never doubted (and have explicitly noted above) that there are interesting things happening in Second Life. The mistake, from my point of view, is in mixing two different questions. Whether some people like Second Life a lot is a completely separate issue from whether a lot of people like it. It is possible for the first assertion to be true and the second one false, and this is the only reading I believe is supported by the low absolute numbers and high abandonment rates. Nor is this an unusual case. We have several examples of platforms with fascinating in-world effects (Alphaworld, Black Sun/Blaxxun, The Palace, Dreamscape, LambdaMOO and environments on the SuperMOO List, etc.), all of which also failed to achieve wide use.

It is here that assertions about Second Life have most often been inconsistent. Before the uselessness of Linden’s population numbers was widely understood, the illusion of a large and rapidly growing community was touted as evidence of Second Life’s success. When both the absolute numbers and growth turned out to be more modest, population was downgraded and other metrics have been introduced as predictive of Second Life’s inevitable success.

A hypothesis which is strengthened by evidence of popularity, but not weakened by evidence of unpopularity, isn’t really a hypothesis, it’s a religious assertion. And a core tenet of the faithful seems to be that claims about Second Life are buttressed by the certain and proximate arrival of virtual worlds generally.

If we had but worlds enough and time…

It is worth pausing at this junction. Many people writing about Second Life make little distinction between ‘Second Life as a particular platform’ and ‘Second Life as an exemplar of the coming metaverse’. I would like to buck this trend, by explicitly noting the difference between those two conversations. I am basing my prediction of continued niche status for Second Life on the current evidence that most people who try it don’t like it. My beliefs about virtual worlds, on the other hand, are more conjectural. Everything below should be read with this caveat in mind.

With that said, I don’t believe that “virtual worlds” describes a coherent category, or, put another way, I believe that the group of things lumped together as virtual worlds have such variable implementations and user adoption rates that they are not well described as a single conceptual group.

I alluded to Pointcast in an earlier article; one of the ways the comparison is apt is in the abuse of categorization as a PR tool. Pointcast’s management claimed that email, the Web, and Pointcast all were about delivering content, and that the future looked bright for content delivery platforms. And indeed it did, except for Pointcast.

The successes of email and of the Web were better explained by their particular utilities than by their membership in a broad class of “content delivery.” Pointcast tried to shift attention from those particularities to a generic label in order to create a club in which it would automatically be included.

I believe a similar thing happens whenever Second Life is lumped with Everquest, World of Warcraft, et al., into a category called virtual worlds. If we accept the validity of this category, then multi-player games provide an existence proof of millions-strong virtual worlds, and the only remaining question is simply when we arrive at wider adoption of more general-purpose versions.

If, on the other hand, we don’t start off by lumping Second Life with Warcraft as virtual worlds, a very different question emerges: why do virtual game worlds outperform non-game worlds in their adoption? This pattern is quite stable over time — it well predates Second Live and World of Warcraft, as with first Ultima Online (1997) and then Everquest (1999) each quickly dwarfing the combined populations of Alphaworld and Black Sun (later Blaxxun) despite the significant lead times of those virtual worlds. What is it about games that would make them a better fit for virtual environments than non-games?

Games have at least three advantages other virtual worlds don’t. First, many games, and most social games, involve an entrance into what theorists call the magic circle, an environment whose characteristics include simplified and knowable rules. The magic circle saves the game from having to live up to expectations carried over from the real world.

Second, games are intentionally difficult. If all you knew about golf was that you had to get this ball in that hole, your first thought would be to hop in your cart and drive it over there. But no, you have to knock the ball in, with special sticks. This is just about the stupidest possible way to complete the task, and also the only thing that makes golf interesting. Games create an environment conducive to the acceptance of artificial difficulties.

Finally, and most relevant to visual environments, our ability to ignore information from the visual field when in pursuit of an immediate goal is nothing short of astonishing (viz. the gorilla experiment.) The fact that we could clearly understand spatial layout even in early and poorly rendered 3D environments like Quake has much to do with our willingness to switch from an observational Architectural Digest mode of seeing (Why has this hallway been accessorized with lava?) to a task-oriented Guns and Ammo mode (Ogre! Quad rocket for you!)

In this telling, games are not just special, they are special in a way that relieves designers of the pursuit of maximal realism. There is still a premium on good design and playability, but the magic circle, acceptance of arbitrary difficulties, and goal-directed visual filtering give designers ways to contextualize or bury at least some platform limitations. These are not options available to designers of non-game environments; asking users to accept such worlds as even passable simulacra subjects those environments to withering scrutiny.

Hubba Hubba

We can also reverse this observation. One question we might ask about successful non-game uses of virtual worlds is whether they too are special cases. One obvious example is erotic imagery. The zaftig avatar has been a trope of 3D rendering since designers have been able to scrape together enough polygons to model a torso, but examples start far earlier than virtual worlds. In fact, visual representation of voluptuous womanhood predates the invention of agriculture by the same historical interval as agriculture predates the present. This is a deep pattern.

It is also a pattern that, like games and unlike ordinary life, has a special relation to visual cues (though this effect is somewhat unbalanced by gender.) If someone is shown a virtual hamburger, it can arouse real hunger. However, to satisfy this hunger, he must then walk away from the image and get his hands on an actual hamburger. This is not the case, to put the matter delicately, with erotic imagery; a fetching avatar can arouse desire, but that desire can then be satiated without recourse to the real.

This pair of characteristics — a human (and particularly male) fixation on even poorly rendered erotic images, plus an ability to achieve a kind of gratification in the presence of those images — means that a sexualized rendering can create both attraction and satisfaction in a way that a rendering of, say, a mountain or an office cannot. As with games, visual worlds work in the context of eros not because the images themselves are so convincing, but because they reach a part of the brain that so desperately wants to be convinced.

More generally, I suspect that the cases where 3D immersion works are, and will continue to be, those uses that most invite the mind to fill in or simply do without missing detail, whether because of a triggering of sexual desire, the fight or flight reflex (many games), avarice (gambling), or other areas where we are willing and even eager to make rapid inferences based on a paucity of data. I also assume that these special cases are not simply adding up to a general acceptance of visual immersion, and that finding another avatar beguiling in a virtual bar is not in fact a predictor of being able to read someone’s face or body language in a virtual meeting as if you were with them. That, I believe, is a neurological problem of a different order.

Jaron Lanier is the Charles Babbage of Our Generation

Here we arrive at the furthest shores of speculation. One of the basic promises of virtual reality, at least in its Snow Crash-inflected version, is that we will be able to re-create the full sense of being in someone’s presence in a mediated environment. This desire, present at least since Shamash appeared to Gilgamesh in a dream, can be re-stated in technological terms as a hope that communications will finally become an adequate substitute for travel. We have been promised that this will come to pass with current technology since ATT demoed a video phone at the 1964 World’s Fair.

I believe this version of virtual reality will in fact be achieved, someday. I do not, however, believe that it will involve a screen. Trying to trick the brain by tricking the eyes is a mug’s game. The brain is richly arrayed with tools to detect and unmask visual trickery — if the eyes are misreporting, the brain falls back on other externally focussed senses like touch and smell, or internally focussed ones like balance and proprioception.

Though the conception of virtual reality is clear, the technologies we have today are inadequate to the task. In the same way that the theory of computation arose in the mechanical age, but had to wait first for electrics and then electronics to be fully realized, general purpose virtual reality is an idea waiting on a technology, and specifically on neural interface, which will allow us to trick the brain by tricking the brain. (The neural interface in turn waits on trifling details like an explanation of consciousness.)

In the meantime, the 3D worlds program in the next decade is likely to resemble the AI program in the last century, where early optimism about rapid progress on general frameworks gave way to disconnected research topics (machine vision, natural language processing) and ‘toy worlds’ environments. We will continue to see valuable but specific uses for immersive environments, from flight training and architectural flythroughs to pain relief for burn victims and treatment for acrophobia. These are all indisputably good things, but they are not themselves general, and more importantly don’t suggest rapid progress on generality. As a result, games will continue to dominate the list of well-populated environments for the foreseeable future, rendering ineffectual the category of virtual worlds, and, critically, many of the predictions being attached thereunto.

[We’ve been experiencing continuing problems with our MT-powered commenting system. We’re working on a fix but for now send you to a temporary page where the discussion can continue.]

Comments (0) + TrackBacks (0) | Category: social software

Against Well-designed Reputation Systems (An Argument for Community Patent)Email This EntryPrint This Article

Posted by Clay Shirky

Intro: I was part of a group of people asked by Beth Noveck to advise the Community Patent review project about the design of a reputation and ranking system, to allow the widest possible input while keeping system gaming to a minimum. This was my reply, edited slightly for posting here.

We’ve all gone to school on the moderation and reputation systems of Slashdot and eBay. In those cases, their growing popularity in the period after their respective launches led to a tragedy of the commons, where open access plus incentives led to nearly constant attack by people wanting to game the system, whether to gain attention for themselves or their point of view in the case of Slashdot, or to defraud other users, as with eBay.

The traditional response to these problems would have been to hire editors or other functionaries to police the system for abuse, in order to stem the damage and to assure ordinary users you were working on their behalf. That strategy, however, would fail at the scale and degree of openness at which those services function. The Slashdot FAQ tells the story of trying to police the comments with moderators chosen from among the userbase, first 25 of them and later 400. Like the Charge of the Light Brigade, however, even hundreds of committed individuals were just cannon fodder, given the size of the problem. The very presence of effective moderators made the problem worse over time. In a process analogous to more roads creating more traffic, the improved moderation saved the site from drowning in noise, so more users joined, but this increase actually made policing the site harder, eventually breaking the very system that made the growth possible in the first place.

EBay faced similar, ugly feedback loops; any linear expenditure of energy required for policing, however small the increment, would ultimately make the service unsustainable. As a result, the only opportunity for low-cost policing of such systems is to make them largely self-policing. From these examples and others we can surmise that large social systems will need ways to highlight good behavior or suppress negative behavior or both. If the guardians are to guard themselves, oversight must be largely replaced by something we might call intrasight, designed in such a way that imbalances become self-correcting.

The obvious conclusion to draw is that, when contemplating the a new service with these characteristics, the need for some user-harnessed reputation or ranking system can be regarded as a foregone conclusion, and that these systems should be carefully planned so that tragedy of the commons problems can be avoided from launch. I believe that this conclusion is wrong, and that where it is acted on, its effects are likely to be at least harmful, if not fatal, to the service adopting them.

There is an alternate reading of the Slashdot and eBay stories, one that I believe better describes those successes, and better places Community Patent to take advantage of similar processes. That reading concentrates not on outcome but process; the history of Slashdot’s reputation system should teach us not “End as they began — build your reputation system in advance” but rather “Begin as they began — ship with a simple set of features, watch and learn, and implement reputation and ranking only after you understand the problems you are taking on.” In this telling, constituting users’ relations as a set of bargains developed incrementally and post hoc is more predictive of eventual success than simply adopting any residue from previous successes.

As David Weinberger noted in his talk The Unspoken of Groups, clarity is violence in social settings. You don’t get 1789 without living through 1788; successful constitutions, which necessarily create clarity, are typically ratified only after a group has come to a degree of informal cohesion, and is thus able to absorb some of the violence of clarity, in order to get its benefits. The desire to participate in a system that constrains freedom of action in support of group goals typically requires that the participants have at least seen, and possibly lived through, the difficulties of unfettered systems, while at the same time building up their sense of membership or shared goals in the group as a whole. Otherwise, adoption of a system whose goal is precisely to constrain its participants can seem too onerous to be worthwhile. (Again, contrast the US Constitution with the Articles of Confederation.)

Most current reputation systems have been fit to their situation only after that situation has moved from theoretical to actual; both eBay and Slashdot moved from a high degree of uncertainty to largely stable systems after a period of early experimentation. Perhaps surprisingly, this has not committed them to continual redesign. In those cases, systems designed after launch, but early in the process of user adoption, have survived to this day with only relatively minor subsequent adjustments.

Digg is the important counter-example, the most successful service to date to design a reputation system in advance. Digg differs from the community patent review process in that the designers of Digg had an enormous amount of prior art directly in its domain (Slashdot, Kuro5hin, Metafilter, et al), and still ended up with serious re-design issues. More speculatively, Digg seems to have suffered more from both system gaming and public concern over its methods, possibly because the lack of organic growth of its methods prevented it from becoming legitimized over time in the eyes of its users. Instead, they were asked to take it or leave it (never a choice users have been know to relish.)

Though more reputation design work may become Digg-like over time, in that designers can launch with systems more complete than eBay or Slashdot did, the ability to survey significantly similar prior art, and the ability to adopt a fairly high-handed attitude towards users who dislike the service, are not luxuries the community patent review process currently enjoys.

The Argument in Two Pictures

The argument I’m advancing can be illustrated with two imaginary graphs. The first concerns plasticity, the ease with which any piece of software can be modified.

Plasticity generally decays with time. It is highest at the in the early parts of the design phase, when a project is in its most formative stages. It is easier to change a list of potential features than a set of partially implemented features, and it is easier to change partially implemented features than fully implemented features. Especially significant is the drop in plasticity at launch; even for web-based services, which exist only in a single instantiation and can be updated frequently and for all users at once, the addition of users creates both inertia, in the direction of not breaking their mental model of the service, and caution in upgrading, so as not to introduce bugs or create downtime in a working service. As the userbase grows, the expectations of the early adopters harden still further, while the expectations of new users follows the norms set up by those adopters; this is particularly true of any service with a social component.

An obvious concern with reputation systems is that, as with any feature, they are easier to implement when plasticity is high. Other things being equal, one would prefer to design the system as early as possible, and certainly before launch. In the current case, however, other things are not equal. In particular, the specificity of information the designers have about the service and how it behaves in the hands of real users moves counter to plasticity over time.

When you are working to understand the ideal design for a particular piece of software, the specificity of your knowledge increases with time. During the design phase, the increasing concreteness of the work provides concomitant gains in specificity, but nothing like launch. No software, however perfect, survives first contact with the users unscathed, and given the unparalleled opportunities with web-based services to observe user behavior — individually and in bulk, in the moment and over time — the period after launch increases specificity enormously, after which it continues to rise, albeit at a less torrid pace.

There is a tension between knowing and doing; in the absence of the ideal scenario where you know just what needs to be done while enjoying complete freedom to do it (and a pony), the essential tradeoff is in understanding which features benefit most from increased specificity of knowledge. Two characteristics that will tend to push the ideal implementation window to post-launch are when a set of possible features is very large, but the set of those features that will ultimately be required is small; and when culling the small number of required features from the set of all possible features can only be done by observing actual users. I believe that both conditions apply a fortiori to reputation and ranking.

Costs of Acting In Advance of Knowing

Consider the costs of designing a reputation system in advance. In addition to the well-known problems of feature-creep (“Let’s make it possible to rank reputation rankings!”) and Theory of Everything technologies (“Let’s make it Semantic Web-compliant!”), reputation systems create an astonishing perimeter defense problem. The number of possible threats you can imagine in advance is typically much larger than the number that manifest themselves in functioning communities. Even worse, however large the list of imagined threats, it will not be complete. Social systems are degenerate, which is to say that there are multiple alternate paths to similar goals — someone who wants to act out and is thwarted along one path can readily find others.

As you will not know which of these ills you will face, the perimeter you will end up defending will be very large and, critically, hard to maintain. The likeliest outcome from such an a priori design effort is inertness; a system designed in advance to prevent all negative behavior will typically have as a side effect deflecting almost all behavior, period, as users simply turn away from adoption.

Working social systems are both complex and homeostatic; as a result, any given strategy for mediating social relations can only be analyzed in the context of the other strategies in use, including strategies adopted by the users themselves. Since the user strategies cannot, by definition, be perfectly predicted in advance, and since the only ungameable social system is the one that doesn’t ship, every social system will have some weakness. A system designed in advance is likely to be overdefended while still having a serious weaknesses unknown the designer, because the discovery and exploitation of that class of weakness can only occur in working, which is to say user-populated, systems. (As with many observations about the design of social systems, these are precedents first illustrated in Lessons from Lucasfilm’s Habitat, in the sections “Don’t Trust Anybody” and “Detailed Central Planning Is Impossible, Don’t Even Try”.)

The worst outcome of such a system would be collapse (the Communitree scenario), but even the best outcome would still require post hoc design to fix the system with regard to observed user behavior. You could save effort while improving the possibility of success by letting yourself not know what you don’t know, and then learning as you go.

In Favor of Instrumentation Plus Attention

The N-squared problem is only a problem when N is large; in most social systems the users are the most important N, and the userbase only grows large gradually, even for successful systems. (Indeed, this scaling up only over time typically provides the ability for a core group, once they have self-identified, to inculcate new users a bit at a time, using moral suasion as their principal tool.) As a result, in the early days of a system, the designers occupy a valuable point of transition, after user behavior is observable, but before scale and culture defeat significant intervention.

To take advantage of this designable moment, I believe that what Community Patent needs, at launch, is only this: metadata, instrumentation, and attention.

Metadata: There are, I believe, three primitive types of metadata required for Community Patent — people, patents, and interjections. Each of these will need some namespace to exist in — identity for the people, and named data for the patents themselves and for various forms of interjection, from simple annotation to complex conversation. In addition, two abstract types are needed — links and labels. A link is any unique pair of primitives — this user made that comment, this comment is attached to that conversation, this conversation is about those patents. All links should be readily observable and extractable from the system, even if they are not exposed in the interface the user sees. Finally, following Schachter’s intuition from, all links should be labelable. (Another way to view the same problem is to see labels as another type of interjection, attached to links.) I believe that this will be enough, at launch, to maximize the specificity of observation while minimizing the loss of plasticity.

Instrumentation: As we know from collaborative filtering algorithms from Ringo to PageRank, it is not necessary to ask users to rank things in order to derive their rankings. The second necessary element will be the automated delivery of as many possible reports to the system designers as can be productively imagined, and, at least as essential, a good system for quickly running ad hoc queries, and automating their production should they prove fruitful. This will help identify both the kinds of productive interactions on the site that need to be defended and the kinds of unproductive interactions they need to be defended from.

Designer Attention: This is the key — it will be far better to invest in smart people watching the social aspects of the system at launch than in smart algorithms guiding those aspects. If we imagine the moment when the system has grown to an average of 10 unique examiners per patent and 10 comments per examiner, then a system with even a thousand patents will be relatively observable without complex ranking or reputation systems, as both the users and the comments will almost certainly exhibit power-law distributions. In a system with as few as ten thousand users and a hundred thousand comments, it will still be fairly apparent where the action is, allowing you the time between Patent #1 and Patent #1000 to work out what sorts of reputation and ranking systems need to be put in place.

This is a simplification, of course, as each of the categories listed above presents its own challenges — how should people record their identity? What’s the right balance between closed and open lists of labels? And so on. I do not mean to minimize those challenges. I do however mean to say that the central design challenge of user governance — self-correcting systems that do not raise crushing participation burdens on the users or crushing policing barriers on the hosts — are so hard to design in advance that, provided you have the system primitives right, the Boyd Strategy of OODA — Orient, Observe, Decide, Act — will be superior to any amount of advance design work.

[We’ve been experiencing continuing problems with our MT-powered commenting system. We’re working on a fix but for now send you to a temporary page where the discussion can continue.]

Comments (0) + TrackBacks (0) | Category: social software

January 4, 2007

Real Second Life numbers, thanks to David KirkpatrickEmail This EntryPrint This Article

Posted by Clay Shirky

I’ve been complaining about bad reporting of Second Life population for some time now. David Kirkpatrick at Fortune has finally gotten some signal out of Linden Labs. Kirkpatrick’s report is here, in the comments. ( comments don’t have permalinks, so scroll down.)

Here are the numbers Philip Rosedale of Linden gave him. These are, I presume, as of Jan 3:

  • 1,525,670 unique people have logged into SL at least once (so now we know: Residents is seeing something a bit over 50% inflation over users.)
  • Of that number, 252,284 people have logged in more than 30 days after their account creation date.
  • Monthly growth in that figure, calculated as the change between last September and last October, was 23%.

    Those of us who wanted the conversation to be grounded in real numbers owe Kirkpatrick our thanks for helping us get there.

    These numbers should have two good effects. First, now that Linden has reported, and Kirkpatrick has published, the real figures, maybe we’ll see the press shift to reporting users and active users, instead of Residents.

    Second, we’re no longer going to be asked to stomach absurd claims of size and growth. The ‘2.3 million user/77% growth in two months’ figures would have meant 70 million Second Life users this time next year. 250 thousand and 23% growth will mean 3 million in a year’s time, a healthy number, but not hyperbolic growth.

    We can start asking more sophisticated questions now, like the use pattern of active users, or the change in monthly growth rates, or whether the Residents-users inflation rate is stable, but those questions are for later. Right now, we’ve got enough real numbers to think about for a while.

  • Comments (9) + TrackBacks (0) | Category: social software

    January 3, 2007

    The future of television and the media triathlonEmail This EntryPrint This Article

    Posted by Clay Shirky

    Mark Cuban doesn’t understand television. He holds a belief, common to connoisseurs the world over, that quality trumps everything else. The current object of his faith in Qualität Über Alles is HDTV. Says Cuban:
    HDTV is the Internet video killer. Deal with it. Internet bandwidth to the home places a cap on the quality and simplicity of video delivery to the home, and to HDTVs in particular. Not only does internet capacity create an issue, but the complexity of moving HDTV streams around the home and to the HDTV is pretty much a deal killer itself.

    “HDTV is the Internet video killer.” Th appeal of this argument — whoever provides the highest quality controls the market — is obvious. So obvious, in fact, that it’s been used before. By audiophiles.

    As January 1, 2000 approaches, and the MP3 whirlpool continues to swirl, one simple fact has made me feel as if I’m stuck at the starting line of the entire download controversy: The sound quality of MP3 has yet to improve above that of the average radio broadcast. Until that changes, I’m merely curious—as opposed to being in the I-want-to-know-it-all-now frenzy that is my usual m.o. when to comes to anything that promises music you can’t get anywhere else. Robert Baird, October, 1999

    MP3s won’t catch on, because they are lower quality than CDs. And this was true, wasn’t it? People cared about audio quality so much that despite other advantages of MP3s (price, shareability, better integration with PCs), they’ve stayed true to the CD all these years. The commercial firms that make CDs, and therefore continue to control the music market, thank these customers daily for their loyalty.

    Meanwhile,back in the real world of the recording business, the news isn’t so rosy

    Cuban doesn’t understand that television has been cut in half. The idea that there should be a formal link between the tele- part and the vision part has ended. Now, and from now on, the form of a video can be handled separately from it’s method of delivery. And since they can be handled separately, they will be, because users prefer it that way.

    But Cuban goes further. He doesn’t just believe that, other things being equal, quality will win; he believes quality is so important to consumers that they will accept enormous inconvenience to get that higher-quality playback. When Cuban’s list of advantages of HDTV includes an inability to watch your own video on it (“the complexity of moving HDTV streams around the home and to the HDTV”), you have to wonder what he thinks a disadvantage would look like.

    This is the season of the HDTV gotcha. After Christmas, people are starting to understand that they didn’t buy a nicer TV, they bought only one part of a Total Controlled Content Delivery Package. Got an HDTV monitor and a new computer for Christmas? You might as well have gotten a Fabergé Egg and a framing hammer for all the useful ways you can combine the two presents.

    Media is a triathlon event. People like to watch, but they also like to create, and to share. Doubling down on the watching part while making it harder for the users to play their own stuff or share with their friends makes a medium worse in the users eyes. By contrast, the last 50 years have been terrible for user creativity and for sharing, so even moderate improvements in either of those abilities make the public go wild.

    When it comes to media quality, people don’t optimize, they satisfice. Once the medium, whether audio or video or whatever, crosses a minimum threshold, users accept it and move on to caring about other attributes. The change in internet video quality from 1996 to 2006 was the big jump, and YouTube is the proof. After this, firms that offer higher social value for video will have an edge over firms that offer higher production values while reducing social value.

    And because the audience for internet video will grow much faster than the audience for HDTV (and will be less pissed, because YouTube doesn’t rely on a ‘bait and switch’ walled garden play) the premium for making internet video better will grow with it. As Richard Gabriel said of programming languages years ago “[E]ven though Lisp compilers in 1987 were about as good as C compilers, there are many more compiler experts who want to make C compilers better than want to make Lisp compilers better.” That’s where video is today. HDTV provides a better viewing experience than internet video, but many more people care about making internet video better than making HDTV better.

    YouTube is the HDTV killer. Deal with it.

    Comments (15) + TrackBacks (0) | Category: social software

    December 26, 2006

    Linden's Second Life numbers and the press's desire to believeEmail This EntryPrint This Article

    Posted by Clay Shirky

    “Here at, we call our website our Kingdom, and any time our webservers serve up a copy of the home page, we record that as a Loyal Subject. We’re very pleased to announce that in the last two months, we have added over 1 million Loyal Subjects to our Kingdom.”

    Put that baldly, you wouldn’t fall for this bit of re-direction, and yet that is exactly what Linden Labs has pulled off with its Residents™ label. By adopting a term that seems like a simple re-branding of “users”, but which is actually unconnected to head count or adoption, they’ve managed to report what the press wants to hear, while providing no actual information.

    If you like your magic tricks to stay mysterious, leave now, but if you want to understand how Linden has managed to disable the fact-checking apparatus of much of the US business press, turning them into a zombie army of unpaid flacks, read on. (And, as with the earlier piece on Linden, this piece has also been published on Valleywag.)

    The basic trick is to make it hard to remember that Linden’s definition of Resident has nothing to do with the plain meaning of the word resident. My dictionary says a resident is a person who lives somewhere permanently or on a long term basis. Linden’s definition of Residents, however, has nothing to do with users at all — it measures signups for an avatar. (Get it? The avatar, not the user, is the resident of Second Life.)

    The obvious costume-party assumption is that there is one avatar per person, but that’s wrong. There can be more than one avatar per account, and more than one account per person, and there’s no public explanation of which of those units Residents measures, and thus no way to tell anything about how many actual people use Second Life. (An embarrassingly First Life concern, I know.)

    Confused yet? Wait, there’s less! Linden’s numbers also suggest that the Residents figure includes even failed attempts to use the service. They reported adding their second million Residents between mid-October and December 14th, but they also reported just shy of 810 thousand logins for the same period. One million new Residents but only 810K logins leaves nearly 200K new Residents unaccounted for. Linden may be counting as Residents people who signed up and downloaded the client software, but who never logged in, or there may be some other reason for the mismatched figures, but whatever the case, Residents is remarkably inflated with regards to the published measure of use.

    (If there are any actual reporters reading this and doing a big cover story on Linden, you might ask about how many real people use Second Life regularly, as opposed to Residents or signups or avatars. As I write those words, though, I realize I might as well be asking Business Week to send me a pony for my birthday.)

    Like a push-up bra, Linden’s trick is as effective as it is because the press really, really wants to believe:

  • “It has a population of a million.” — Richard Siklos, New York Times
  • “In the Internet-based virtual world known as Second Life, for instance, more than 1 million citizens have created representations of themselves known as avatars…” — Michael Yessis, USA TODAY
  • “Since it started about three years ago, the population of Second Life has grown to 1.2 million users.” — Peter Valdes-Dapena, CNN
  • “So far, it’s signed up 1.3 million members.” — David Kirkpatrick, Fortune

    Professional journalists wrote those sentences. They work for newspapers and magazines that employ (or used to employ) fact-checkers. Yet here they are, supplementing Linden’s meager PR budget by telling their readers that Residents measures something it actually doesn’t.

    This credulity appears even in the smallest items. I discovered the “Residents vs Logins” gap when I came across a Business 2.0 post by Erick Schonfeld, where he included the mismatched numbers while congratulating Linden on a job well done. When I asked the obvious question in the comments — How come there are fewer logins than new Residents in the same period? — I got a nice email from Mr. Schonfeld, complimenting me on a good catch.

    Now I’m generally pretty enthusiastic about taking credit where it isn’t due, but this bit of praise failed to meet even my debased standards. The post was a hundred words long, and it had only two numbers in it. I didn’t have to use forensic accounting to find the discrepancy, I just used subtraction (an oft-overlooked tool in the journalistic toolkit, but surprisingly effective when dealing with numbers.)

    This is the state of business reporting in an age when even the pros want to roll with the cool blogger kids. Got a paragraph that contains only two numbers, and they don’t match? No problem! Post it anyway, and on to the next thing.

    The prize bit of PReporting so far, though, has to be Elizabeth Corcoran’s piece for Forbes called A Walk on the Virtual Side, where she claimed that Second Life had recently passed “a million unique customers.”

    This is three lies in four words. There isn’t one million of anything human inhabiting Second Life. There is no one-to-one correlation between Residents and users. And whatever Residents does measure, it has nothing to do with paying customers. The number of paid accounts is in the tens of thousands, not the millions (and remember, if you’re playing along at home, there can be more than one account per person. Kits, cats, sacks, and wives, how many logged into St. Ides?)

    Despite the credulity of the Fourth Estate (Classic Edition), there are enough questions being asked in the weblogs covering Second Life that the usefulness is going to drain out of the ‘Resident™ doesn’t mean resident’ trick over the next few months. We’re going to see three things happen as a result.

    The first thing that’s going to happen, or rather not happen, is that the regular press isn’t going go back over this story looking for real figures. As much as they’ve written about the virtual economy and the next net, the press hasn’t really covered Second Life as business story or tech story so much as a trend story. The sine qua non of trend stories is that a trend is fast-growing. The Residents figure was never really part of the story, it just provided permission to write about about how crazy it is that all the kids these days are getting avatars. By the time any given writer was pitching that story to their editors, any skepticism about the basic proposition had already been smothered.

    No journalist wants to have to write “When we told you that Second Life had 1.3 million members, we in no way meant to suggest that figure referred to individual people. Fortune regrets any misunderstanding.” And since no one wants to write that, no one will. They’ll shift their coverage without pointing out the shift to their readers.

    The second thing that is going to happen is an increase in arguments of the form “We mustn’t let Linden’s numbers blind us to the inevitability of the coming metaverse.” That’s the way it is with things we’re asked to take on faith — when A works, it’s evidence of B, but if A isn’t working as well as everyone thought, it’s suddenly unrelated to B.

    Finally, there is going to be a spike in the number of the posts claiming that the two million number was never important anyway, the press’s misreporting was all an innocent mistake, Linden was planning to call those reporters first thing Monday morning and explain everything. Tateru Nino has already kicked off this genre with a post entitled The Value of One. The flow of her argument is hard to synopsize, but you can get a sense of it from this paragraph:

    So, a hundred thousand, one million, two million. Those numbers mean something to us, but not because they have intrinsic, direct meaning. They have meaning because they’re filtered through the media, disseminated out into the world, believed by people, who then act based on that belief, and that is where the meaning lies.

    Expect more, much more, of this kind of thing in 2007.

  • Comments (10) + TrackBacks (0) | Category: social software

    December 12, 2006

    Second Life: What are the real numbers?Email This EntryPrint This Article

    Posted by Clay Shirky

    Second Life is heading towards two million users. Except it isn’t, really. We all know how this game works, and has since the earliest days of the web:

    Member of the Business Press: “How many users do you have?”
    CEO of Startup: (covers phone) “Hey guys, how many rows in the ‘users’ table?”
    [Sound F/X: Typing]
    Offstage Sysadmin: “One million nine hundred and one thousand one hundred and seventy-three.”
    CEO: (Into phone) “We have one point nine million users.”

    Someone who tries a social service once and bails isn’t really a user any more than someone who gets a sample spoon of ice cream and walks out is a customer.

    So here’s my question — how many return users are there? We know from the startup screen that the advertised churn of Second Life is over 60% (as I write this, it’s 690,800 recent users to 1,901,173 signups, or 63%.) That’s not stellar but it’s not terrible either. However, their definition of “recently logged in” includes everyone in the last 60 days, even though the industry standard for reporting unique users is 30 days, so we don’t actually know what the apples to apples churn rate is.

    At a guess, Second Life churn measured in the ordinary way is in excess of 85%, with a surge of new users being driven in by the amount of press the service is getting. The wider the Recently Logged In reporting window is, the bigger the bulge of recently-arrived-but-never-to-return users that gets counted in the overall numbers.

    I suspect Second Life is largely a “Try Me” virus, where reports of a strange and wonderful new thing draw the masses to log in and try it, but whose ability to retain anything but a fraction of those users is limited. The pattern of a Try Me virus is a rapid spread of first time users, most of whom drop out quickly, with most of the dropouts becoming immune to later use. Pointcast was a Try Me virus, as was LambdaMOO, the experiment that Second Life most closely resembles.

    Press Pass

    I have been watching the press reaction to Second Life with increasing confusion. Breathless reports of an Immanent Shift in the Way We Live® do not seem to be accompanied by much skepticism. I may have been made immune to the current mania by ODing on an earlier belief in virtual worlds:

    Similar to the way previous media dissolved social boundaries related to time and space, the latest computer-mediated communications media seem to dissolve boundaries of identity as well. […] I know a respectable computer scientist who spends hours as an imaginary ensign aboard a virtual starship full of other real people around the world who pretend they are characters in a Star Trek adventure. I have three or four personae myself, in different virtual communities around the Net. I know a person who spends hours of his day as a fantasy character who resembles “a cross between Thorin Oakenshield and the Little Prince,” and is an architect and educator and bit of a magician aboard an imaginary space colony: By day, David is an energy economist in Boulder, Colorado, father of three; at night, he’s Spark of Cyberion City—a place where I’m known only as Pollenator.

    This wasn’t written about Second Life or any other 3D space, it was Howard Rheingold writing about MUDs in 1993. This was a sentiment I believed and publicly echoed at the time. Per Howard, “MUDs are living laboratories for studying the first-level impacts of virtual communities.” Except, of course, they weren’t. If, in 1993, you’d studied mailing lists, or usenet, or irc, you’d have a better grasp of online community today than if you’d spent a lot of time in LambdaMOO or Cyberion City. Ou sont les TinyMUCKs d’antan?

    You can find similar articles touting 3D spaces shortly after the MUD frenzy. Ready for a blast from the past? “August 1996 may well go down in the annals of the Internet as the turning point when the Web was released from the 2D flatland of HTML pages.” Oops.

    So what accounts for the current press interest in Second Life? I have a few ideas, though none is concrete enough to call an answer yet.

    First, the tech beat is an intake valve for the young. Most reporters don’t remember that anyone has ever wrongly predicted a bright future for immersive worlds or flythrough 3D spaces in the past, so they have no skepticism triggered by the historical failure of things like LambdaMOO or VRML. Instead, they hear of a marvelous thing — A virtual world! Where you have an avatar that travels around! And talks to other avatars! — which they then see with their very own eyes. How cool is that? You’d have to be a pretty crotchety old skeptic not to want to believe. I bet few of those reporters ever go back, but I’m sure they’re sure that other people do (something we know to be false, to a first approximation, from the aforementioned churn.) Second Life is a story that’s too good to check.

    Second, virtual reality is conceptually simple. Unlike ordinary network communications tools, which require a degree of subtlety in thinking about them — as danah notes, there is no perfect metaphor for a weblog, or indeed most social software — Second Life’s metaphor is simplicity itself: you are a person, in a space. It’s like real life. (Only, you know, more second.) As Philip Rosedale explained it to Business Week “[I]nstead of using your mouse to move an arrow or cursor, you could walk your avatar up to an (AMZN) shop, browse the shelves, buy books, and chat with any of the thousands of other people visiting the site at any given time about your favorite author over a virtual cuppa joe.”

    Never mind that the cursor is a terrific way to navigate information; never mind that Amazon works precisely because it dispenses with rather than embraces the cyberspace metaphor; never mind that all the “Now you can shop in 3D efforts” like the San Francisco Yellow Pages tanked because 3D is a crappy way to search. The invitation here is to reason about Second Life by analogy, which is simpler than reasoning about it from experience. (Indeed, most of the reporters writing about Second Life seem to have approached it as tourists getting stories about it from natives.)

    Third, the press has a congenital weakness for the Content Is King story. Second Life has made it acceptable to root for the DRM provider, because of their enlightened user agreements concerning ownership. This obscures the fact that an enlightened attempt to make digital objects behave like real world objects suffers from exactly the same problems as an unenlightened attempt, a la the RIAA and MPAA. All the good intentions in the world won’t confer atomicity on binary data. Second Life is pushing against the ability to create zero-cost perfect copies, whereas Copybot relied on that most salient of digital capabilities, which is how Copybot was able to cause so much agida with so little effort — it was working with the actual, as opposed to metaphorical, substrate of Second Life.

    Finally, the current mania is largely push-driven. Many of the articles concern “The first person/group/organization in Second Life to do X”, where X is something like have a meeting or open a store — it’s the kind of stuff you could read off a press release. Unlike Warcraft, where the story is user adoption, here most of the stories are about provider adoption, as with the Reuters office or the IBM meeting or the resident creative agencies. These are things that can be created unilaterally and top-down, catnip to the press, who are generally in the business of covering the world’s deciders.

    The question about American Apparel, say, is not “Did they spend money to set up stores in Second Life?” Of course they did. The question is “Did it pay off?” We don’t know. Even the recent Second Life millionaire story involved eliding the difference between actual and potential wealth, a mistake you’d have thought 2001 would have chased from the press forever. In illiquid markets, extrapolating that a hundred of X are worth the last sale price of X times 100 is a fairly serious error.

    Artifacts vs. Avatars

    Like video phones, which have been just one technological revolution away from mass adoption since 1964, virtual reality is so appealingly simple that its persistent failure to be a good idea, as measured by user adoption, has done little to dampen enthusiasm for the coming day of Keanu Reeves interfaces and Snow Crash interactions.

    I was talking to Irving Wladawsky-Berger of IBM about Second Life a few weeks ago, and his interest in the systems/construction aspect of 3D seems promising, in the same way video phones have been used by engineers who train the camera not on their faces but on the artifacts they are talking about. There is something to environments for modeling or constructing visible things in communal fashion, but as with the video phone, they will probably involve shared perceptions of artifacts, rather than perceptions of avatars.

    This use, however, is specific to classes of problems that benefit from shared visual awareness, and that class is much smaller that the current excitement about visualization would suggest. More to the point, it is at odds with the “Son of MUD+thePalace” story currently being written about Second Life. If we think of a user as someone who has returned to a site after trying it once, I doubt that the number of simultaneous Second Life users breaks 10,000 regularly. If we raise the bar to people who come back for a second month, I wonder if the site breaks 10,000 simultaneous return visitors outside highly promoted events.

    Second Life may be wrought by its more active users into something good, but right now the deck is stacked against it, because the perceptions of great user growth and great value from scarcity are mutually reinforcing but built on sand. Were the press to shift to reporting Recently Logged In as their best approximation of the population, the number of reported users would shrink by an order of magnitude; were they to adopt industry-standard unique users reporting (assuming they could get those numbers), the reported population would probably drop by two orders. If the growth isn’t as currently advertised (and it isn’t), then the value from scarcity is overstated, and if the value of scarcity is overstated, at least one of the engines of growth will cool down.

    There’s nothing wrong with a service that appeals to tens of thousands of people, but in a billion-person internet, that population is also a rounding error. If most of the people who try Second Life bail (and they do), we should adopt a considerably more skeptical attitude about proclamations that the oft-delayed Virtual Worlds revolution has now arrived.

    Comments (85) + TrackBacks (0) | Category: social software

    November 20, 2006

    Social Facts, Expertise, Citizendium, and CarrEmail This EntryPrint This Article

    Posted by Clay Shirky

    I want to offer a less telegraphic account of the relationship between expertise, credentials, and authority than I did in Larry Sanger, Citizendium, and the Problem of Expertise, and then say why I think the cost of coordination in the age of social software favors Wikipedia over Citizendium, and over traditionally authoritative efforts such as Britannica.

    Make a pot of coffee; this is going to be long, and boring.

    Those of us who write about Wikipedia, both pro and con, often mix two different views: descriptive — Wikipedia is/is not succeeding — and judgmental — Wikipedia is/is not good. (For the record, my view is that Wikipedia is a success, and that society is better off with Wikipedia than it would be without it.) What I love about the Citizendium proposal is that, by proposing a fusion of collaborative construction and expert authority, it presses people who dislike or mistrust Wikipedia to say whether they think that the wiki form of communal production can be improved, or is per se bad.

    Nicholas Carr, in What will kill Citizendium, came out in the latter camp. Explaining why he thinks Ctizendium is a bad idea, he offers his prescription for the right way to do things: “[…] you keep the crowd out of it and, in essence, create a traditional encyclopedia.” No need for that ‘in essence’ there. The presence of the crowd is what distinguishes wiki production; this is a defense of the current construction of authority, suggesting that the traditional mechanism for creating encyclopedias is the correct one, and alternate forms of construction are not.

    This is certainly a coherent point of view, but one that I believe will fail in practical terms, because it is uneconomical. (Carr, in his darker moments, seems to believe something similar, but laments what the economics of peer production mean. This is a “Wikipedia is succeeding/is not good” argument.) In particular, I believe that the costs of nominating and then deferring to experts will make Citizendium underperform its competition, relative to the costs of merely involving experts as ordinary participants, as Wikipedia does.

    Expertise, Credentials, and Authority

    First, let me say that I am a realist, which is to say that I believe in a reality that is not socially constructed. The materials that make up my apartment, wood and stone and so on, actually exist, and are independent of any observer. A real tree that falls in a real forest displaces real air, even if no one is there to interpret that as sound.

    I also believe in social facts, things that are true because everyone agrees they are true. My apartment itself is made of real stuff, but its my-ness is built on agreements: my landlady leases it to me, that lease is predicated on her ownership, that ownership is recognized by the city of New York, and so on. Social facts are no less real than non-social facts — my apartment is actually my apartment, my wife is my wife, my job is my job — they are just real for different reasons.

    If everyone stopped agreeing that my job was my job (I quit or was fired, say), I could still walk down to NYU and draw network diagrams on a whiteboard at 1pm on a Tuesday, but no one would come to listen, because my ramblings wouldn’t be part of a class anymore. I wouldn’t be faculty; I’d be an interloper. Same physical facts — same elevator and room and white board and even the same person — but different social facts.

    Some facts are social, some are not. I believe that Sanger, Carr and I all agree that expertise is not a social fact. As Carr says ‘An architect does not achieve expertise through some arbitrary social process of “credentialing.” He gains expertise through a program of study and apprenticeship in which he masters an array of facts and techniques drawn from such domains as mathematics, physics, and engineering.’ I agree with that, and amended my earlier sloppiness in distinguishing between having expertise and being an expert, after being properly called on it by Eric Finchley in the comments.

    However, though Carr’s description is accurate, is it incomplete: an architect does not achieve expertise through credentialing, but an architect does not become an architect through expertise either. An architect is someone with expertise who has also been granted an architect’s credentials. These credentials are ideally granted on proof of the kinds of antecedents that indicate expertise — in the case of architects, relevant study (itself certified with the social fact of a degree) and significant professional work.

    Consider the following case: a young designer with an architect’s degree designs a building, and a credentialed architect working at the same firm then affixes her stamp to the drawings. The presence of the stamp means that a contractor can use the drawings to do certain kinds of work; without it the drawings shouldn’t be used for such things. Both the expertise and the credentials are necessary to make a set of drawings usable, but in this fairly common scenario, the expertise and the credentials are held by different people.

    This system is designed to produce enough liability for architects that they will supervise the uncredentialed; if they fail to, their own credentials will be taken away. Now consider a disbarred architect (or lawyer or doctor.) There has been no change in their expertise, but a great change in their credentials. Most of the time, we can take the link between authority, credentials, and expertise for granted (its why we have credentials, in fact), but in edge cases, we can see them as separate things.

    The clarity to be gotten from all this definition is a bit of a damp squib: Carr and I are in large agreement about the Citizendium proposal. He thinks that conferring authority is the hard challenge for Citizendium; I think that conferring authority is the hard challenge for Citizendium. He thinks that the openness of a wiki is incompatible with Citizendium’s proposed form of conferring authority, as do I. And we both believe this weakness will be fatal.

    Where we disagree is in what this means for society.

    The Cost of Credentials

    Lying on a bed in an emergency room, you think “Oh good, here comes the doctor.” Your relief comes in part because the doctor has the expertise necessary to diagnose and treat you, and in part because the doctor has the authority to do things like schedule you for surgery if you need it. Whatever your anxieties at that moment, they don’t include the possibility that the nurses will ignore the doctor’s diagnosis, or refuse to treat you in the manner the doctor suggests.

    You don’t worry that expertise and authority are different kinds of things, in other words, because they line up perfectly from your point of view. You simply ascribe to the visible doctor many things that are actually true of the invisible system the doctor works in. The expertise resides in the doctor, but the authority is granted by the hospital, with credentials helping bridge the gap.

    So here’s the thing: it’s incredibly expensive to create and maintain such systems, including especially the cost of creating and policing credentials and authority. We have to make and enforce myriad refined distinctions — not just physician and soldier and chairman but ‘admitting physician’ and ‘second lieutenant’ and ‘acting chairman.’ We don’t let people get married or divorced without the presence of official oversight. Lots of people can drive the bus; only bus drivers may drive the bus. We make it illegal to impersonate an officer. And so on, through innumerable tiny, self-reinforcing choices, all required to keep the links between expertise, credentials and authority functional.

    These systems are beneficial for society. However, they are not absolutely beneficial, they are only beneficial when their benefits outweigh their costs. And we live in an era where all kinds of costs — social costs, coordination costs, Coasean costs — are undergoing a revolution.

    Cost Changes Everything

    Earlier, writing about folksonomies, I said “We need a phrase for the class of comparisons that assumes that the status quo is cost-free.” We still need that; I propose “Cost-free Present” — when people believe in we live in a cost-free present, they also believe that any value they see in the world is absolute, not relative. A related assumption is that any new system that has disadvantages relative to the present one is therefore inferior; if the current system creates no costs, then any proposed change that creates new bad outcomes, whatever the potential new good outcomes, is worse than maintaining the status quo.

    Meanwhile, out here in the real world, cost matters. As a result, when the cost structure for creating, say, an encyclopedia changes, our existing assumptions about encyclopedic value have to be re-examined, because current encyclopedic values are relative, not absolute. It is possible for low-cost, low-value systems to be better than high-cost, high-value systems in the view of the society adopting them. If the low-cost system can increase in value over time while remaining low cost, even better.

    Pick your Innovator’s Dilemma: the Gutenberg bible was considerably less beautiful than scribal copies, the Model T was less well constructed than the Curved Dash Olds, floppy disks were considerably less reliable than hard drives, et cetera. So with Wikipedia and Encyclopedia Britannica: Wikipedia began life as a lost-cost, low-value alternative, but it was accessible, shareable, and improvable. Britannica, by contrast, has always been high-value, but it is both difficult and expensive for readers to get to, and worse, they can’t use what they see — a Britannica reader can’t copy and post an article, can’t email the contents to their friends, can’t even email those friends the link with any confidence that they will be able to see it.

    Barriers to both access and re-use are built into the Britannica cost structure, and without those barriers, it will collapse. Nothing about the institution of Britannica has changed in the five years of Wikipedia’s existence, but in the current ecosystem, the 1768 model of creation — you pay us and we make an Encyclopedia — has been transformed from a valuable service to a set of self-perpetuating, use-crippling barriers.

    This what’s wrong with Cost-free Present arguments: the principal competitive advantages of Wikipedia over Britannica, such as shareability or rapid refactoring (as of the Planet entry after Pluto’s recent demotion) are things which were simply not possible in 1768. Wikipedia is not a better Britannica than Britannica; it is a better fit for the current environment than Britannica is. The measure of possible virtues of an encyclopedia now include free universal access and unlimited re-use. As a result, maintaining Britannica costs more in a world with Wikipedia than it did in a world without it, in the same way scribal production became more expensive after the invention of movable type than before, without the scribes themselves doing anything different.

    If we do what we always did, we’ll get the result we always got

    Citizendium seems predicated on several related ideas about cost and value: having expertise and being an expert are roughly the same thing; the costs of certifying experts will be relatively low; building and running software that confers a higher degree of authority to them than on non-expert users will be similarly low; and the appeal to non-experts of participating in such a system will be high. If these things are true, than a hybrid of voluntary participation and expert authority will be more valuable than either extreme.

    I am betting that those things aren’t true, because the costs of certifying experts and insuring deference to them — the costs of creating and sustaining the necessary social facts — will sandbag the system, making it too annoying to use.

    The first order costs will come from the certification and deference itself. By proposing to recognize external credentialing mechanisms, Citizendium sets itself up to take on the expenses of determining thresholds and overlaps of expertise. A masters student in psychology doing work on human motivation may know more about behavioral economics than a Ph.D. in neo-classical economics. It would be easy to label them both experts, but on what grounds should their disputes be adjudicated?

    On Wikipedia, the answer is simple — deference is to contributions, not to contributors, and is always provisional. (As with the Pluto example enough, even things as seemingly uncontentious as planethood turned out to be provisional.) Wikipedia certainly has management costs (all social systems do), but it has the advantage that those costs are internal, and much of the required oversight is enforced by moral suasion. It doesn’t take on the costs of forcing deference to experts because it doesn’t recognize the category of ‘expert’ as primitive in the system. Experts contribute to Wikipedia, but without requiring any special consideration.

    Citizendium’s second order costs will come from policing the system as a whole. If the process of certification and enforcement of deference become even slightly annoying to the users, they will quickly become non-users. The same thing will happen if the projection of force needed to manage Citizendium delegitimizes the system in the eyes of the contributors.

    The biggest risk with Wikipedia is ongoing: lousy or malicious edits, an occurrence that happens countless times a day. The biggest risk with Citizendium, on the other hand, is mainly up front, in the form of user inaction. The Citizendium project assumes that the desire of ordinary users to work alongside and be guided by experts is high, but everything in the proposal seems to raise the costs of contribution, relative to Wikipedia. If users do not want to participate in a system where the costs of participating are high, Citizendium will simply fail to grow.

    Comments (12) + TrackBacks (0) | Category: social software

    September 22, 2006

    What is the problem with deference to experts on Wikipedia?Email This EntryPrint This Article

    Posted by Clay Shirky

    Interesting pair of comments in Larry Sanger, Citizendium, and the Problem of Expertise, on the nature and seriousness of experts not contributing to Wikipedia:
    22. David Gerard on September 22, 2006 07:08 AM writes…

    Plenty of people complain of Wikipedia’s alleged “anti-expert bias”. I’ve yet to see solid evidence of it. Unless “expert-neutral” is conflated to mean “anti-expert.” Wikipedia is expert-neutral - experts don’t get a free ride. Which is annoying when you know something but are required to show your working, but is giving us a much better-referenced work.

    One thing the claims of “anti-expert bias” fail to explain is: there’s lots of experts who do edit Wikipedia. If Wikipedia is so very hostile to experts, you need to explain their presence.
    Permalink to Comment

    23. engineer_scotty on September 22, 2006 01:19 PM writes…

    I’ve been studying the so-called “expert problem” on Wikipedia—and I’m becoming more and more convinced that it isn’t and expert problem per se; it is a jackass problem. As in some Wikipedians are utter jackasses—in this context, “jackass” is an umbrella category for a wide variety of problem behaviors which are contrary to Wikipedia policy—POV pushing, advocacy of dubious theories, vandalism, abusive behavior, etc. Wikipedia policy is reasonably good at dealing with vandalism, abusive behavior and incivility (too good, some think, as WP:NPA occasionally results in good editors getting blocked for wielding the occasional cluestick ‘gainst idiots who sorely need it). It isn’t currently good at dealing with POV-pushers and crackpots whose edits are civil but unscholarly, and who repeatedly insert dubious material into the encyclopedia. Recent policy proposals are designed to address this.

    Many experts who have left, or otherwise have expressed dissatisfaction with Wikipedia, fall into two categories: Those who have had repeated bad experiences dealing with jackassses, and are frustrated by Wikipedia’s inability to restrain said jackasses; and those who themselves are jackasses. Wikipedia has seen several recent incidents, including one this month, where notable scientists have joined the project and engaged in patterns of edits which demonstrated utter contempt for other editors of the encyclopedia (many of whom were also PhD-holding scientists, though lesser known), attempted to “own” pages, attempted to portray conjecture or unpublished research as fact, or have exaggerated the importance or quality of their own work. When challenged, said editors have engaged in (predictable) tirades accusing the encyclopedia of anti-intellectualism and anti-expert bias—charges we’ve all heard before.

    The former sort of expert the project should try to keep. The latter, I think the project is probably better off without; and I suspect they would wear out their welcomes quickly on Citizendium as well.
    I would love to see a few case studies, linked to the History and Talk pages of a few articles— “Here was the expert contribution, here was the jackass edit, this is what was lost”, etc. Reading Engineer Scotty’s comment, and given the general sense of outraged privilege that seems to run through much of the “Experts have their work edited without permission!” literature, I am guessing that the problem is not so much experts contributing and then being driven away as it is non-contributions by people unwilling to work in an environment wherre their contributions aren’t sacrosanct.

    Comments (5) + TrackBacks (0) | Category: social software

    September 20, 2006

    Larry Sanger on me on CitizendiumEmail This EntryPrint This Article

    Posted by Clay Shirky

    A response from Larry Sanger, posted here in its entirety:

    Thanks to Clay Shirky for the opportunity to reply here on Many2Many
    to his “Larry Sanger, Citizendium, and the Problem of Expertise,” First, two points about Clay’s style of argumentation, which I simply cannot let go without comment. Then some replies to his actual arguments.

    1. Allow me to identify my own core animating beliefs, thank you very much.

    Clay’s piece annoying tendency to characterize my assumptions uncharitably and without evidence, and to psychologize about me. Thus, Clay says things like: “Sanger‚s published opinions seem based on three beliefs”; “Sanger wants to believe that expertise can survive just fine outside institutional frameworks”; “Sanger’s core animating belief seems to be a faith in experts”; “Sanger’s view seems to be that expertise is a quality like height”; and “Sanger also underestimates the costs of setting up and then enforcing a process that divides experts from the rest of us.”

    I find myself strongly disagreeing with Clay’s straw Sanger. However, I am not that Sanger! Last time I checked, I was made of flesh and blood, not straw.

    2. May I borrow that crystal ball when you’re done with it?

    Repeatedly, Clay makes dire predictions for the Citizendium. “Structural issues…will probably prove quickly fatal”; “institutional overhead…will stifle Citizendium”; “policing certification will be a common case, and a huge time-sink” so “the editor-in-chief will then have to spend considerable time monitoring that process”; “Citizendium will re-create the core failure of Nupedia”; “Sanger believes that Wikipedia goes too far in its disrespect of experts; what killed Nupedia and will kill Citizendium is that they won’t go far enough.”

    I think Clay lacks any good reason to think the Citizendium will fail; but clearly he badly wants it to fail, and his comments are animated by wishful thinking. That, anyway, seems the most parsimonious explanation. To borrow one of Clay’s phrases, and return him the favor: it is interesting “how consistent Clay has been about his beliefs” on the low value of officially-recognized expertise in online communities. “His published opinions seem based on” the belief in the supreme value and efficacy of completely flat self-organizing communities. The notion of experts being given special authority, even very circumscribed authority, does extreme violence to this “core animating belief” (to borrow another of Clay’s phrases). It must, therefore, be impossible.

    Less flippantly now. I do make a point of being properly skeptical about all of my projects—that’s another thing I’ve been consistent about. You can probably still find writings from 2000 and 2001 in which I said I didn’t know whether Nupedia or Wikipedia would work. I have no idea if the Citizendium will work. What I do know is that it is worth a try, and we’ll do our best to solve problems that we can anticipate and as they arise.

    By the way, there’s a certain irony in the situation, isn’t there? Clay Shirky, respected expert about online communities, holds forth about a new proposed online community, and does what so many experts love to do: make bold predictions about the prospects of items in their purview. Meanwhile, I, the alleged expert-lover, cast aspersions on his abilities to make such predictions. If my “core animating belief” were “a faith in experts,” why would I lack faith in this particular expert?

    3. I want to be a social fact, too!

    Let’s move on to Clay’s actual arguments. He begins his first argument with something perfectly true, that expertise (in the relevant sense, an operational concept of expertise) is a social fact, that this social fact is conferred (not always formally, but often) by institutions, and that, therefore, one cannot have expertise without (in some sense) “institutional overhead.” So far, so good. The current proposal—which is open to debate, at this early stage, even from Clay himself—addresses this situation by proposing to avoid editor application review committees in favor of self-designation of editorial status. The details are relevant, so let me quote them from the FAQ:

    We do not want editors to be selected by a committee, which process is too open to abuse and politics in a radically open and global project like this one is. Instead, we will be posting a list of credentials suitable for editorship. (We have not constructed this list yet, but we will post a draft in the next few weeks. A Ph.D. will be neither necessary nor sufficient for editorship.) Contributors may then look at the list and make the judgment themselves whether, essentially, their CVs qualify them as editors. They may then go to the wiki, place a link to their CV on their user page, and declare themselves to be editors. Since this declaration must be made publicly on the wiki, and credentials must be verifiable online via links on user pages, it will be very easy for the community to spot [most] false claims to editorship.
    What then is Clay’s criticism? “The problem” at the beginning of the argument was that “experts are social facts.” Yeah, so? So, says Clay,
    Sanger expects that decertification will only take place in unusual cases. This is wrong; policing certification will be a common case, and a huge time-sink. If there is a value to being an expert, people will self-certify to get at that value, not matter what their credentials. The editor-in-chief will then have to spend considerable time monitoring that process, and most of that time will be spent fighting about edge cases.

    My initial reaction to this was: how on Earth could Shirky know all that? Furthermore, isn’t it quite obvious that, far from being a static proposal, this project is going to be able to move nimbly (I usually propose radical changes and refinements to my projects) in order to solve just such problems, should they arise?

    In any event, based on my own experience, I counter-predict that Clay will probably be wrong in his prediction. There will probably be a lot of people who humorously, out of cluelessness, or whatever, claim to be editors.

    For the easy cases, which will probably be most of them, constables will be able to rein people in, nearly as easy as they can rein in vandalism. No doubt we will have a standard procedure for achieving this. As to the borderline (“edge”) cases (e.g., some grad students and independent scholars), Clay gives us no reason to think that the editor-in-chief will have to spend large amounts of time fighting about them. Unlike Wikipedia, and like many OSS projects, there will be a group of people authorized to select the “release managers” (so to speak). This policy will be written into the project charter, support of which will be a requirement of participation in the project.

    The review process for editor declarations, therefore, will be clear and well-accepted enough—that, after all, is the whole point of establishing a charter and “rule of law” in the online community—that the process can be expected to work smoothly. Mind you, it will be needed because of course there will be borderline cases, and disgruntled people, but Clay has given no reason whatsoever to think it will dominate the entire proceedings.

    Besides, this is a responsibility I propose to delegate to a workgroup; I will probably be too busy to be closely involved in it.

    Far from being persuasive, it is actually ironic that Clay cites primordial fights I had with trolls on Wikipedia as evidence of his points. It was precisely due to a lack of clearly-circumscribed authority and widely-accepted rules that I had to engage in such fights. Consequently, the Citizendium is setting up a charter, editors, and constables precisely to prevent such problems.

    4. Warm and fuzzy yes, a hierarchy no.

    Clay nicely sums up his next argument this way:

    Real experts will self-certify; rank-and-file participants will be delighted to work alongside them; when disputes arise, the expert view will prevail; and all of this will proceed under a process that is lightweight and harmonious. All of this will come to naught when the citizens rankle at the reflexive deference to editors; in reaction, they will debauch self-certification (leading to irc-style chanop wars), contest expert prerogatives, raising the cost of review to unsupportable levels (Wikitorial, round II,) take to distributed protest (q.v.Hank the Angry Drunken Dwarf), or simply opt-out (Nupedia in a nutshell.)

    (By the way, Clay is completely wrong about citizen participation in Nupedia. They made up the bulk of authors in the pipeline. Our first article was by a grad student. An undergrad wrote several biology articles. There have been so many myths are made about Nupedia, so completely divorced from reality, that it has become a fascinating and completely fact-free Rohrschach test for everything bad that anyone wants to say about expert authority in open collaboration.)

    The Citizendium is, by Clay’s lights, a radical experiment that does violence to his cherished notions of what online communities should be like. Persons inclined to “debauch self-certification” as on IRC chatrooms will be removed from the project; and others will not protest at such perfectly appropriate treatment, because we will have already announced this as a policy.

    Through self-selection the community can be expected to be in favor of such policies; those who dislike them will always have Wikipedia.

    That’s part of the beauty of a world with both a Citizendium and a Wikipedia in it. Those who (like you, Clay) instinctively hate the Citizendium—we’ve seen a little of this in blogs lately, calling the very idea “Wikipedia for stick-in-the-muds,” “Wikipedia for control freaks,” a “horror,” etc.—will always have Wikipedia. I strongly encourage you to stick with Wikipedia if you dislike the idea of the Citizendium that much. That will make matters easier for everyone. If other people want to organize themselves in a different way—a way you’d never dream of doing—then please give them room to do so. As a result we’ll have one project for people who agree with you, Clay, and one for people who agree with me, and the world will be richer.

    Clay does give some more support for thinking that an editor-guided wiki is unworkable. He says that the viability of a community resembles a “U curve” with one end being a total hierarchy and the other end being “a functioning community with a core group.” Apparently, projects that are neither hierarchies nor communities, which Clay implies is where the Citizendium would fit, would incur too many “costs of being an institution” and “significant overhead of process.” What I find particularly puzzling about this is how he describes the ends of U curve. I would have expected him to say hierarchy on one end and a totally flat, leaderless community on the other end. But instead, opposite the hierarchy is “a functioning community with a core group.” How is it, then, that the Citizendium as proposed would not constitute “a functioning community with a core group”?

    Let me put this more plainly, setting aside Clay’s puzzling theoretical apparatus. What the world has yet to test is the notion of experts and ordinary folks (and remember: experts working outside their areas of expertise are then “ordinary folks”) working together, shoulder-to-shoulder, on a single project according to open, open source principles. That is the radical experiment I propose. This actually hearkens back to the way OSS projects essentially work. So far, to my knowledge, experts have not been invited in to “gently guide” open content projects in a way roughly analogous to the way that senior developers gently guide OSS projects, deciding what changes are in the next release and what isn’t. You might say that the analogy does not work because senior developers of OSS projects are chosen based on the merits of their contributions within the project. But what if we regard an encyclopedia as continuous with the larger world of scholarship, so that scholarly work outside of the narrow province of a single project becomes relevant for determining a senior content developer? For an encyclopedia, that’s simply a sane variant on the model.

    Whereas OSS projects have special, idiosyncratic requirements, encyclopedias frankly do not. There’s no point to creating an insular community, an “in group” of people who have mastered the particular system, because it’s not about the system—it’s about something any good scholar can contribute to, an encyclopedia. Then, if the larger, self-selecting community invites and welcomes such people to join them as “senior content developers,” why not think the analogy with OSS is adequately preserved?

    (For more of the latter argument please see a new essay I am going to try to circulate among academics.)

    Comments (9) + TrackBacks (0) | Category: social software

    September 18, 2006

    Larry Sanger, Citizendium, and the Problem of ExpertiseEmail This EntryPrint This Article

    Posted by Clay Shirky

    The interesting thing about Citizendium, Larry Sanger’s proposed fork of Wikipedia designed to add expert review, is how consistent Sanger has been about his beliefs over the last 5 years. I’ve been reviewing the literature from the dawn of Wikipedia, born from the failure of the process-laden and expert-driven Nupedia, and from then to now, Sanger’s published opinions seem based on three beliefs:

    1. Experts are a special category of people, who can be readily recognized within their domains of expertise.
    2. A process of open creation in which experts are deferred to as of right will be superior to one in which they are given no special treatment.
    3. Once experts are identified, that deference will mainly be a product of moral suasion, and the only place authority will need to intrude are edge cases.

    All three beliefs are false.

    There are a number of structural issues with Citizendium, many related to the question of motivation on the part of the putative editors; these will probably prove quickly fatal. More interesting to me, though, is is the worldview behind Sanger’s attitude towards expertise, and why it is a bad fit for this kind of work. Reading the Citizendium manifesto, two things jump out: his faith in experts as a robust and largely context-free category of people, and his belief that authority can exist largely free of expensive enforcement. Sanger wants to believe that expertise can survive just fine outside institutional frameworks, and that Wikipedia is the anomaly. It can’t, and it isn’t.

    Experts Don’t Exist Independent of Institutions

    Sanger’s core animating belief seems to be a faith in experts. He took great care to invite experts to the Nupedia Advisory Board, and he has consistently lamented that Wikipedia offers no special prerogatives for expert review, and no special defenses against subsequent editing of material written by experts. Much of his writing, and the core of Citizendium, is based on assumptions about how experts should be involved in a project like this.

    The problem Citizendium faces is that experts are social facts — society typically recognizes experts through some process of credentialling, such as the granting of degrees, professional certifications, or institutional engagement. We have a sense of what it means that someone is a doctor, a judge, an architect, or a priest, but these facts are only facts because we agree they are. If I say “I sentence you to 45 days in jail”, nothing happens. If a judge says “I sentence you to 45 days in jail”, in a court of law, dozens of people will make it their business to act on that imperative, from the bailiff to the warden to the prison guards. My words are the same as the judges, but the judge occupies a position of authority that gives his words an effect mine lack, an authority only exists because enough people agree that it does.

    Sanger’s view seems to be that expertise is a quality like height — some people are obviously taller than others, and the rest of us have no problem recognizing who the tall people are. But expertise isn’t like that at all; it is in fact highly subject to shifts in context. A lawyer from New York can’t practice in California without passing the bar there. A surgeon from India can’t operate on a patient in the US without further certification. The UN representative from Yugoslavia went away when Yugoslavia did, and so on.

    As a result, you cannot have expertise without institutional overhead, and institutional overhead is what stifled Nupedia, and what will stifle Citizendium. Sanger is aware of this challenge, and offers mollifying details:

    […]we will be posting a list of credentials suitable for editorship. (We have not constructed this list yet, but we will post a draft in the next few weeks. A Ph.D. will be neither necessary nor sufficient for editorship.) Contributors may then look at the list and make the judgment themselves whether, essentially, their CVs qualify them as editors. They may then go to the wiki, place a link to their CV on their user page, and declare themselves to be editors. Since this declaration must be made publicly on the wiki, and credentials must be verifiable online via links on user pages, it will be very easy for the community to spot false claims to editorship.

    We will also no doubt need a process where people who do not have the credentials are allowed to become editors, and where (in unusual cases) people who have the credentials are removed as editors.

    Sanger et al. set the bar for editorship, editors self-certify, then, in order to get around the problems this will create, there will be an additional certification and de-certification process internal to the site. On Citizendium, if you are competent but uncredentialed, you will have to be vetted before you are allowed to ascend to the editor’s chair, and if you are credentialed but incompetent, you’re in until decertification. And, critically, Sanger expects that decertification will only take place in unusual cases.

    This is wrong; policing certification will be a common case, and a huge time-sink. If there is a value to being an expert, people will self-certify to get at that value, not matter what their credentials. The editor-in-chief will then have to spend considerable time monitoring that process, and most of that time will be spent fighting about edge cases.

    Sanger himself experienced this in his fight with Cunctator at the dawn of Wikipedia; Cunc questioned Sanger’s authority, leading Sanger to defend it with increasing vigor. As Sanger said at the time “…in order to preserve my time and sanity, I have to act like an autocrat. In a way, I am being trained to act like an autocrat.” Sanger’s authority at Wikipedia required his demonstrating it, yet this very demonstration made his job harder, and ultimately untenable. This the common case; as any parent can tell you, exercise of presumptive authority creates the conditions under which it is tested. As a result, Citizendium will re-create the core failure of Nupedia, namely putting at the center of the effort a process whose maintenance takes more energy than can be mustered by a volunteer project.

    “We’re a Warm And Fuzzy Hierarchy”: The Costs of Enforcement

    In addition to his misplaced faith in the rugged condition of expertise, Sanger also underestimates the costs of setting up and then enforcing a process that divides experts from the rest of us. Curiously, this underestimation seems to be borne of a belief that most of the world shares his views on the appropriate deference to expertise:

    Can you really expect headstrong Wikipedia types to work under the guidance of expert types in this way?

    Probably not. But then, the Citizendium will not be Wikipedia. We do expect people who have proper respect for expertise, for knowledge hard gained, to love the opportunity to work alongside editors. Imagine yourself as a college student who had the opportunity to work alongside, and under the loose and gentle direction of, your professors. This isn’t going to be a top-down, command-and-control system. It is merely a sensible community: one where the people who have made it their life’s work to study certain areas are given a certain appropriate authority—without thereby converting the community into a traditional top-down academic editorial scheme.

    Well, can you expect the experts to want to work “shoulder-to-shoulder” with nonexperts?

    Yes, because some already do on Wikipedia. Furthermore, they will have an incentive to work in this project, because when it comes to content—i.e., what the experts really care about—they will be in charge.

    These passages evince a wounded sense of purpose: Experts are real, and it is only sensible and proper that they be given an appropriate amount of authority. The totality of the normative view on display here is made more striking because Sanger never reveals the source of these judgments. “Sensible” according to whom? How much authority is “appropriate”? How much control is implied by being “in charge”, and what happens when that control is abused?

    These responses are also mutually contradictory. Citizendium, the manifesto claims, will not be a traditional top-down academic scheme, but experts will be in charge of the content. The only way experts can be in charge without top-down imposition is if every participant internalizes respect for authority to the point that it is never challenged in the first place. One need allude only lightly to the history of social software since at least Communitree to note that this condition is vanishingly rare.

    Citizendium is based less on a system of supportable governance than on the belief that such governance will not be necessary, except in rare cases. Real experts will self-certify; rank-and-file participants will be delighted to work alongside them; when disputes arise, the expert view will prevail; and all of this will proceed under a process that is lightweight and harmonious. All of this will come to naught when the citizens rankle at the reflexive deference to editors; in reaction, they will debauch self-certification (leading to irc-style chanop wars), contest expert preogatives, rasing the cost of review to unsupportable levels (Wikitorial, round II,) take to distributed protest (q.v. Hank the Angry Drunken Dwarf), or simply opt-out (Nupedia in a nutshell.)

    The “U”-Curve of Organization and the Mechanisms of Deference

    Sanger is an incrementalist, and assumes that the current institutional framework for credentialling experts and giving them authority can largely be preserved in a process that is open and communally supported. The problem with incrementalism is that the very costs of being an institution, with the significant overhead of process, creates a U curve — it’s good to be a functioning hierarchy, and its good to be a functioning community with a core group, but most of the hybrids are less fit than either of the end points.

    The philosophical issue here is one of deference. Citizendium is intended to improve on Wikipedia by adding a mechanism for deference, but Wikipedia already has a mechanism for deference — survival of edits. I recently re-wrote the conceptual recipe for a Menger Sponge, and my edits have survived, so far. The community has deferred not to me, but to my contribution, and that deference is both negative (not edited so far) and provisional (can always be edited.)

    Deference, on Citizendium will be for people, not contributions, and will rely on external credentials, a priori certification, and institutional enforcement. Deference, on Wikipedia, is for contributions, not people, and relies on behavior on Wikipedia itself, post hoc examination, and peer-review. Sanger believes that Wikipedia goes too far in its disrespect of experts; what killed Nupedia and will kill Citizendium is that they won’t go far enough.

    Comments (34) + TrackBacks (0) | Category: social software

    June 7, 2006

    Reactions to Digital MaoismEmail This EntryPrint This Article

    Posted by Clay Shirky

    Last5 week, published Jaron Lanier’s Digital Maoism piece, warning of the dangers of new collectivism, and singling out Wikipedia for criticism. Today, Edge has several reactions to Lanier’s warnings, from people who think about social software from a variety of perspectives, including Douglas Rushkoff, Quentin Hardy, Yochai Benkler, me (and I re-print my piece here), Cory Doctorow, Kevin Kelly, Esther Dyson, Larry Sanger, Fernanda Viegas & Martin Wattenberg, Jimmy Wales, and George Dyson.

    I include my reaction here as well, as Edge doesn’t have comments.

    Jaron Lanier is certainly right to look at the downsides of collective action. It’s not a revolution if nobody loses, and in this case, expertise and iconoclasm are both relegated by some forms of group activity. However, “Digital Maoism” mischaracterizes the present situation in two ways. The first is that the target of the piece, the hive mind, is just a catchphrase, used by people who don’t understand how things like Wikipedia really work. As a result, criticism of the hive mind becomes similarly vague. Second, the initial premise of the piece — there are downsides to collective production of intellectual work — gets spread it so widely that it comes to cover RSS aggregators, American Idol, and the editorial judgment of the NY Times. These are errors of overgeneralization; it would be good to have a conversation about Wikipedia’s methods and governance, say, but that conversation can’t happen without talking about its actual workings, nor can it happen if it is casually lumped together with other, dissimilar kinds of group action.

    The bigger of those two mistakes appears early:

    “The problem I am concerned with here is not the Wikipedia in itself. It’s been criticized quite a lot, especially in the last year, but the Wikipedia is just one experiment that still has room to change and grow. […] No, the problem is in the way the Wikipedia has come to be regarded and used; how it’s been elevated to such importance so quickly.”

    Curiously, the ability of the real Wikipedia to adapt to new challenges is taken at face value. The criticism is then directed instead at people proclaiming Wikipedia as an avatar of a golden era of collective consciousness. Let us stipulate that people who use terms like hive mind to discuss Wikipedia and other social software are credulous at best, and that their pronouncements tend towards caricature. What “Digital Maoism” misses is that Wikipedia doesn’t work the way those people say it does.

    Neither proponents nor detractors of hive mind rhetoric have much interesting to say about Wikipedia itself, because both groups ignore the details. As Fernanda Viegas’s work shows, Wikipedia isn’t an experiment in anonymous collectivist creation; it is a specific form of production, with its own bureaucratic logic and processes for maintaining editorial control. Indeed, though the public discussions of Wikipedia often focus on the ‘everyone can edit’ notion, the truth of the matter is that a small group of participants design and enforce editorial policy through mechanisms like the Talk pages, lock protection, article inclusion voting, mailing lists, and so on. Furthermore, proposed edits are highly dependant on individual reputation — anonymous additions or alterations are subjected to a higher degree of both scrutiny and control, while the reputation of known contributors is publicly discussed on the Talk pages.

    Wikipedia is best viewed as an engaged community that uses a large and growing number of regulatory mechanisms to manage a huge set of proposed edits. “Digital Maoism” specifically rejects that point of view, setting up a false contrast with open source projects like Linux, when in fact the motivations of contributors are much the same. With both systems, there are a huge number of casual contributors and a small number of dedicated maintainers, and in both systems part of the motivation comes from appreciation of knowledgeable peers rather than the general public. Contra Lanier, individual motivations in Wikipedia are not only alive and well, it would collapse without them.

    “The Digital Maoism” argument is further muddied by the other systems dragged in for collectivist criticism. There’s the inclusion of American Idol, in which a popularity contest is faulted for privileging popularity. Well, yes, it would, wouldn’t it, but the negative effects here don’t come from some new form of collectivity, they come from voting, a tool of fairly ancient provenance. Decrying Idol’s centrality is similarly misdirected. This season’s final episode was viewed by roughly a fifth of the country. By way of contrast, the final episode of M*A*S*H was watched by three fifths of the country. The centrality of TV, and indeed of any particular medium, has been in decline for three decades. If the pernicious new collectivism is relying on growing media concentration, we’re safe. is similarly and oddly added to the argument, but there is in fact no meta-collectivity algorithm at work here — Popurls just an aggregation of RSS feeds. You might as well go after if that’s the kind of thing that winds you up. And the ranking systems that are aggregated all display different content, suggesting real subtleties in the interplay of algorithm and audience, rather than a homogenizing hive mind at work. You wouldn’t know it, though, to read the broad-brush criticism of Popurls here. And that is the missed opportunity of “The Digital Maoism”: there are things wrong with RSS aggregators, ranking algorithms, group editing tools, and voting, things we should identify and try to fix. But the things wrong with voting aren’t wrong with editing tools, and the things wrong with ranking algorithms aren’t wrong with aggregators. To take the specific case of Wikipedia, the Seigenthaler/Kennedy debacle catalyzed both soul-searching and new controls to address the problems exposed, and the controls included, inter alia, a greater focus on individual responsibility, the very factor “Digital Maoism” denies is at work.

    The changes we are discussing here are fundamental. The personal computer produced an incredible increase in the creative autonomy of the individual. The internet has made group forming ridiculously easy. Since social life involves a tension between individual freedom and group participation, the changes wrought by computers and networks are therefore in tension. To have a discussion about the plusses and minuses of various forms of group action, though, is going to require discussing the current tools and services as they exist, rather than discussing their caricatures or simply wishing that they would disappear.

    Comments (15) + TrackBacks (0) | Category:

    May 25, 2006

    News of Wikipedia's Death Greatly ExaggeratedEmail This EntryPrint This Article

    Posted by Clay Shirky

    Nicholas Carr has an odd piece up, reacting to the ongoing question of Wikipedia governance as if it is the death of Wikipedia. In Carr’s view
    Where once we had a commitment to open democracy, we now have a commitment to “making sure things are not excessively semi-protected.” Where once we had a commune, we now have a gated community, “policed” by “good editors.” So let’s pause and shed a tear for the old Wikipedia, the true Wikipedia. Rest in peace, dear child. You are now beyond the reach of vandals.
    Now this is odd because Carr has in the past cast entirely appropriate aspersions on pure openess as a goal, noting, among other things, that “The open source model is not a democratic model. It is the combination of community and hierarchy that makes it work. Community without hierarchy means mediocrity.”

    Carr was right earlier, and he is wrong now. Carr would like Wikipedia to have committed itself to openess at all costs, so that changes in the model are failure conditions. That isn’t the case however; Wikipedia is committed to effectiveness, and one of the things it has found to be effective is openess, but where openess fails to provide the necessary defenses on it’s own, they’ll make changes to remain effective. The changes in Wikipedia do not represent the death of Wikipedia but adaptation, and more importantly, adaptation in exactly the direction Carr suggests will work.

    We’ve said it here before: Openness allows for innovation. Innovation creates value. Value creates incentive. If that were all there was, it would be a virtuous circle, because the incentive would be to create more value. But incentive is value-neutral, so it also creates distortions — free riders, attempts to protect value by stifling competition, and so on. And distortions threaten openess.

    As a result, successful open systems create the very conditions that require a threaten openess. Systems that handle this pressure effectively continue (Slashdot comments.) Systems that can’t or don’t find ways to balance openess and closedness — to become semi-protected — fail (Usenet.)

    A huge number of our current systems are hanging in the balance, because the more valuable a system, the greater the incentive for free-riding. Our largest and most spontaneous sources of conversation and collaboration are busily being retrofit with filters and logins and distributed ID systems, in an attempt to save some of what is good about openess while defending against Wiki spam, email spam, comment spam, splogs, and other attempts at free-riding. Wikipedia falls into that category.

    And this is the possibility that Carr doesn’t entertain, but is implicit in his earlier work — this isn’t happening because the Wikipedia model is a failure, it is happening because it is a success. Carr attempts to deflect this line of thought by using a lot of scare quotes around words like vandal, as if there were no distinction between contribution and vandalism, but this line of reasoning runs aground on the evidence of Wikipedia’s increasing utility. If no one cared about Wikipedia, semi-protection would be pointless, but with Wikipedia being used as reference material in the Economist and the NY Times, the incentive for distortion is huge, and behavior that can be sensibly described as vandalism, outside scare quotes, is obvious to anyone watching Wikipedia. The rise of governance models is a reaction to the success that creates incentives to vandalism and other forms of attack or distortion.

    We’ve also noted before that governance is a certified Hard Problem. At the extremes, co-creation, openess, and scale are incompatible. Wikipedia’s principle advantage over other methods of putting together a body of knowledge is openess, and from the outside, it looks like Wikipedia’s guiding principle is “Be as open as you can be; close down only where there is evidence that openess causes more harm than good; when this happens, reduce openess in the smallest increment possible, and see if that fixes the problem.”

    People who build or manage large-scale social software form the experimental wing of political philosophy — in the same way that the US Constitution is harder to change than local parking regulations, Wikipedia is moving towards a system where evidence of abuse generates anti-bodies, and those anti-bodies vary in form and rigidity depending on the nature and site of the threat. By responding to the threats caused by its growth, Wikipedia is moving the hierachy+community model that Carr favored earlier. His current stance — that this change is killing the model of pure openess he loved — is simply crocodile tears.

    Comments (21) + TrackBacks (0) | Category: social software

    February 14, 2006

    Powerlaws: 2006 Dance Re-mixEmail This EntryPrint This Article

    Posted by Clay Shirky

    The question of inequality and unfairness has come up again, from Seth’s Gatekeepers posts and subsequent conversation, to pointers to Clive Thompson’s A-Listers article in New York magazine, which article discusses the themes from Powerlaws, Weblogs, and Inequality (though without mentioning that essay or noting that the original powerlaw work was done in 2003.)

    The most interesting thing I’ve read on the subject was in Doc Searls post:

    I’ve always thought the most important thesis in Cluetrain was not the first, but the seventh: Hyperlinks subvert hierarchies.
    What I’ve tried to say, in my posts responding to Tristan’s, Scott’s and others making the same point, is nothing more than what David Weinberger said in those three words.

    I thought I was giving subversion advice in the post that so offended Seth. But maybe I was wrong. Maybe being widely perceived as a high brick in the blogosphere’s pyramid gives my words an unavoidable hauteur — even if I’m busy insisting that all the ‘sphere’s pyramids are just dunes moving across wide open spaces.
    I’ll just add that, if ya’ll want to subvert some hierarchies, including the one you see me in now, I’d like to help.

    The interesting thing to me here is the tension between two facts: a) Doc is smart and b) that line of thinking is unsupportable, even in theory. The thing he wants to do — subvert the hierarchy of the weblog world as reflected in lists ranked by popularity — is simply impossible to do as a participant.

    Part of the problem here is language. Hierarchy has multiple definitions; the sort of hierarchy-subverting that networks do well is routing around or upending nested structures, whether org charts or ontologies. This is the Cluetrain idea that hyperlinks subvert hierarchies.

    The list of weblogs ranked by poularity is not a hierarchy in that sense, however. It is instead a ranking by status. The difference is critical, since what’s being measured when we measure links or traffic is not structure but judgment. When I’m not the CEO, I’m not the CEO because there’s an org chart, and I’m not at the top of it. There is an actual structure holding the hierarchy in place; if you want to change the hierarchy, you change the structure.

    When I’m not the #1 blogger, however, there are no such structural forces making that so. Ranking systems don’t work that way; they are just lists ordered by some measured characteristic. To say you want to subvert that sort of hierarchy makes little sense, because there are only two sorts of attack: you can say that what’s being measured isn’t important (and if it isn’t, why try to subvert it in the first place?), or you can claim that lists are irrelevant (which is tough if the list is measuring something real and valuable.)

    Lists are different from org charts. The way to subvert a list is to opt out; were Doc to stop writing, he would cede his place in the rankings to others. At the other extreme, for him to continue to champion the good over the mediocre, as he sees it, sharpens the very hierarchy he wants to subvert. Huis clos.

    The basic truth of such ranking systems is unchanged: for you to win, someone else must lose, because rank is a differential. Furthermore, in this particular system, the larger the blogsphere grows, the greater the inequality will be between being the most- and median-trafficked weblog.

    All of that is the same as it was in 2003. The power law is always there, any time anyone wants to worry about it. Why the worrying happens in spasms instead of steadily is one of the mysteries of the weblog world.

    The only things that are different in 2006 are the rise of groups and of commercial interests. Of the top 10 Technorati-measured blogs, (Disclosure: I am an advisor to Technorati), all but one of them are either run by more than one poster, or generate revenue from ads or subscriptions. (The exception is PostSecret, whose revenue comes from book sales, not directly from running the site.) Four of the top five and five of the ten are both group and commercial efforts — BoingBoing, Engadget, Kos, Huffington Post, and Gizmodo.

    Groups have wider inputs and outputs than individuals — the staff of BoingBoing or Engadget can review more potential material, from a wider range of possibilities, and post more frequently, than can any individual. Indeed, the only two of those ten blogs operating in the classic “Individual Outlet” mode are at #9 and 10 — Michelle Malkin and Glenn Reynolds, respectively.

    And blogs with business models create financial incentives to maximize audience size, both because that increases potential subscriber and advertisee pools, but also because a high ranking is attractive to advertisers even outside per capita calculations of dollars per thousand viewers.

    (As an aside, there’s a pair of interesting technical questions here: First, how big is the A-list ad-rate premium over pure per-capita calculations? Second, if such a premium exists, is it simply a left-over bias from broadcast media, or does popularity actually create measurable value over mere audience count for the advertiser? Only someone with access to ad rate cards from a large sample could answer those questions, however.)

    In his post Shirky’s Law, Hugh Macleod quotes me saying:

    Once a power law distribution exists, it can take on a certain amount of homeostasis, the tendency of a system to retain its form even against external pressures. Is the weblog world such a system? Are there people who are as talented or deserving as the current stars, but who are not getting anything like the traffic? Doubtless. Will this problem get worse in the future? Yes.

    I still think that analysis is correct. From the perspective of 2003, it’s the future already, and attaining the upper reaches of traffic, for even very committed bloggers, is much harder. That trend will continue. In February of 2009, I expect far more than the Top 10 to be dominated by professional, group efforts. The most popular blogs are no longer quirky or idiosyncratic individual voices; hard work by committed groups beats individuals working in their spare time for generating and keeping an audience.

    Comments (7) + TrackBacks (3) | Category: social software

    December 7, 2005

    Sanger on Seigenthaler’s criticism of WikipediaEmail This EntryPrint This Article

    Posted by Clay Shirky

    Larry Sanger, in regards to John Seigenthaler’s criticism of Wikipedia:

    I have long worried that something like this would happen—from the very start of Wikipedia, in fact. Last year I wrote a paper, “Why Collaborative Free Works Should Be Protected by the Law” (here’s another copy). When Seigenthaler interviewed me for his column, I sent him a copy of the paper and he agreed that it was prophetic. It is directly relevant to the part of Seigenthaler’s column that says: “And so we live in a universe of new media with phenomenal opportunities for worldwide communications and research—but populated by volunteer vandals with poison-pen intellects. Congress has enabled them and protects them.” That was a part of Seigenthaler’s column that bothered me: what exactly does Seigenthaler want Congress to do?

    Comments (82) + TrackBacks (0) | Category: social software

    June 13, 2005

    Wikipedia and slashdot: I was wrongEmail This EntryPrint This Article

    Posted by Clay Shirky

    In Wikipedia, Authority, and Astroturf, I made a guess about the relation between EliasAlucard, who created the Wikipedia entry on SymphonyOS, and esavard, who created the slashdot post about SymphonyOS that Rob Malda added to the front page of slashdot on June 8th.

    Michael Snow has followed up on the issue, and I was wrong. Esavard did not know Elias, and was not acting on concert with him. I owe an apology to both Esavard and Ryan Quinn, the technical lead for Symphony. I apologize to you both.

    The Wikipedia entry itself is more complicated. Snow notes that there is a vote as to whether to delete the SymphonyOS entry from Wikipedia, and its running strongly to leave it. This, in my view, is the right answer; the fact of a Wikipedia entry on a software project should be tied to its existence, rather than being a referendum on other aspects of the project.

    Furthermore, the entry has now been edited to a much more neutral point of view, including, in particular, the deletion of the Trivia section, which was created with a single piece of trivia — that the site had been slashdotted on June 8. There were, in my view, two things wrong with that section: first, if the section really was trivial, it should not, by definition, have been included. If it was not trivial, it should have had another name, but there’s no obvious alternative section for it, since the fact of the slashdotting is unrelated to the technical merit of the effort.

    Second, and more importantly, though the entry mentioned slashdot, it didn’t link to the actual slashdot thread on SymphonyOS, surely far more important than the effect slashdot traffic had on its servers. By mentioning the slashdot effect without pointing to slashdot itself, the Trivia section had the look of an advertisement.

    There’s a long thread on this issue on the Talk page, which is interesting both for Elias’ declarations of autonomy w/r/t to an article he clearly feels he owns (my favorite quote: “So what if this is an advertisement campaign? What are you going to do about it? Nothing.”) and for the view it offers about how the Wikipedia community works generally, with a kind of measured deliberativeness that is quite rare in online communities.

    Comments (1) + TrackBacks (0) | Category: social software

    June 9, 2005

    Wikipedia, Authority, and AstroturfEmail This EntryPrint This Article

    Posted by Clay Shirky

    Slashdot, one of my few ‘must scan three times a day’ sites, has notoriously poorly coordinated and unskeptical editors. As a result, they often run stories that are different from ads only in that /. doesn’t charge for the service.

    Yesterday, though, I saw a new wrinkle: a post sent in by an esavard, using the already pointless sound and fury around the Apple/Intel matchup, to flog a new! improved! YALD (Yet Another Linux for the Desktop) with the goals — who could imagine such audacious goals! — of making Linux easier to use, making applications simpler to create, and just generally making sure everyone has a pony.

    So, to add a little foam to what was pretty small beer, esavard pointed to the Wikipedia entry about their YALD, saying “If you want to know more about Symphony OS, a good starting point is a Wikipedia article describing the innovations proposed by this new desktop OS.

    Now at that point the Wikipedia entry was around three weeks old, had been edited 29 times, and 20 of those edits were by the same user, EliasAlucard. The first edit to that page after being picked up by slashdot (from an IP address with no associated username and with no other history of edits) added a note under the header Trivia: “On 8 June 2005, the Symphony OS website was a victim of the Slashdot effect.” (I deleted this bit of self-aggrandizement just now, though we’ll see how long Elias lets it go.)

    Then, today, when someone pointed out on the related Talk page that our pal EliasAlucard had created a Wikipedia advertisement, he replied “Guess what? No one cares about your opinion of what it looks like. Give it a rest already.”

    This is an interesting kind of spam, or maybe we could call it a reputation hack. I have no way of knowing who esavard is in relation to EliasAlucard, but I am betting they are pretty closely related. They create a Wikipedia page, point to it as if to demonstrate independent interest for the project in their potential slashdot post, then point to the slashdot effect on the Wikipedia page as proof of said independent interest. Voila, an instant trend.

    This is the downside of the mass amateurization of publishing. Since the threshold for exclusion from the Wikipedia is so low, there is almost no value in thinking “Hey, it’s got a Wikipedia article — must be serious.” We have the sense-memory of that way of thinking from the days where it cost money to publish something, and this class of reputation hack relies on that memory to seed the network with highly targeted ads.

    And it’s a hard hack to stop, since it isn’t exactly vandalism. Most articles have only a few editors in the early days, so it’s an attack that doesn’t have an obvious signature either. It’s relatively to see how to defend against vandalism of high-stakes pages, but it’s hard to see how to defend against the creation of pages where so little is at stake for anyone but the advertiser.

    Comments (11) + TrackBacks (0) | Category: social software

    June 8, 2005

    Uncyclopedia and CategoriesEmail This EntryPrint This Article

    Posted by Clay Shirky

    Uncyclopedia, a Wikipedia parody. Hadda happen, and as an added flavor bonus it includes categorization jokes:
    People and Animals
    Writers - Celebrities - Kings of Iceland - Living People - Dead People - Persons of indeterminate mortal status - Wankers - Deities

    Handy Categories
    * Coherent
    * Incoherent
    * Years
    * Everything

    Useless Categories
    * Beans
    * Island of L’aard
    * Morality
    * Typographical Symbols

    Comments (1) + TrackBacks (0) | Category: social software

    June 6, 2005

    Ebay Neg Tool: Cat-Mouse Reputation ProblemsEmail This EntryPrint This Article

    Posted by Clay Shirky

    Two years ago, the economist Paul Resnick wrote about his work on eBay:

    I think there are two problems with the official and community encouragement to resolve disputes before leaving negative feedback. First, patterns of mild dissatisfaction are not recorded, so lots of useful information is lost. Second, sellers have become overly sensitive any negative or even neutral effect because it is so rare. If negative feedback were given 5% or 10% of the time, on average, then sellers would worry about keeping their percentage down, but wouldn’t be as concerned about any particular feedback.

    Negative feedback is rare because it is powerful, as a kind of nuclear option, but as a result, there is a huge information assymetry, where frequently but mildly poor sellers are less likely to be spotted.

    Earlier this year, Toolhaus launched Ebay Negs!, which is the next phase of that cat/mouse game.

    Ebay Negs! lets you view all the negative feedback an eBay user has received. To use it, first highlight the ebay username you want to check with your mouse, then right click and select “Ebay Negs!” You will then be transferred to a page at where all the negative feedback remarks that user have received will be displayed.

    This assumes the very imbalance that Resnick was talking about in 03 — indeed, the comments posted on the tool page all call it a time saver, indicating how little value is placed on even an overwhelming preponderance of positive comments.

    This is analogous to stocks falling when a company exactly meets its earnings target. Since the target was announced by the company itself, and since the accounting tricks that can be used to massage earnings are many, a company that can’t beat a hurdle it sets for itself is assumed to be in trouble. In the same way, if a negative rating on eBay means that all communal norms and attempts at dispute resolution failed, making tools for ferreting out even single examples of negative comments worth the users’s time.

    It’s interesting that as transparent a market as eBay has grown an information assymetry problem all its own, and tools like eBay Neg, while helpful to individual buyers in the short run, and just going to ratchet up the overall pressure more.

    Comments (1) + TrackBacks (0) | Category: social software The day the email diedEmail This EntryPrint This Article

    Posted by Clay Shirky has a brief summary of what happened to the workplace during an email outage:

    So how’d we fare this time around? Well, we’re glad to report that the removal of cold, impersonal email from our workplace reminded us of the value of getting up and talking with each other, reforging lasting connections that will do far more for us than any fancy software system could ever do. Yeah right. And then we went out and planted a tree.

    No, what really happened was a day of false starts, fluttering hands and embarrassed shrugs, vaguely agonizing and occasionally amusing. […] Those with email also became lifelines for meeting organizers — because our calendars are all tied into our email, most of our schedules were instantly erased, leaving harried-looking meeting organizers trying to find people with working email who could peek at the organizers’ schedules, or who’d been invited to a meeting and could reply-all to the invite as a method of reconstructing the list of attendees.

    The key losses to the workplace from the lack of email included not just the data stored in the mail itself, but a critical — and now irreplaceable — social lubricant.

    Comments (0) + TrackBacks (0) | Category: social software

    May 16, 2005

    Ontology Is Overrated: Social advantages in taggingEmail This EntryPrint This Article

    Posted by Clay Shirky

    This spring, I gave a pair of talks on opposite coasts on the subject of categorization and tagging. The first was entitled Ontology Is Overrated, given at the O’Reilly ETech conference in March. Then, in April I gave a talk at IMCExpo called Folksonomies & Tags: The rise of user-developed classification.

    I’ve just put up an edited concatenation of those two talks, coupled with invaluable editorial suggestions from Alicia Cervini. It’s called Ontology is Overrated — Categories, Links, and Tags. Though much of it is not about social software per se, I try to extend the argument that the ‘people infrastucture’ hidden in traditional classification systems is an Achilles’ heel for systems that have to operate at internet scale, and that the logic of tagging overcomes that weakness:

    DSM-IV, the 4th version of the psychiatrists’ Diagnostic and Statistical Manual, is a classic example of an classification scheme that works because of these characteristics [of the user base]. DSM IV allows psychiatrists all over the US, in theory, to make the same judgment about a mental illness, when presented with the same list of symptoms. There is an authoritative source for DSM-IV, the American Psychiatric Association. The APA gets to say what symptoms add up to psychosis. They have both expert cataloguers and expert users. The amount of ‘people infrastructure’ that’s hidden in a working system like DSM IV is a big part of what makes this sort of categorization work.

    This ‘people infrastructure’ is very expensive, though. One of the problem users have with categories is that when we do head-to-head tests — we describe something and then we ask users to guess how we described it — there’s a very poor match. Users have a terrifically hard time guessing how something they want will have been categorized in advance, unless they have been educated about those categories in advance as well, and the bigger the user base, the more work that user education is.

    More at Ontology is Overrated — Categories, Links, and Tags.

    Comments (15) + TrackBacks (0) | Category: social software

    May 11, 2005

    Google Acquires DodgeballEmail This EntryPrint This Article

    Posted by Clay Shirky

    Google, the publicly held Mountain View, CA firm best known for its search engine, has acquired dodgeball, a social networking tool for mobile urbanites and one of the earliest examples of mobile social software.

    The next paragraph contains one hundred w00ts.

    w00t w00t w00t!!! w00t!!!! w00t w00t w00t!! w00t! w00t!!!! w00t w00t!!! w00t!!! w00t! w00t!!! w00t!!!! w00t!!! w00t!!!! w00t!!! w00t!!! w00t w00t!!! w00t!! w00t!!! w00t!! w00t!! w00t!! w00t!!!! w00t!!! w00t!! w00t w00t!!! w00t! w00t w00t!! w00t!!! w00t!! w00t!! w00t! w00t w00t w00t w00t! w00t!! w00t! w00t!! w00t!!! w00t!! w00t!! w00t!!!! w00t!!!! w00t!!!! w00t!!! w00t!!! w00t!!!! w00t!! w00t!! w00t w00t!! w00t!!!! w00t!!! w00t! w00t!!!! w00t w00t w00t!!!! w00t! w00t!! w00t! w00t w00t!!! w00t!!!! w00t!!! w00t!! w00t!!!! w00t!!! w00t!! w00t w00t! w00t!! w00t!! w00t! w00t!!! w00t!!! w00t w00t!!! w00t! w00t!!!! w00t w00t!!! w00t w00t!! w00t! w00t!! w00t!!!! w00t!!! w00t w00t!!! w00t! w00t!! w00t!!!

    Dennis Crowley and Alex Rainert were students of mine at ITP. I’ve watched them build Dodgeball over the last few years, which was both inspiring and instructional. Given the level of thought and effort they’ve put into it, this is really good news, for them and for Google.

    More to say later, but the important thing now is that Dodgeball adds to a really interesting set of ‘sand in the oyster’ issues for Google. Google has historically been information-centric. The content and character of social relations don’t fit well into that view of the world, but matter, a lot, to users. (As we’ve often said around here, community != content.)

    Gmail, Orkut, and now Dodgeball all touch this issue. Dodgeball in particular is built on a mix of three different kinds of maps: maps of location (118 rivington St), maps of place (a bar called The Magician), and maps of social environment (“I’m here. Where are my friends?”) By mixing them, Dodgeball mingles informational and social aspects of a user’s life into something more valuable than either of those things in isolation.

    As Brewster Kahle says ‘If you want to solve, hard problems, have hard problems.” The integration of information-centric and social-centric views of the world will be awfully valuable, if Google gets them right.

    So congrats to Dodgeball and to Google!

    Comments (11) + TrackBacks (0) | Category: social software

    May 2, 2005, and an answer to Tim BrayEmail This EntryPrint This Article

    Posted by Clay Shirky

    Some of us talking about tagging a have launched a group weblog called “You’re It: A blog on tagging,” at (Authors are Christian Crumlish, David Weinberger, Don Turnbull, Jon Lebkowsky, Kaliya Hamlin, Mary Hodder, Timo Hannay, and me.)

    My introductory post there pointed to my earlier tagging articles at M2M. My first real post is a response to Tim Bray’s question: “Are there any questions you want to ask, or jobs you want to do, where tags are part of the solution, and clearly work better than old-fashioned search?” I think the answer is Yes, and try to delinate some of the reasons why.

    (And, because tagging straddles social and organizational concerns, I’ll have to figure out when to post here vs there, but I’m planning to x-post pointers generally.)

    Comments (2) + TrackBacks (0) | Category: social software

    April 19, 2005

    Sanger, Part IIEmail This EntryPrint This Article

    Posted by Clay Shirky

    The second half of Larry Sanger’s piece on Wikipedia and Nupedia is up. I haven’t even read the whole thing yet, but it’s fascinating, especially as it goes considerably deeper into the governance issues.

    It is one thing to lack any equivalent to “police” and “courts” that can quickly and effectively eliminate abuse; such enforcement systems were rarely entertained in Wikipedia’s early years, because according to the wiki ideal, users can effectively police each other. It is another thing altogether to lack a community ethos that is unified in its commitment to its basic ideals, so that the community’s champions could claim a moral high ground. So why was there no such unified community ethos and no uncontroversial “moral high ground”? I think it was a simple consequence of the fact that the community was to be largely self-organizing and to set its own policy by consensus. Any loud minority, even a persistent minority of one person, can remove the appearance of consensus.

    Read it.

    Comments (2) + TrackBacks (0) | Category: social software

    April 18, 2005

    Sanger on WikipediaEmail This EntryPrint This Article

    Posted by Clay Shirky

    Over on slashdot, Larry Sanger has published the first in an N-part series (N>1) on the early history of the Wikipedia (and the failed Nupedia) projects.

    It has all of the benefits and disadvantages of being written by someone present at the creation: the details of early choices are fascinating, while the score-settling is a bit tedious. (He takes Daniel Pink to task for misquoting the tiny number of finished Nupedia articles, even though the gap between Wikipedia and Nupedia covers orders of magnitude.)

    What’s most fascinating, though, is not the historical element, but Sanger’s own position. He understands why Wikipedia works and Nupedia didn’t, and yet is constantly maintaining that the Wikipedia would benefit from being more like the planned Nupedia:

    This point bears some emphasis: Wikipedia became what it is today because, having been seeded with great people with a fairly clear idea of what they wanted to achieve, we proceeded to make a series of free decisions that determined the policy of the project and culture of its supporting community. Wikipedia’s system is neither the only way to run a wiki, nor the only way to run an open content encyclopedia. Its particular conjunction of policies is in no way natural, “organic,” or necessary. It is instead artificial, a result of a series of free choices, and we could have chosen differently in many cases; and choosing differently on some issues might have led to a project better than the one that exists today.

    I have a hard time understanding how a loosely bound community, choosing among available options, isn’t an organic process, but Sanger has always been convinced that setting and enforcing a Nupedian-style respect for authority was a) possible for Wikipedia and b) desirable for Wikipedia. (I’ve disagreed with Sanger on both points in the past, but based on a less complete re-telling than this looks to be.)

    In any case, since the whole piece isn’t yet published, it’s too soon to see how the various themes will develop, but for anyone following Wikipedia, this will be a key piece of writing.

    Comments (6) + TrackBacks (0) | Category: social software

    CFP: Wikimania 2005Email This EntryPrint This Article

    Posted by Clay Shirky

    Wikimania 200, the First International Wikimedia Conference will be held in Frankfurt from August 4-8, 2005 to 8 August 2005.

    Two key upcoming dates are:
    - May 10 - Abstract deadline for panels, papers, posters and presentations [Notification: by May 25]
    - May 30 - Submission deadline for research paper drafts

    Says the submission page:
    Original research is welcome, but not required. Be bold in your submissions! Wikimania is meant to be both a scientific conference and a social event. Relevant topics include:

    * Wiki research: How do wikis, and the Wikimedia wikis in particular, operate? Which processes scale and which ones don’t? What kinds of people or social structures are well-suited to wikis? How does introducing a wiki into existing project groups change group dynamics?
    * Wiki sociology: What motivates Wikimedians and what drives them away? Who are they, anyway? And where do they come from?
    * Wiki critics: Critical positions are welcome: why Wikipedia will never be an encyclopedia, why Wikinews can never substitute newspapers, why amateurs shouldn’t be allowed to edit, and so forth.
    * Wiki technology ideas: What can we do to address perceived and real problems, for example, peer review? How can we provide better-nuanced or more immediate user feedback?
    * Wiki software ideas […]
    * Wiki community ideas […]
    * Wiki project ideas […]
    * Wiki content ideas […]
    * Multimedia […]
    * Free knowledge […]
    * Collaborative writing […]
    * Multilingualism […]

    Comments (0) + TrackBacks (0) | Category: social software

    April 5, 2005

    Banning blogging, 'Toothing, and YozEmail This EntryPrint This Article

    Posted by Clay Shirky

    Phil Gyford, in With great audiences…, wonders what it takes to get a story propagated in the weblog world, and is afraid that the answer is merely ‘attention grabbing headline + a patina of Old Media validity.’ He writes about a “banning blogging” story picked up from the traditional press, where the weblog…

    …got carried away with the newspaper’s headline, repeating it in theirs even though a cursory read of the newspaper article reveals that no one “banned blogging.” The newspaper claims the principal doesn’t think blogging is educational, and Cory could certainly have criticised him for this alone, although it would make for a less dramatic post. The repetition of the lie about the principal banning blogging, rather than his apparent opinion, is possibly also what prompted a reader to suggest people should email the principal to complain.

    Phil posts about BoingBoing, but the pattern is quite general — you can see misleading posts like San Francisco Attempts to Regulate Blogging almost daily on slashdot.

    The pressure to give things a dramatic headline, online or off, is tremendous, because if you don’t get readers with the headline, you won’t get them at all. This leads, in the weblog world, to a curious moral hazard, where fact-checking can be left to the furthest upstream source. “Well, if the Osceola Star-Ledger, with their enormous resources, can’t fact-check the article, how can I be expected to???” And so we get ourselves in high dudgeon at injustices that may never have happened, because they are the kind of thing we would hate if they had happened.

    Thiscontrasts with with the magnificent distributed fact-checking done elsewhere, as with the Trent Lott or Dan Rather investigations. The choice to fact-check vigorously, even when a story is reported by well-funded news outlets, seems only to happen when the writers in question disagree with the story, while the decision to accept the fact-checking of any traditional media outlet, in order to be able to fast-forward to the aforementioned high dudgeon, seems to come when the weblogger likes repeating or even amplifying the claims made further upstream.

    Which brings me to ‘toothing.

    ‘Toothing was the craze for arranging on-the-spot trysts among users of Bluetooth-capable cellphones, as reported by Wired last March. Except ‘toothing was a hoax, as the perpetrator revealed after seeing this slashdot thread.

    It seems harmless, except that many of the subsequent references weren’t about ‘toothing per se (understandable, as there was nothing to study), but rather referenced ‘toothing as one member of a set of activities mobile technologies enabled. ‘Toothing went from being a thing to being a touchstone for reasoning about mobile technologies generally.

    A couple years ago, I spent some time on the trail of the urban legend that half the world had never made a phone call. While ‘toothing was never likely to acheive that degree of saturation, it was, like the ‘half the world’ phrase, a distortion not only in itself but as the avatar of larger social patterns.

    I checked the M2M archives, and to my relief, we didn’t write about ‘toothing, though probably not out of any native skepticism, or we would have written to de-bunk it. Yoz Grahame is the only person I know of who got this right, in the voluptously titled Sex-Crazed Brits Just Doing It Everywhere, Like, Everywhere Man, You Can’t Stop Them, They’re Like Dogs In Heat Or Something, And Dude, I Gotta Get Me Some Of That:

    Pausing only to spill some famous London ale down the front of his XXL-sized rugby shirt, Barry outlined some key points in the rapidly-evolving lexicon of British desire. “So what you do, right, is you spot a nice tart over by the bar and you think, lovely, I’ll have a bit of that. And you tip her the wink, you know? And then, if she looks back at you, she’s gagging for it.”

    “Just like Bluetooth signalling,” I commented as I tapped hurried notes into my Zaurus. “Ingenious!”

    One lesson we could all take from this is “Pay more attention to Yoz”, which couldn’t hurt, but a better motto is ‘WWYD?’ Note that he didn’t fact-check the ‘toothing story, he sense-checked it. The thing wrong with the toothing story isn’t that the participants of the toothign scene aren’t IDed, it’s that the story itself doesn’t make any sense. Most of us will not be able to afford the calling and re-calling of sources to double-check a quote, but all of us can ask ourselves, just before we hit Submit, ‘Is this true?’

    And the time we should be most careful to do that is if we feel really satisfied with what we’ve written — “How dare the House of Representatives propose a mandatory bar code tattooed on the foreheads of liberal bloggers!!! Must. Denounce. Now.”

    All the phrases we use to separate the weblog world from other media outlets weaken with elapsed time — old media, new media, traditional media, all of it suggests that newcomers join the club when they’ve been around long enough to be familiar. As weblogs continue their symbiosis with the forms of media that went before, we will make ourselves targets of truly malevolent hoaxes if we simply decide to repeat what we agree with. The echo chamber is of far less danger overall than unchecked amplification.

    Comments (16) + TrackBacks (1) | Category: social software

    March 31, 2005

    Stuff that gets spammed, part NEmail This EntryPrint This Article

    Posted by Clay Shirky

    I hardly know what to make of this — has discovered that WordPress, the great open source blogging platform, has been pimping out it’s highly rated home page to an SEO (Search Engine “Optimization”) firm, effectively selling the community capital it built up to spammers by “publishing” articles that are hidden to users but visible to spiders.

    There’s also a bizarre defense of this practice on Planet Wordpress, on the grounds that WordPress needed money to grow, and wasn’t getting it from donations.

    This is such an interesting and uncharted area — as the net gets bigger and karma, previously bottled up in human relations, becomes convertible for real currency, in everything from ZeroDegrees/ style spam to real sales of virtual characters to this, we are going to have to find ways to defend against this sort of karmic hijacking.

    Comments (6) + TrackBacks (0) | Category: social software

    March 30, 2005

    Hedlund Reads Yahoo 360Email This EntryPrint This Article

    Posted by Clay Shirky

    Marc Hedlund examines Yahoo 360 using Lessons from Lucasfilm’s Habitat (Best. Essay. EVAR.) as his guide, since one of the authors of LLH was Randall Farmer, one of the creators of Y360.

    Hedlund comes away skeptical, noting that the lack of interoperable standards and widely available APIs violate some of the LLH tenets, as with the LLH assertion “Data communications standards are vital.”

    Those who do not learn the lessons of Habitat are doomed to repeat them, indeed. In 360, we see this problem, the lack of communication standards, expressed most acutely in the IM sidebar, which lists the online status of all of your buddies — excuse me, your Yahoo buddies. You can IM them and send them messages in the system (messages which are like email but not email, so that you have yet a third voice with which to speak to a subset of your friends). Why do I need a web view on my IM buddy list when I have that list on my computer already? If 360 becomes your home, perhaps that would be useful.

    The fault here is easy to see with a thought experiment. Let’s say Yahoo 360 were implemented today by a startup, a company without ties or loyalty to an existing body of users. Would they make the same decision? Is it in the best interest of new users to 360 to have their Yahoo buddies be the only ones available for sharing, or is that more in the interest of Yahoo?

    Data communication standards are vital, and the lack of them has kept IM from becoming a platform for innovation as email and the web have become. 360 suffers from the lack of a standard just as would any startup, but it hasn’t sought out a solution, as would a company that needed new users to survive.

    I’m less convinced than Marc that this is fatal, starting from the premise that much human congress happens within essentially arbitrary divisions like this one — you know your co-workers on the 5th floor or your neighbors on your street better than you know the people on the 6th floor, or on the next block over.

    However, I am, like Marc, convinced that this ‘proprietary standards and messaging’ weakness will prevent 360 from becoming a complete digital hub. It may simply be a good fusion of Orkut and fotolog.

    Comments (3) + TrackBacks (0) | Category: social software

    March 28, 2005

    Business data pointEmail This EntryPrint This Article

    Posted by Clay Shirky

    Just got email from a headhunter looking for leads for a ‘VP of Social Computing,’ whose job will include building and managing a staff of 75-80 (!) people.

    No word on what the company is (though it’s obviously large) and the work doubtless includes a number of more broadcast-oriented efforts as well (e.g. weblogs and RSS as publishing tools as well as conversational ones,) but it was interesting to a) see a VP level hire in this area and b) to see how large a staff is being imagined.

    Comments (4) + TrackBacks (0) | Category: social software

    March 22, 2005

    Allen on altruism and group sizeEmail This EntryPrint This Article

    Posted by Clay Shirky

    Interesting speculation over on Life With Alacrity about Dunbar, Altruistic Punishment, and Meta-Moderation — Allen disucsses work on an agent-based simulation that suggests a phase transition from cooperating groups to Tragedy of the Commons scenarios at ~15 people, a much lower number than many of us assumed commons-based problems arose. (My assumptions had been closer to 25.)

    This is a very interesting result. To explain it in different terms, if you have a system that depends on sharing some commons and there are no process or trust metrics, a group as small as 16 may find themselves not cooperating very effectively.
    The idea of commons can be as simple as how much speaking time participants in a meeting share. The time that each participant uses during the meeting can be considered the shared “commons”. If there are no enforced rules, with a group size of 16 there will inevitably be someone who will abuse the time and speak more than their share.

    The one big caveat is that this is based on studies of agents, not actual humans, making the results fairly provisional. However, the study at least points to some experimental designs that could be tried with real live groups.

    Comments (0) + TrackBacks (0) | Category: social software

    March 9, 2005

    One World, Two Maps (thoughts on the Wikipedia debate)Email This EntryPrint This Article

    Posted by Clay Shirky

    When thinking about technological change, there are two kinds of people, or rather, people with two kinds of maps of the world — radial, and Cartesian. Radial maps are circular, and express position in relative coordinates — angle and distance — from the center. Cartesian maps are grids, and express position in absolute coordinates. Each of the views has good and bad points on their own, but reading danah on Wikipedia has made me contemplate the tendency of the two groups to talk past each other.

    Radial people assume that any technological change starts from where we are now — reality is at the center of the map, and every possible change is viewed as a vector, a change from reality with both a direction and a distance. Radial people want to know, of any change, how big a change is it from current practice, in what direction, and at what cost.

    Cartesian people assume that any technological change lands you somewhere — reality is just one point of many on the map, and is not especially privileged over other states you could be in. Cartesian people want to know, for any change, where you end up, and what the characteristics of the new landscape are. They are less interested in the cost of getting there.

    Radial people tend to think more about change than end state, and more about local maxima (are things getting better?) than about a global maximum (are things as good as they could be?) Cartesian people think more about end state than change, and more about global than local maxima.

    I am a radial person; danah is a Cartesian person. Cory Doctorow is a radial person; Nicholas Negroponte is a Cartesian person. Richard Gabriel is radial; Alan Kay is Cartesian. This is not a question of technology but outlook. Extreme Programming is a radial method; the Capability Maturity Model is Cartesian. Open Source groups tend towards radial methods, closed source groups tend towards Cartesian methods. It’s incrementalism vs. planned jumps, evolution vs. directed labor.

    When we make mistakes, radial people tend to overestimate the value of incrementalism, and to underestimate the gap between local and global maxima. When they make mistakes, Cartesian people tend to underestimate the cost in moving from reality to some imagined alternate state, and to overestimate their ability to predict what a global maximum would look like.

    This is, plainly, an overstatement of the Everyone is a Pirate or a Ninja sort, but I think there is a grain of truth to it — when Negroponte rails against incrementalism, there’s an interesting discussion to be had about how big he thinks a change has to be before it no longer counts as an increment, but there’s no denying that he is advancing different idea about technological improvement than Gabriel is in his Worse Is Better argument. There’s a similar difference in the way danah or Matt Locke talk about Wikipedia vs. the way Cory or I do. There are lots of blended cases, but the basic impulse is different.

    This has been an era of radial triumphs, because radial maps tend to be better guides to large, homeostatic systems. When thinking about change on the internet, the tools that have been driven by a thousand tiny adoptions and alterations have tended to be more important than the tools designed in advance to change the landscape. However, radial vision requires that someone, somewhere, have pushed through a large, destabilizing change, in order for the radial people to be playing in new terrain with lots of unexplored local maxima. Shawn Fanning could only change the world in 1999 because Vint Cerf changed the world in 1969.

    Bob Spinrad, who used to run PARC (an echt Cartesian organization) said “The only institutions that fund pure research are either monopolies or think they are.” Cartesian development is economically draining, and never pays for itself in the short term, so it’s no accident that R&D happens outside traditional profit maximizing institutions, whether governmental, academic, or monopolists.

    You can see the differences in the two worldviews most clearly when we argue across that gap. I literally cannot understand danah’s complaints; I read “The problem that i’m having with the Wikipedia hype is the assumption that it is the panacea for it too has its problems”, and I wonder who she’s talking about. The radialists praising the Wikipedia are not saying it’s perfect, or even good in any absolute sense — we don’t ever talk about absolute quality.

    Wikipedia interests us because it’s better, and sustainably better, than what went before — it’s a move from a simple product (“Pay us and we’ll write an encyclopedia”) to a complex system, where a million differing, internal motivations of the users and contributors are causing an encyclopedia to coalesce. How cool is that? (The radialist motto…)

    But danah and Matt cannot understand our enthusiasm. From the Cartesian point of view, the thing that would excite you would be dramatic change to a new state. Radialists never say things like ‘panacea’ or ‘utopia’, but the Cartesians hear us saying those things, or think they do, because otherwise what would the fuss be about? Mere incrementalism is nothing more than a Panglossian fetishization of reality, and excitement about a technological change that doesn’t create a dramatic new equilibrium is simply hype, from the Cartesian point of view.

    And so, when they see us high-fiving over Wikipedia, the Cartesians think we’ve taken leave of our senses, and, more to the point, they think we’ve misunderstood what is happening. They then launch a corrective set of arguments, pointing out, for example, that Wikipedia still leaves unanswered questions about social exclusion. But this, from a radialist point of view, is no more meaningful than pointing out that Wikipedia doesn’t cure skin cancer — no one ever said it would. Anything that was bad at Point A and is still bad at Point B gets factored out of the radialist critique. Any change where most of the bad things are still bad but a few of the bad things are somewhat less bad seems like a good thing to us, and if it can happen in a way that requires less energy, or better harnesses individual motivation, that seems like a great thing.

    And so we go, back and forth, tastes great, less filling. We want to ask them why they aren’t excited about Wikipedia, since it is, to us, so obviously progress, but they want to know “Progress towards what?” They can’t even read their map without a posited end state. And they want to ask us why we’re not concerned about where all this is going, but we don’t have an answer to that question, because our maps only show us the way up the next hill, not what we’ll see when we get there.

    There’s no answer to any of this — as Grandma used to say, “Both your maps are nice.” But after months of cognitive dissonance — I both admire and love danah; what she’s saying about Wikipedia simply confuses me — I think now have a way of understanding why the current conversation seems so unmoored.

    Comments (7) + TrackBacks (0) | Category: social software

    March 1, 2005

    Matt Locke on folksonomiesEmail This EntryPrint This Article

    Posted by Clay Shirky

    Wonderful Matt Locke piece on folksonomies, which introduces not one but two substantial ideas to the debate:

    Perhaps this illustrates the limit of folksonomies - they are only useful in a context in which nothing is at stake. [Emphasis his] Folksonomies are, in essence, just vernacular vocabularies; the ad-hoc languages of intimate networks. They have existed as long as language itself, but have been limited to the intimate networks that created them. At the point in which something is at stake, either within that network or due to its engagement with other networks (legal, financial, political, etc) vernacular communication will harden into formal taxonomy, and in this process some of its slipperiness and playfulness will be lost.

    He relates this to the idea of play from finite and infinite games. (I’m more optimistic about the shift here than he is, for reasons I’ll discuss below, but I think he’s spot on about the gap between palyful and serious categorization.)

    The other idea, from Bowker and Star’s marvelous Sorting Things Out, is about the inherent tension in classification generally:

    Bowker and Star identify three values that are in competition within classfication structures: comparability, visibility and control. Folksonomies have elevated visibility, but at the expense of comparability (being able to translate classifications across taxonomies or contexts) and control (the ability of the classification to limit interpretation, rather than interpret ‘emergent’ behaviour). Whilst nothing is at stake, and there is little lost by not being able to transfer taxonomies from one context to the other, or users are not disadvantaged by the need to independently assess and contextualise meaning, folksonomies will provide a useful service.

    Just a fantastic post.

    The only place I vary from Matt (it’s not even a disagreement, really, just a prediction about the future) is in the eventual value of folksonomy. He likens folksonomies to vernacular vocabularies, but this doesn’t describe their first-order importance, at least not where systems like are concerned.

    Here’s what’s radical about what protends: My vocabulary on folksonomy is personal, not vernacular — no one knows or needs to know which class I’m talking about when I tag something ‘class’, or that I use LOC to mean Library of Congress. This isn’t the same as, say, the dictionary of thieves slang from the mid-18th c. because no one else needs to know my bookmark system, and I don’t need to know anyone else’s, or, to quote Adam Smith: “It is not from the benevolence of the butcher, the brewer, or the baker, that we can expect our dinner, but from their regard to their own interest.”

    This is really, truly different, because it uses the intiution of markets — aggregate self-interest creates shared value. Locke points to the loss of control as one of the downsides of folksonomic classification (at least in its del-style form), but there are significant upsides as well. The LOC has no top-level category for queer issues, but does, because its users want it to.

    By forcing a less onerous choice between personal and shared vocabularies, shows us a way to get categorization that is low-cost enough to be able to operate at internet scale, while ensuring that the emergent consensus view does not have to be pushed onto any given participant.

    Which is why it mystifies me that both Matt and danah are so concerned with exclusion — who’s excluded here, who isn’t also excluded from using the internet generally? Put another way, is anyone excluded from using who has better representation in other classification schemes?

    The answer is “If you don’t like the way something is tagged, tag it yourself. No one can tell you not to.” Prior ot, controlled vocabularies were almost inevitably vocabularies that pushed the politics of the creators onto the users; that is upended here.

    Comments (9) + TrackBacks (0) | Category: social software

    February 28, 2005

    Who's afraid of Wikipedia?Email This EntryPrint This Article

    Posted by Clay Shirky

    danah said, in Academia and Wikipedia, “All the same, i roll my eyes whenever students submit papers with Wikipedia as a citation.”

    I didn’t comment on this at the time, but grading papers over the weekend, I had a student cite the Wikipedia for the first time, referencing its entry on the OSI Reference Model. Seeing it in the footnotes, I wondered what the fuss was about. The Wikipedia article is a perfectly good overview of the Reference Model, and students should document, to the extent they are able, the sources of their research. When they have learned something from the Wikipedia, in it goes; to exclude it would in fact be dishonest.

    Curiously, the Wikipedia reference came in the same week that another student was referring to Walter Benjamin’s The Work of Art in the Age of Mechanical Reproduction, an essay that is tremendously influential and, in a bunch of non-trivial ways, wrong about the inherent politicization of reproducible art, and especially of film. I’m much more worried about students overestimating the value of the Benjamin essay, because of its patina of authority, than I am about them overestimating the value of the Wikipedia as a source for explaining the 7-layer networking model.

    And I assume I am hardly alone in the academy. Hundreds, if not thousands of us must be getting papers this year with Wikipedia URLs in the footnotes, and despite the moral panic, the Wikipedia is a fine resource on a large number of subjects, and can and should be cited in those cases. There are articles, as danah has pointed out, where it would be far better to go to the primary sources, but that would be as true were a student to cite any encyclopedia. If someone cited the Wikipedia to discuss Benjamin’s work, I’d send them back to the trenches, but I would also do that if they cited Encyclopedia Britannica.

    To borrow some Hemingway, this is how the academy will get used to Wikipedia — slowly, then all at once.

    Comments (10) + TrackBacks (0) | Category: social software

    February 21, 2005

    Social Physics (.org)Email This EntryPrint This Article

    Posted by Clay Shirky

    Fascinating new effort called Social Physics, affiliated with Berkman, with two large goals:

    - Create a robust, multi-disciplinary, multi-constituency community for addressing, vetting and conducting experiments in such issues as privacy, authentication, reputation, transparency, trust building and information exchange.

    - Develop a reusable, open source software framework based on the Eclipse Rich Client Platform that provides core services including: identity management, social network data models, authentication management, encryption, and privacy controls. On top of this framework we are also developing a demo app that provides identity management and social networking functions, tools to create peer-to-peer identity sharing and facilities to support communities of interest around emerging topics.

    I’m generally skeptical of identity management — it has the same hollow ring as knowledge management — but since the focus here is on trust building, rather than simple transactions that treat trust as a binary condition or simple threshold, this will be worth watching.

    Comments (0) + TrackBacks (0) | Category: social software

    February 16, 2005

    Social Software: Stuff that gets you laid...Email This EntryPrint This Article

    Posted by Clay Shirky

    JWZ has a great rant on the brokenated nature of groupware, written after a conversation with a friend building an open-source groupware project:

    If you want to do something that’s going to change the world, build software that people want to use instead of software that managers want to buy.

    When words like “groupware” and “enterprise” start getting tossed around, you’re doing the latter. You start adding features to satisfy line-items on some checklist that was constructed by interminable committee meetings among bureaucrats, and you’re coding toward an externally-dictated product specification that maybe some company will want to buy a hundred “seats” of, but that nobody will ever love. With that kind of motivation, nobody will ever find it sexy. It won’t make anyone happy.

    He then offered a more upbeat definition of social software than ‘stuff that gets spammed’:

    But with a groupware product, nobody would ever work on it unless they were getting paid to, because it’s just fundamentally not interesting to individuals.

    So I said, narrow the focus. Your “use case” should be, there’s a 22 year old college student living in the dorms. How will this software get him laid?

    That got me a look like I had just sprouted a third head, but bear with me, because I think that it’s not only crude but insightful. “How will this software get my users laid” should be on the minds of anyone writing social software (and these days, almost all software is social software).

    “Social software” is about making it easy for people to do other things that make them happy: meeting, communicating, and hooking up.

    Comments (8) + TrackBacks (1) | Category: social software

    February 3, 2005

    Tagging's power lawEmail This EntryPrint This Article

    Posted by Clay Shirky

    Ben Hyde looks at four popular bookmarks at and plots how many times each is tagged with the same word. E.g, BoingBoing is tagged as “blog” 200 times and as “news” 90 times. The curve is that of a classic power law: The most frequently used tags are used waaaay more frequently than lesser-used tags.

    Ben stresses that four bookmarks don’t constitute a significant sample, but wouldn’t we expect a folksonomy to assume the shape of a power law distribution?

    Comments (0) + TrackBacks (0) | Category: social software

    February 1, 2005

    Tags run amok!Email This EntryPrint This Article

    Posted by Clay Shirky

    Back In The Day, when I was trying to explain what I meant when I was talking about social software, but before Coates pulled my fat out of the fire by doing the work for me, I had all these wicked abstruse definitions that made everyone’s eyes glaze over.

    The only definition I ever found that created the lighbulb moment I was feeling was “Social software is stuff that gets spammed.” Not a perfect definition, but servicable in its way.

    Comes now tag spam from user DaFox, as if to illustrate the principle — a single link, whose extended description is a variation on the form “Best site EVAR!” and who has tagged the site (for his or her own retrieval doubtless) with the following tags:
    .imported .net 10placesofmycity 2005 3d academic accessibility activism advertising ai amazon amusing animation anime apache api app apple apps architecture art article articles astronomy audio backup bands bittorrent blog blogging blogs book bookmark books browser business c canada career china christian clothing cms code coding collaboration color comic comics community computer computers computing cooking cool creativity css culture daily database deals …

    The list includes another couple hundred items — that must be some site, containing as it does not just the above listed items but info relevant to Ruby programming, New York City, typography, economics, and porn. DaFox is the Canter and Siegal for the social software generation.

    Comments (6) + TrackBacks (1) | Category: social software

    Folksonomy: The Soylent Green of the 21st CenturyEmail This EntryPrint This Article

    Posted by Clay Shirky

    In What Do Tags Mean, Tim Bray says “There is no cheap metadata” (quoting himself from the earlier On Search.) He’s right, of course, in both the mathematical sense (metadata, like all entropy-fighting moves, requires energy) and in the human sense — in On Search, he talks about the difficulties of getting users to enter metadata.

    And yet I keep having this feeling that folksonomy, and particularly amateur tagging, is profound in a way that the ‘no cheap metadata’ dictum doesn’t cover.

    Imagine a world where there was really no cheap metadata. In that world, let’s say you head on down to the local Winn-Dixie to do your weekly grocery accrual. In that world, once you pilot your cart abreast of the checkout clerk, the bargaining begins.

    You tell her what you think a 28 oz of Heinz ketchup should cost. She tells you there’s a premium for the squeezable bottle, and if you’re penny-pinching, you should get the Del Monte. You counter by saying you could shop elsewhere. And so on, until you arrive at a price for the ketchup. Next out of your cart, the Mrs. Paul’s fish sticks…

    Meanwhile, back in the real world, you don’t have to do anything of the kind. When you get to the store, you find that, mirabile dictu, the metadata you need is already there, attached to the shelves in advance of your arrival!

    Consider what goes into pricing a bottle of Heinz: the profit margin of the tomato grower, the price of a barrel of oil, local commercial rents, average disposable incomes in your area, and the cost of providing soap in the employee bathrooms. Yet all those inputs have already been calculated, and the resulting price then listed on handy little stickers right there on the shelves. And you didn’t have to do any work to produce that metadata.

    Except, of course, you did. Everytime you pick between the Heinz and the Del Monte, it’s like clicking a link, the simplest possible informative transaction. Your choice says “The Heinz, at $2.25 per 28 oz., is a better buy than the Del Monte at $1.89.” This is so simple it doesn’t seem like you’re producing metadata at all — you’re just getting ketchup for your fish sticks. But in aggregate, those choices tell Del Monte and Heinz how to capture the business of the price-sensitive and premium-tropic, respectively.

    That looks like cheap metadata to me. And the secret is that that metadata is created through aggregate interaction. We know how much more Heinz ketchup should cost than Del Monte because Heinz Inc. has watched what customers do when they raise or lower their prices, and those millions of tiny, self-interested transactions have created the metadata that you take for granted. And when you buy ketchup, you add your little bit of preference data to the mix.

    So this is my Get Out of Jail Free card to Tim’s conundrum. Cheap metadata is metadata made by someone else, or rather by many someone elses. Or, put another way, the most important ingredient in folksonomy is people.

    I think cheap metadata has (at least) these characteristics:

    1. It’s made by someone else
    2. Its creation requires very few learned rules
    3. It’s produced out of self-interest (Corrolary: it is guilt-free)
    4. Its value grows with aggregation
    5. It does not break when there is incomplete or degenerate data

    And this is what’s special about tagging. Lots of people tag links on, so I gets lots of other people’s metadata for free. There is no long list of rules for tagging things ‘well,’ so there are few deflecting effects from transaction cost. People tag things for themselves, so there are no motivation issues. The more tags the better, because with more tags, I can better see both communal judgment and the full range of opinion. And no one cares, for example, that when I tag things ‘loc’ I mean the Library of Congress — the system doesn’t break with tags that are opaque to other users.

    This is what’s missing in the “Users don’t tag their own blog posts!” hand wringing — they’re not supposed to. Tagging is done by other people. As Cory has pointed out, people are not good at producing metadata about their own stuff, for a variety of reasons.

    But other people will tag your posts if they need to group them, find them later, or classify them for any other reason. And out of this welter of tiny transactions comes something useful for someone else. And because the added value from the aggregate tags is simply the product of self-interest + ease of use + processor time, the resulting metadata is cheap. It’s not free, of course, but it is cheap.

    Comments (5) + TrackBacks (0) | Category: social software

    January 30, 2005

    del.icio.uus Tag StemmingEmail This EntryPrint This Article

    Posted by Clay Shirky

    Matt Biddulph has put up a tag stemmer, which will take your username (or indeed any username) and point out the possible inconsistencies based on word stemming (tag/tags/tagging, etc.) It will also take a URL, scan all users who tagged it, and look for the same thing.

    What it will not (yet) do is return the full list of tags sorted by frequency, listing both tags with alternate stems and those without, but I assume this is simply a matter of time.

    This is part of why I think tags are such a big deal — they are annotations for the only native unit of accounting the Web has, namely the URL; the annotations are themselves URLs that can be further annotated; and they are simple enough in both concept and technical design that third-party services like ‘stemtags’ can easily be built on top of the system.

    Comments (4) + TrackBacks (0) | Category: social software

    Looking for avid wikipedia usersEmail This EntryPrint This Article

    Posted by Clay Shirky

    Press request by proxy — for someone working on a story about the wikipedia, is there anyone out there whou would consider themselves either an avid user of wikipedia, or an avid contributor to Wikipedia? If so, drop me a line.

    Comments (0) + TrackBacks (0) | Category: social software

    January 29, 2005

    Folksonomy is better for cultural values: A response to danahEmail This EntryPrint This Article

    Posted by Clay Shirky

    danah’s great piece on cultural issues in folksonomy gets to a key piece of the debate, namely that we can’t talk about categorization issues like accuracy without also talking about the culture that created the categories. However, I feel a curious disconnect between her exposition of the issues and her tone. There seems to be skepticism about folksonomic tagging in her post (though possibly it is just the reflexive skepticism of the academy.)

    In any case, I want to point out that, for almost all the issues she raises, those characteristics are worse, much worse, in formal classification schemes than in folksonomies in general, and folksonomic tagging in particular.

    At the risk of running good writing through the sausage grinder (and re-ordering it to boot) her list of issues is broadly:

    1. There are perspective problems (e.g. the tag ‘me’ on Flickr.)
    2. Tags can be gamed (in the manner of the MLK/Technorati tags)
    3. Classification schemes are always culturally dependent
    4. Many terms are contested
    5. Some words cannot be simply translated literally
    6. Some words have multiple or conflicting meanings

    All the items on that list are true, and items 1 and 2 on the list are genuine design issues. System gaming is an issue, and can be fought with, inter alia, opacity of ranking method (the Google way), reputation markets (the Ebay way), continual post-hoc edits (the Wiki way), and so on. Each of these solutions may be tried in different places where folksonomy takes hold.

    And for the relative tag problem, there may be a small enough number of those kind of tags — me, toread, unfiled, etc — that we can make a dictionary filter. But the relativity can also be interesting when crossed-tabbed with the identity of the tagger; I don’t want ‘toread’ or ‘funny’ generally, but I do want Liz’s ‘toread’ tags, and Matt Webb’s ‘funny’ links.

    Items 3-6 on that list are different because while they are problematic in folksonomies, they are more — much more — problematic in top-down classification systems. Folksonomies represent progress in those areas, in other words.

    You want cultural dependence? The Library of Congress, in its top level categories for geographic regions, lists “The Balkan Penninsula” as one main entity, and “Asia” as another. Contested terms? Try finding queer literature in any library classification scheme. And so on. Folksonomic tagging improves on this by exposing cultural dependence and contestedness, rather than denying its existence, or hiding it by fiat.

    (As an aside, the signal loss from the pressures brought to bear on official categorizations is a common theme in classification generally. The entire alt. hierarchy in usenet came into being because there was a proposal to create rec.drugs, and there was concern that usenet, running in part over an NSF-funded network, would be shut down. The alt.* hierarchy was a compromise, to allow some face saving in suggesting that the *.drugs group was not ‘official’. And of course, alt. (an early folksonomy, albeit highly compromised by usenet’s hierarchical design) ballooned to many times the size of the ‘official’ usenet.)

    The aggregate good of tags is not that they create consensus or accuracy; they observably don’t, and this is very observability is much of their value. Pick any popular link, click on the “and X other people” link under the URL, and you’ll see how that page is tagged by dozens or hundreds of people. There is both broad alignment around a few terms, but there is also a long tail of other views, which you don’t get in formal systems.

    To take but one example, of the 114 people who tagged the Buffyology database, 66 tagged it buffy, 58 tagged it tv, and only 12 tagged it database, the third most popular tag. But 4 people tagged it sf or scifi, 2 tagged it fantasy, and 2 tagged it vampire. So the ambiguity between the literature of fantasy and of science fiction is exposed in the tagging, and the possibility of viewing Buffy as a thing related not mainly to TV but to vampires is also preserved.

    So this is what I don’t get: I can’t imagine that anyone concerned about hegemony and marginalization would prefer professionally structured categories over folksonomy. If you care about contested terms and the risks of marginalization,, Flickr, et al do more to improve our access to, and understanding of, marginalization and contestation than any current alternative.

    Comments (4) + TrackBacks (0) | Category: social software

    January 28, 2005

    Ross and danah in an article on Friendster in the NYTEmail This EntryPrint This Article

    Posted by Clay Shirky

    You can tell how behind I am on my reading — just got to the Monday NY Times article on Friendster, to find our own Ross and danah quoted:

    “Social networking is at this very interesting point,” said Ross Mayfield, a pioneer in the social networking field and the chief executive of Socialtext, which sells software for collaborative writing and editing via the Internet. “These companies are at the stage where they need to demonstrate real results in terms of revenues and their business model. That voyeuristic fascination of seeing who has the most friends has worn off for a lot of people.”

    Comments (0) + TrackBacks (0) | Category: social software

    January 26, 2005

    Britannica not so great on the fact checking department after allEmail This EntryPrint This Article

    Posted by Clay Shirky

    It’s so good, I don’t even want to comment:

    A SCHOOLBOY with a fascination for Poland and wildlife has uncovered several significant errors in the latest — the fifteenth — edition of the Encyclopaedia Britannica. Lucian George, 12, a pupil at Highgate Junior School in North London, was delving into the volumes on Poland and wildlife in Central Europe when he noted the mistakes.

    More, much more, here.

    Comments (4) + TrackBacks (0) | Category: social software

    When usenet was the world...Email This EntryPrint This Article

    Posted by Clay Shirky

    AOL is no longer offering usenet access.

    This feels to me like they’re tearing down an old diner in a neighborhood I used to live in. I never go there anymore, but I spent 5 years of my life on usenet, and 2 of those years in a fever I can’t characterise as anything other than addiction. I learned to write there, and it’s one of only two places where I had people I’d call real friends who I never met IRL. (The other was Old Man Murray, RIP.)

    The moment AOL started offering newsgroup access, it created the ‘long September’, where the flood of clueless newbies became more or less permanent, in contrast with previous years where every September saw an influx of freshman at colleges with usenet access, who were then flamed to high heaven acculturated, and the whole thing settled back down by October.

    It was also the first sign that the logic of value through interconnection was higher than value through exclusivity. (AOL’s genius, unrivalled, I believe, in the modern era, was to say one thing to investors and another to users, telling Wall Street it was a media company, but selling communications tools and the attendant access to community to its users, functions that always dwarfed use of its media properties.)

    There’s not much to eulogize here — the era when you had to explain to the press that the internet was more than just usenet are long over, and AOL’s vital place in the ecosystem wanes each day with the fortunes of dial-up access generally. Over at MeFi, they’re trying to revel in the idea that the long September is ending, but that’s not whats happening here — AOL’s decision to disconnect its NNTP servers cedes usenet to spam and usenet access to Google, which things were each a done deal some time ago.

    Still, it feels kind of funny. Usenet was such a spectacular experiment in the annals of human communication, the idea that it’s value isn’t worth the cost of keeping the servers running comes as a marker of things I already knew, but which still feel different when they become facts in the world.

    Comments (2) + TrackBacks (0) | Category:

    January 25, 2005

    Ontology as a term of artEmail This EntryPrint This Article

    Posted by Clay Shirky

    After my little tirade yesterday, my friend Kio pointed out that ontology is a term of art for almost every group that uses it, and that it has very different meanings in those various groups.

    For the metaphysicians, ontology is inquiry into the nature and relations of being, with a particular focus on fundamental categories. That’s not what I mean.

    The definition of ontology I’m referring to is derived more from AI than philosophy: a formal, explicit specification of a shared conceptualization. (Other glosses of this AI-flavored view can be found using Google for define:ontology.) It is that view that I am objecting to.

    Comments (3) + TrackBacks (0) | Category: social software

    January 24, 2005

    Good post on folksonomy; another on taggingEmail This EntryPrint This Article

    Posted by Clay Shirky

    Great post on folksonomy from
    Folksonomy Notes: Considering the Downsides, Behavioral Trends, and Adaptation

    One thing that I mentioned in response to Liz’s post was that I feel we should keep in mind how adaptive we humans are. It is a fundamental talent we have. Too often, I think, we ignore this quality, pushing for consistency over everything else, when all we need is a little explanation of how things work. Once we know how they work, we’re fine. I’m not advocating a willy-nilly approach to designing architectures for humans: I’m advocating a willy-nilly approach to designing architectures by humans, who use a willy-nilly approach when reading and writing and speaking words.

    and another on tagging by Tim Bray: What Do Tags Mean?

    I think that it would be nice if a huge number of web pages converged on using a simple, flat, shared set of tags with entries like vancouver and mac os x and tsunami relief, which the current setup works well for.

    But I think it would also be nice if, once we have Atom, there are feeds about Petroleum Geology with their own tags, and feeds about Military Training too, and they each have their own drill tag. Which Atom would support nicely.

    Of course, the only people who would need to know about the Petroleum or Military tags would be people specifically looking for that kind of stuff; someone looking for a drill tag generically would probably get both and maybe that would be fine.

    Bottom line: I suspect Technorati, and anyone else who takes this up, should offer an (optional) “scheme” field in their tag search capability, which would be handy for those who care and invisible for those who don’t.

    Comments (2) + TrackBacks (0) | Category: social software

    Tags != folksonomies && Tags != Flat name spacesEmail This EntryPrint This Article

    Posted by Clay Shirky

    Grrrr - I hate not having the time to write the post I want to write on this, but here goes…

    Tags are labels attached to things. This procedure is absolutely orthogonal to whether professionals or amateurs are doing the tagging.

    Professionals often think tags are covalent with folksonomies because their minds have been poisoned by the false dream of ontology, but also because tagging looks too easy (in the same way the Web looked too easy to theoreticians of hypertext.) Not only are tags amenable to being used as controlled vocabularies, it’s happening today, where groups are agreeing about how to tag things so as to produce streams of e.g. business research.

    More importantly, tags are not the same as flat name spaces. The LiveJournal interests list, the first large-scale folksonomy I became aware of (though before the label existed) is flat. The interest list has one meaning: Person X has Interest Y, included as part of in List L. All L is attached to X, and all Y’s are equivalent in L.

    Tags don’t work that way at all. Tags are multi-dimensional, and only look flat, in the way Venn diagrams look flat. When I tag something ‘socialsoftware drupal’, I enable searches of the form “socialsoftware & drupal”, “socialsoftware &! (and not) drupal”, “drupal &! socialsoftware”, and so on.

    Hierarchy is a degenerate case of tags. If hierarchy floats your boat, by all means tag hierarchically. If I tag so that A &! B returns no results, and a search on A alone returns the same items as A & B, then A is a subset of B at the moment.

    This last point is key — the number one fucked up thing about ontology (in its AI-flavored form - don’t get me started, the suckiness of ontology is going to be my ETech talk this year…), but, as I say, the number one thing, out of a rich list of such things, is the need to declare today what contains what as a prediction about the future. Let’s say I have a bunch of books on art and creativity, and no other books on creativity. Books about creativity are, for the moment, a subset of art books, which are a subset of all books.

    Then I get a book about creativity in engineering. Ruh roh. I either break my ontology, or I have to separate the books on creativity, because when I did the earlier nesting, I didn’t know there would be books on creativity in engineering. A system that requires you to predict the future up front is guaranteed to get worse over time.

    And the reason ontology has been even a moderately good idea for the last few hundred years is that the physical fact of books forces you to predict the future. You have to put a book somewhere when you get it, and as you get more books, you can neither reshelve constantly, nor buy enough copies of any given book to file it on all dimensions you might want to search for it on later.

    Ontology is a good way to organize objects, in other words, but it is a terrible way to organize ideas, and in the period between the invention of the printing press and the invention of the symlink, we were forced to optimize for the storage and retrieval of objects, not ideas. Now, though, we can scrap of the stupid hack of modeling our worldview on the dictates of shelf space. One day the concept of creativity can be a subset of a larger category, and the next day it can become a slice that cuts across several categories. In hierarchy land, this is a crisis; in tag land, it’s an operation so simple it hardly merits comment.

    The move here is from graph theory (arrange everything in a tree graph, so that graph traversal becomes the organizing principle) to set theory (sets have members, and the overlap or non-overlap of those memberships becomes the organizing principle.) This is analogous to the change in how we handle digital data. The file system started out as a tree graph. Then we added symlinks (aliases, shortcuts), which said “You can organize things differently than you store them, and you can provide more than one mode of access.”

    The URI goes all the way in that direction. The URI says “Not only does it not matter where something is stored, it doesn’t matter whether it’s stored. A URI that generates the results on the fly is as valid as one that points to a disk.” And once something is no longer dependent on tree graph traversals to find it, you can dispense with hierarchical assumptions about categorizing it too.

    Comments (13) + TrackBacks (1) | Category: social software

    January 23, 2005

    The Innovator's LemmaEmail This EntryPrint This Article

    Posted by Clay Shirky

    To respond to David’s question about folksonomies Aren’t we going to innovate our way out of this? My answer is yes, but only for small values of “out.” A big part of what’s coming is accepting and adapting to the mess, instead of exiting it.

    Seeing people defend professional classification as a viable option for large systems is giving me horrible flashbacks to the arguments in 1993 about why gopher was superior to the Web. Gopher was categorized by professionals, and it was hierarchical. Gotta love hierarchy for forcing organization. You can’t just stick things any old place — to be able to add something to a hierarchy, you have to say where it goes, and what it goes with, next to, under, and above. And if you do it right, you can even call it an ontology, which means you get to charge extra. (I loves me some ontologies.)

    The Web, meanwhile, was chaos. Chaos! You could link anything to anything else! Melvil Dewey would plotz if he saw such a tuml. How on earth could you organize the Web? The task is plainly impossible.

    And you know what? The gopher people were right. The Web is chaos, and instead of getting the well-groomed world of gopher, we’ve adapted to the Web by meeting it half way.

    ...continue reading.

    Comments (7) + TrackBacks (0) | Category: social software

    January 22, 2005

    Folksonomies are a forced move: A response to LizEmail This EntryPrint This Article

    Posted by Clay Shirky

    Liz’s fantastic posts on folksonomy (one, two) detail the new issues we’re facing or will face around folksonomic organization. In the first post, though, she takes on my earlier argument about the economic value of folksonomy, saying

    Clay argues that detractors from wikipedia and folksonomy are ignoring the compelling economic argument in favor of their widespread use and adoption. Perhaps. But I’m arguing that it’s just as problematic to ignore the compelling social, cultural, and academic arguments against lowest-common-denominator classification. I don’t want to toss out folksonomies. But I also don’t want to toss out controlled vocabularies, or expert assignment of categories. I just don’t believe that all expertise can be replicated through repeated and amplified non-expert input.

    I don’t believe that either, so I want to re-state my views on the subject.

    I believe that folksonomies will largely displace professionally produced meta-data, and that this will not take very long to happen. However, I do not think that folksonomy is better than controlled vocabularies or expert judgment, except for completely tautological definitions of ‘better’, where the rise of folksonomy is viewed as prima facie evidence of superiority. This is not the position I take.

    If I had to craft a statement I thought both Liz and I could agree with, it would be that technology always involves tradeoffs among various characteristics in a particular environment. She goes on to list some of those characteristics, including especially the risks from lowest-common-denominator classifications. So far, so sympatico.

    Here’s where I think we disagree. She thinks economic value is another of the characteristics to be traded off. I think economic value is the environment.

    Put another way, I don’t think it matters what is lost by not having professionally produced metadata in any environment where that is not an option anyway, by virtue of being priced out of the realm of possibility.

    So when she says I am urging an uncritical acceptance of folksonomies, she is half right. I am not in favor of uncriticality; indeed, in the post she references, I note that well-designed metadata is better than folksonomies on traditional axes of comparison.

    But she’s right about the ‘acceptance’ half. It doesn’t matter whether we “accept” folksonomies, because we’re not going to be given that choice. The mass amateurization of publishing means the mass amateurization of cataloging is a forced move. I think Liz’s examination of the ways that folksonomies are inferior to other cataloging methods is vital, not because we’ll get to choose whether folksonomies spread, but because we might be able to affect how they spread, by identifying ways of improving them as we go.

    To put this metaphorically, we are not driving a car, with gas, brakes, reverse and a lot of choice as to route. We are steering a kayak, pushed rapidily and monotonically down a route determined by the enviroment. We have a (very small) degree of control over our course in this particular stretch of river, and that control does not extend to being able to reverse, stop, or even significantly alter the direction we’re moving in.

    Comments (4) + TrackBacks (1) | Category: social software

    More on Social Software as a termEmail This EntryPrint This Article

    Posted by Clay Shirky

    I got an email from Alex Pang at the Institute for the Future, asking about the future of social software, which response I started by writing about the value of the term in the recent past. (I’ll post my predictions for the future under separate cover.) This is in a way a continuation of the conversation started about social software as a term, kicked off last fall by Chrisopher Allen’s history and definition of the word.

    Alex’s questions were Where do you think social software will be in ten years? Will it be the foundation of a discrete category of applications or services? Will social software-like capabilities be built into other software? Will the whole concept be as outdated as a KC and the Sunshine Band album?

    Yes, in 10 years, the phrase will be outdated. We won’t need it anymore because the value of social interaction will be folded into a large number of applications, sometimes as built-in features, sometimes as external services that get integrated in the manner of web services.

    Looking back, the phrase ‘social software’ has served three functions. First, it called attention to an explosion of new work that was otherwise seemingly unrelated: at first glance, isn’t like Meetup isn’t like Socialtext. The label made it both possible and fruitful to examine those similarities, and to imagine how applications like those might be combined or extended.

    ...continue reading.

    Comments (3) + TrackBacks (0) | Category: social software

    January 7, 2005

    folksonomies + controlled vocabulariesEmail This EntryPrint This Article

    Posted by Clay Shirky

    There’s a post by Louis Rosenfeld on the downsides of folksonomies, and speculation about what might happen if they are paired with controlled vocabularies.

    …it’s easy to say that the social networkers have figured out what the librarians haven’t: a way to make metadata work in widely distributed and heretofore disconnected content collections.

    Easy, but wrong: folksonomies are clearly compelling, supporting a serendipitous form of browsing that can be quite useful. But they don’t support searching and other types of browsing nearly as well as tags from controlled vocabularies applied by professionals. Folksonomies aren’t likely to organically arrive at preferred terms for concepts, or even evolve synonymous clusters. They’re highly unlikely to develop beyond flat lists and accrue the broader and narrower term relationships that we see in thesauri.

    I also wonder how well Flickr,, and other folksonomy-dependent sites will scale as content volume gets out of hand.

    This is another one of those Wikipedia cases — the only thing Rosenfeld is saying that’s actually wrong is that ‘lack of development’ bit — is less than a year old and spawning novel work like crazy, so predicting that the thing has run out of steam when people are still freaking out about Flickr seems like a fatally premature prediction.

    The bigger problem with Rosenfeld’s analysis is its TOTAL LACK OF ECONOMIC SENSE. We need a word for the class of comparisons that assumes that the status quo is cost-free, so that all new work, when it can be shown to have disadvantages to the status quo, is also assumed to be inferior to the status quo.

    The advantage of folksonomies isn’t that they’re better than controlled vocabularies, it’s that they’re better than nothing, because controlled vocabularies are not extensible to the majority of cases where tagging is needed. Building, maintaining, and enforcing a controlled vocabulary is, relative to folksonomies, enormously expensive, both in the development time, and in the cost to the user, especailly the amateur user, in using the system.

    Furthermore, users pollute controlled vocabularies, either because they misapply the words, or stretch them to uses the designers never imagined, or because the designers say “Oh, let’s throw in an ‘Other’ category, as a fail-safe” which then balloons so far out of control that most of what gets filed gets filed in the junk drawer. Usenet blew up in exactly this fashion, where the 7 top-level controlled categories were extended to include an 8th, the ‘alt.’ hierarchy, which exploded and came to dwarf the entire, sanctioned corpus of groups.

    The cost of finding your way through 60K photos tagged ‘summer’, when you can use other latent characteristics like ‘who posted it?’ and ‘when did they post it?’, is nothing compared to the cost of trying to design a controlled vocabulary and then force users to apply it evenly and universally.

    This is something the ‘well-designed metadata’ crowd has never understood — just because it’s better to have well-designed metadata along one axis does not mean that it is better along all axes, and the axis of cost, in particular, will trump any other advantage as it grows larger. And the cost of tagging large systems rigorously is crippling, so fantasies of using controlled metadata in environments like Flickr are really fantasies of users suddenly deciding to become disciples of information architecture.

    This is exactly, eerily, as stupid as graphic designers thinking in the late 90s that all users would want professional but personalized designs for their websites, a fallacy I was calling “Self-actualization by font.” Then the weblog came along and showed us that most design questions agonized over by the pros are moot for most users.

    Any comparison of the advantages of folksonomies vs. other, more rigorous forms of categorization that doesn’t consider the cost to create, maintain, use and enforce the added rigor will miss the actual factors affecting the spread of folksonomies. Where the internet is concerned, betting against ease of use, conceptual simplicity, and maximal user participation, has always been a bad idea.

    Comments (15) + TrackBacks (0) | Category: social software

    Jake responds to my post on Wikipedia and authorityEmail This EntryPrint This Article

    Posted by Clay Shirky

    [Ed. Note: Jake wrote such a long and thoughtful comment responding to my post of yesterday on the Wikipedia and authority that I wanted to re-post it as an entry, with comments and trackbacks of its own.

    I have re-formatted it to use box-style quotes and expanded some acronynms, but changed none of the substance. -clay ]

    Clay writes:

    Picking up on yesterday’s theme of authority, the authority of, say, Coleridge’s encyclopedia was the original one: authority derived from the identity of the author. This is like trusting Mom’s Diner, or the neighborhood tailor — personal reputation is worth preserving, and helps assure quality. The authority of Britannica, by contrast, is the authority of a commercial brand. Their sales are intimately tied into their reputation for quality, so we trust them to maintain those standards, in order to preserve an income stream. This is like trusting Levis or McDonald’s — you don’t know the individuals who made your jeans or your french fries, but the commercial incentive the company has in preserving its brand makes the level of quality predictable and stable.

    Jake comments:

    Yes, but a brand is some sort of an ethereal thing. [I think a ‘not’ was dropped, as in “… not some sort of ethereal thing.” — ed] It is a symbolic representation of a product or the underlying institution that created it. Trademark rights are common law in nature and they attach through use.

    So while it is true that at some level this sort of structure is a reification, in practical terms its role in building trust must be considered. To me this is really the core of the concerns about Wikipedia. Is the structure adequate to the task of establishing something authoritative enough to be useful.

    ...continue reading.

    Comments (3) + TrackBacks (0) | Category: social software

    January 6, 2005

    Coates' new shorthand definition of social softwareEmail This EntryPrint This Article

    Posted by Clay Shirky

    Tom glosses himself, coming up with a pithier and more example-driven definition of social software:

    Social Software can be loosely defined as software which supports, extends, or derives added value from, human social behaviour - message-boards, musical taste-sharing, photo-sharing, instant messaging, mailing lists, social networking.

    Comments (6) + TrackBacks (0) | Category: social software

    Wikipedia: The nature of authority, and a LazyWeb request...Email This EntryPrint This Article

    Posted by Clay Shirky

    There was another point of danah’s I wanted to respond to, but yesterday’s post had gotten quite long enough, and in any case, I had a slightly different take in mind for this, including a LazyWeb request at the end, relating to this image:

    danah says “Wikipedia appears to be a legitimate authority on a vast array of topics for which only one individual has contributed material. This is not the utopian collection of mass intelligence that Clay values.” This misconstrues a dynamic system as a static one. The appropriate phrase is “…for which only one individual has contributed material so far.”

    Wikipedia is not a product, it is a system. The collection of mass intelligence that I value unfolds over time, necessarily. Like democracy, it is messier than planned systems at any given point in time, but it is not just self-healing, it is self-improving. Any given version of Britannica gets worse over time, as it gets stale. The Wikipedia, by contrast, whose version is always the Wiki Now, gets better over time as it gets refreshed. This improvement is not monotonic, but it is steady.

    ...continue reading.

    Comments (8) + TrackBacks (0) | Category: social software

    January 5, 2005

    Wikipedia: Me on boyd on Sanger on WalesEmail This EntryPrint This Article

    Posted by Clay Shirky

    My response to danah’s response about the Wikipedia/anti-elitism debate:

    First, some background. I have the same “yes, but” reaction over and over to Wikipedia detractors — much of what both Sanger and boyd say is wrong with the Wikipedia is wrong with it, but then there’s this incredible leap from “The site as it stands has faults” to “…and so it must be ignored or radically altered.”

    Reading pieces like Sanger’s, I feel like I’m being told that bi-planes fly better than F-16s because F-16’s are so heavy. You cannot understand how well things fly without understanding both weight and thrust.

    It’s a similar leap to assume that, since the Wikipedia has disadvantages relative to the Encyclopedia Britannica, Britannica must therefore be better. The real question is “Weighing the advantages and disadvantages of the Wikipedia against the advantages and disadvantages of Britannica, under what conditions is Britannica better, and under what conditions is Wikipedia better?”

    ...continue reading.

    Comments (11) + TrackBacks (0) | Category: social software

    January 4, 2005

    Taggle: A proposed Google for FolksonomiesEmail This EntryPrint This Article

    Posted by Clay Shirky

    Post at brianstorms about federated folksonomies a la and flickr, with an image-as-thought-experiment that sums up the idea neatly:

    Comments (2) + TrackBacks (0) | Category:

    Reagle on the WikipediaEmail This EntryPrint This Article

    Posted by Clay Shirky

    As if on cue, I got a pointer from Joseph Reagle to his recent paper on the Wikipedia. It’s a fascinating piece, largely concerned with disagreement and dispute resolution among participants, but relative to the current debate about the Sanger piece, this bit of history jumped out at me:

    Wikipedia is the populist offshoot of the Nupedia project started in March of 2000 by Jimbo Wales and Larry Sanger. Nupedia’s mission was to create a free encyclopedia via rigorous expert review under a free documentation license. Unfortunately, this process moved rather slowly and having recently been introduced to Wiki, Sanger persuaded Wales to set up a scratch-pad for potential Nupedia content where anyone could contribute. However, “There was considerable resistance on the part of Nupedia’s editors and reviewers, however, to making Nupedia closely associated with a website in the wiki format. Therefore, the new project was given the name ‘Wikipedia’ and launched on its own address,, on January 15 [2001]”

    Wikipedia proved to be so successful that when the server hosting Nupedia crashed in September of 2003 (with little more than 23 “complete” articles and 68 more in progress) it was never restored.

    The idea of a Wikipedia, but vetted, runs aground on the simple math of relative growth. The ‘filter, then publish model’ of Nupedia, for all its considerable virtues, is simply inadequate to deal with rapid growth (only 23 articles completed!). The characteristics of the Wikipedia’s success are in its ability to grow with minimal constraints, even when that means that the whole is a work in progress.

    Comments (0) + TrackBacks (0) | Category: social software

    January 3, 2005

    K5 Article on Wikipedia Anti-elitismEmail This EntryPrint This Article

    Posted by Clay Shirky

    Slashdot has a roundup of criticism of the Wikipedia, including a pointer to a Kuro5hin article by Larry Sanger, a co-founder of the Wikipedia, making three strong criticisms of the Wikpedia as it stands.

    The first criticism is that the Wikpedia lacks the perception of acccuracy:

    My point is that, regardless of whether Wikipedia actually is more or less reliable than the average encyclopedia, it is not perceived as adequately reliable by many librarians, teachers, and academics. The reason for this is not far to seek: those librarians etc. note that anybody can contribute and that there are no traditional review processes. You might hasten to reply that it does work nonetheless, and I would agree with you to a large extent, but your assurances will not put this concern to rest.

    This analysis seems to be correct on the surface, and at the same time deeply deeply wrong. Of course librarians, teachers, and academics don’t like the Wikipedia. It works without privilege, which is inimical to the way those professions operate.

    This is not some easily fixed cosmetic flaw, it is the Wikipedia’s driving force. You can see the reactionary core of the academy playing out in the horror around Google digitizing books held at Harvard and the Library of Congress — the NY Times published a number of letters by people insisting that real scholarship would still only be possible when done in real libraries. The physical book, the hushed tones, the monastic dedication, and (unspoken) the barriers to use, these are all essential characteristics of the academy today.

    It’s not that it doesn’t matter what academics think of the Wikipedia — it would obviously be better to have as many smart people using it as possible. The problem is that the only thing that would make the academics happy would be to shoehorn it into the kind of filter, then publish model that is broken, and would make the Wikipedia broken as well.

    Sanger’s second complaint is about governance:

    Far too much credence and respect accorded to people who in other Internet contexts would be labelled “trolls.” There is a certain mindset associated with unmoderated Usenet groups and mailing lists that infects the collectively-managed Wikipedia project: if you react strongly to trolling, that reflects poorly on you, not (necessarily) on the troll. If you attempt to take trolls to task or demand that something be done about constant disruption by trollish behavior, the other listmembers will cry “censorship,” attack you, and even come to the defense of the troll.

    This complaint is right, I think, inasmuch as it hits the core problem of Wikipedia (and of social software generally), namely governance. How do you take a group of individuals who disagree and get them to co-create, and to agree to be bound by a decision-making process that will assure that no one gets everything they want? And how do you also make that system open?

    However, Sanger gives Wales and the Wikipedia contributors too little credit here, I think. Governance is a certified Hard ProblemTM, and at the extremes, co-creation, openness, and scale are incompatible. The Wikipedia’s principle advantage over other methods of putting together a body of knowledge is openness, and from the outside, it looks like the Wikipedia’s guiding principle is “Be as open as you can be; close down only where there is evidence that openness causes more harm than good; when this happens, reduce openness in the smallest increment possible, and see if that fixes the problem.” Lather, rinse, repeat.

    You can see this incrementalism in the Wikipedia crew’s creeping approach to limiting edits — not allowing edits on the home page, paragraph level edits on long articles, etc. These kinds of solutions were deployed only in response to particular problems, and only after those problems were obviously too severe to be dealt with in any other way.

    This pattern means that there will always be problems with governance on the Wikipedia, by definition. If you don’t lock down, you will always get the problems associated with not locking down. However, to take the path Sanger seems to be advocating — lock down more, faster — risks giving up the Wikipedia’s core virtue. The project may yet fail because there is no sweet spot between openess and co-creation at Wikipedia scale. But to lock down pre-emptively won’t be avoiding that failure but accelerating it.

    Sanger’s final point, that the Wikipedia is anti-elitist, is quite similar to his first complaint. Yes, it is impossible for experts on a subject to post their views without molestation but that’s how wikis work. It’s certainly easy to imagine systems where experts are deferred to mechanically. Much of the world, including, significantly, the academy, works that way. But if you want a system that works that way, you don’t want a wiki, and if you want a wiki, you won’t get a system that works that way.

    In place of ordained expertise, my guess is that the Wikipedia will move further towards a ‘core group’ strategy, where there will be increasing separation of powers between committed and casual users, and the system will gain a kind of deference, not for expertise (a fairly elusive quality that Sanger invokes but never defines)

    It’s been fascinating to watch the Kubler-Ross stages of people committed to Wikipedia’s failure: denial, anger, bargaining, depression, acceptance. Denial was simple; people who didn’t think it was possible simply dis-believed. But the numbers kept going up. Then they got angry, perhaps most famously in the likening of the Wikipedia to a public toilet by a former editor for Encyclopedia Brittanica. Sanger’s post marks the bargaining phase; “OK, fine, the Wikipedia is interesting, but whatever we do, lets definitely make sure that we change it into something else rather than letting the current experiment run unchecked.”

    Next up will be a glum realization that there is nothing that can stop people from contributing to the Wikipedia if they want to, or to stop people from using it if they think it’s useful. Freedom’s funny like that.

    Finally, acceptance will come about when people realize that head-to-head comparisons with things like Britannica are as stupid as comparing horseful and horseless carriages — the automobile was a different kind of thing than a surrey. Likewise, though the Wikipedia took the -pedia suffix to make the project comprehensible, it is valuable as a site of argumentation and as a near-real-time reference, functions a traditional encyclopedia isn’t even capable of. (Where, for example, is Britannica’s reference to the Indian Ocean tsunami?)

    The Wikipedia is an experiment in social openness, and it will stand or fall with the ability to manage that experiment. Whining like Sanger’s really only merits one answer: the Wikipedia makes no claim to expertise or authority other than use-value, and if you want to vote against it, don’t use it. Everyone else will make the same choice for themselves, and the aggregate decisions of the population will determine the outcome of the project.

    And 5 years from now, when the Wikipedia is essential infrastructure, we’ll hardly remember what the fuss was about.

    Comments (36) + TrackBacks (0) | Category: social software

    A Really Simple Chat, v3.0b: Back Channels 'R' ItEmail This EntryPrint This Article

    Posted by Clay Shirky

    I’ve admired Manuel Kiessling’s marvelous A Really Simple Chat (ARSC) program since I saw Greg Elin using it as conference support back in 2002. Greg and I played around with it some more that fall for a small social software conference, adding some features specific to backchannel support (detailed in In-Room Chat as a Social Tool.)

    Now Manuel is beta-testing 3.0 of ARSC, and it’s got a lot of native support for backchannel features — there’s a specific ‘in room chat’ room, with Jerry Michalski’s ‘red card/green card’ system built in. There is also a defult ‘Display’ user (beta login ‘Display’, passwd ‘arsc’) whose view of the chat is optimized for projection or plasma screen by boosting the font size and dropping the input features.

    He’s also added user levels and two-level moderation, along with a number of other new features.

    The goal, says Manuel, is to “…make it possible to set up a digital backchannel in under 2 minutes, without any dirty hacks.” You can play with the beta version on his site, or install your own.

    As always, the advantages of ARSC are its ability to circulate access to a chat via URL, which is often simpler than getting people to download special software for irc., and allows a variety of strategies for inclusivity and exclusivity.

    I still have some LazyWeb requests for ARSC. The main one is to be able to turn the current message post-processing tool into a full-fledged Atom feed, to make it easier to archive, monitor, and even bridge between ARSC and irc.

    I’d also like to see some sort of ‘scroll-speed’ setting, where, when there are more than a dozen or so interjections in a minute, the later comments are pre-cached and roll out on the screen at some specified maximum pace, rather than the earlier comments flying off the screen. When things get crazy, make the machines do the work, not the people…

    And Manuel has registered as well — nothing there yet, but he says he’s going to “…set up a wiki there for all things social software/in-room chat/digital backchannel.”

    Comments (2) + TrackBacks (0) | Category: social software

    January 1, 2005

    Good Piece on FolksonomiesEmail This EntryPrint This Article

    Posted by Clay Shirky

    Good piece by Adam Mathes, Folksonomies - Cooperative Classification and Communication Through Shared Metadata:

    Perhaps the most important strength of a folksonomy is that it directly reflects the vocabulary of users. In an information retrieval system, there are at least two, and possibly many more vocabularies present (Buckland, 1999). These could include that of the user of the system, the designer of the system, the author of the material, the creators of the classification scheme; translating between these vocabularies is often a difficult and defining issue in information systems. As discussed earlier, a folksonomy represents a fundamental shift in that it is derived not from professionals or content creators, but from the users of information and documents. In this way, it directly reflects their choices in diction, terminology, and precision.

    Comments (0) + TrackBacks (0) | Category: social software

    December 22, 2004

    Notes from ITP: Flickr-as-web-services editionEmail This EntryPrint This Article

    Posted by Clay Shirky

    Been away, working on a bunch of things including, most speculatively, a proposal for a book with the working title Organization in the Age of Social Devices, where devices refers both to our tools and to the things people do with those tools when left to their own devices. The collected themes of the book will be no surprise to readers here.

    All that is so 2006, however, and this is still 2004, so I want to try to capture some of what I’ve been seeing this semester at ITP. Unlike last year, where the fall semester largely resolved itself for me into a single big surprise (the pattern I’m calling Situated Software,) this year I’m seeing lots of distributed effects, with no one common thread, so I’m going to do a series of posts of things I’ve seen.

    So, first of all, ITP is Flickr-obsessed. The community is either in the grip of a fast-moving addiction, or we’re an epicenter of a pandemic; time will tell.

    I’ll start with two quick Flickr stories…

    ...continue reading.

    Comments (1) + TrackBacks (0) | Category: social software

    November 17, 2004

    Monitor110: Collective wisdom for investorsEmail This EntryPrint This Article

    Posted by Clay Shirky

    Monitor110 is taking the Technorati pattern and customizing it for investors. It’s for-fee and alpha, so its not easy to test-drive, but they claim near-real-time monitoring of 6M+ sources, which is half again as large as the Technorati universe; given their subscribing/scraping pattern, this means that the RSS universe has grown considerably larger than the weblog universe alone.

    This falls in the latent social software category, but it’s interesting to see the ‘collective wisdom of the weblog world’ pattern of blogdex et al becoming moving from the general to the specific.

    Comments (2) + TrackBacks (0) | Category: social software

    November 12, 2004

    Blogging as activity, blogging as identityEmail This EntryPrint This Article

    Posted by Clay Shirky

    I was mis-somethinged in the paper today — not misquoted or misattributed, exactly, but something like misconstrued, which makes me think both of danah’s post about blogging metaphors and Liz’s post about presenting blogs as diaries vs. blogs as outlets for research.

    In an article on weblogging and politics in today’s NY Times, Tom Zeller, the reporter who wrote the story, has me saying this:
    [Shirky] suggests that the online fact-finding machine has come unmoored, and that some bloggers simply “can’t imagine any universe in which a fair count of the votes would result in George Bush being re-elected president.”

    I said the part inside the quotes, and it is, in a narrow way, accurate — I have seen a remarkable profusion of posts about the election that assumes that evidence that e-voting errors are part of a theft of the election on a grand scale.*

    But the sentiment overall is not something I believe — I am in fact on the record over at as saying that the unmasking of the National Guard memos was the most critical event in the use of internet technology in this campaign.

    What rubs me wrong is that the quote is framed in a way that makes it about identity, not activity. One way to present this would have been to define an axis of interest: ‘some Democrats “can’t imagine any universe in which a fair count of the votes would result in George Bush being re-elected president.”’ Another would have been to define a relatively neutral category: ‘some writers “can’t imagine any universe in which a fair count of the votes would result in George Bush being re-elected president.”’

    Neither of those seems wrong, but the way it’s phrased, I seem to be suggesting that there are bloggers unmoored from the fact-checking pattern because they are bloggers, rather than because they are Democratic partisans who publish their thoughts using weblog tools. And that’s where it goes wrong.

    I have long been of the opinion that the word weblog has no crisp meaning anymore, and is going to fade as a defining term for the same reason ‘portal’ did — there are too many patterns to be conveniently contained by one word. But here the nature of weblogging and webloggers is defined, from outside, as not just a category, but an identity.

    And that I think, is not just wrong but unfair. So many people have weblogs now that anyone wanting to say anything sweeping and negative about the weblog world — they’re all bored teenagers, they rant instead of writing, they are conspiracy theorist, whatever — can find, in 10 minutes on technorati, a hundred weblogs that support their point of view.

    If I had the interview to do over again, I’d say that many of the people who “can’t imagine any universe in which a fair count of the votes would result in George Bush being re-elected president” are blogging about it. Blogging is increasingly an activity rather than an identity, too widespread and various to be pigeonholed, and should be treated as such when peopel are writing about it.


    • As for my own views, I voted for Kerry and I’m sorry Bush won. I also think e-voting is a disaster. But I don’t think the two are linked — Bush’s majority was too large, and even if Kerry could win the electoral college with a reversal in Ohio or Florida, he still lost the popular vote. I thought it was rotten to have a popular win and an electoral loss in 2000, and I still hold that principle.

    Comments (5) + TrackBacks (0) | Category: social software

    November 8, 2004

    IMsmarterEmail This EntryPrint This Article

    Posted by Clay Shirky is a service designed to add GMail-style functions to your IM conversations, by setting up a proxy service that archives all your IM conversations, making them persistent, accessible from multiple clients, searchable, combinable, etc. (They also have a blog feature, where you can set up a blog through the service, and sub to blogs created by other IMsmarter users, but that feels added-on and irrelevant compared to the main offering.)

    I’ve only been playing with it a while, and it’s still clunky in parts (search, for example, only searches the text of a conversation, but not the username, so when I say “Hmm, I was talking to Alex, and he said something about…” I can’t (or can’t find a way to) search on his name directly.

    Architecturally, though, it’s another turn of the centralization/decentralization screw, where adding a centralized server upstream of a P2Pish app like IM creates novel value. And by using a proxy, which is a pretty low-level tool, they get to be platform and client-agnostic, since most clients have to support proxying to deal with firewalls.

    To be determined: whether AOL tries to kill it. That, more than any subsequent features, will determine its success or failure.

    Comments (8) + TrackBacks (0) | Category: social software

    November 5, 2004

    Group as UserEmail This EntryPrint This Article

    Posted by Clay Shirky

    A year and a half ago I suggested, in a speech at the Etech conference called A Group Is Its Own Worst Enemy, that…

    It has to be hard to do at least some things on the system for some users, or the core group will not have the tools that they need to defend themselves [against rogue users].

    Now, this pulls against the cardinal virtue of ease of use. But ease of use is wrong. Ease of use is the wrong way to look at the situation, because you’ve got the Necker cube [of individual and group effects] flipped in the wrong direction. The user of social software is the group, not the individual.

    I think we’ve all been to meetings where everyone had a really good time, we’re all talking to one another and telling jokes and laughing, and it was a great meeting, except we got nothing done. Everyone was amusing themselves so much that the group’s goal was defeated by the individual interventions.

    The user of social software is the group, and ease of use should be for the group. If the ease of use is only calculated from the user’s point of view, it will be difficult to defend the group from the “group is its own worst enemy” style attacks from within.

    I’ve just put up another piece on this subject — Group As User: Flaming and the Design of Social Software — about what we can learn from the history of flaming in mailing lists and other conversational spaces:

    Flaming is one of a class of economic problems known as The Tragedy of the Commons. Briefly stated, the tragedy of the commons occurs when a group holds a resource, but each of the individual members has an incentive to overuse it. (The original essay used the illustration of shepherds with common pasture. The group as a whole has an incentive to maintain the long-term viability of the commons, but with each individual having an incentive to overgraze, to maximize the value they can extract from the communal resource.)

    In the case of mailing lists (and, again, other shared conversational spaces), the commonly held resource is communal attention. The group as a whole has an incentive to keep the signal-to-noise ratio low and the conversation informative, even when contentious. Individual users, though, have an incentive to maximize expression of their point of view, as well as maximizing the amount of communal attention they receive. It is a deep curiosity of the human condition that people often find negative attention more satisfying than inattention, and the larger the group, the likelier someone is to act out to get that sort of attention.

    However, proposed responses to flaming have consistently steered away from group-oriented solutions and towards personal ones. The logic of collective action, alluded to above, rendered these personal solutions largely ineffective. Meanwhile attempts at encoding social bargains weren’t attempted because of the twin forces of door culture (a resistance to regarding social features as first-order effects) and a horror of censorship (maximizing individual freedom, even when it conflicts with group goals.)

    The piece goes on to contrast wiki and weblogish ways of avoiding flaming, and uses that as input for suggesting a number of possible experiments with social form.

    Comments (5) + TrackBacks (0) | Category: social software

    October 21, 2004

    Merholz on Metadata for the MassesEmail This EntryPrint This Article

    Posted by Clay Shirky

    Great Peter Merholz piece, Metadata for the Masses, continuing the folksonomy/ethnoclassification thread

    We’re beginning to see ethnoclassification in action on the social bookmarks site, and the photo sharing site Flickr. […] The primary benefit of free tagging is that we know the classification makes sense to users. It can also reveal terms that “experts” might have overlooked. “Cameraphone” and “moblog” are newborn words that are already among Flickr’s most popular; such adoption speed is unheard of in typical classifications. For a content creator who is uploading information into such a system, being able to freely list subjects, instead of choosing from a pre-approved “pick list,” makes tagging content much easier. This, in turn, makes it more likely that users will take time to classify their contributions.

    Comments (1) + TrackBacks (0) | Category:

    October 14, 2004

    Social software as a termEmail This EntryPrint This Article

    Posted by Clay Shirky

    danah is right, Allen has done us all a great favor by posting his work on the term social software. I want to address her despisity (despision? despisement?) of the term, though, especially as my shayna punim graces the 2000+ section of Allen’s doc.

    I don’t think the term ‘social software’ is perfect, but I do think it’s optimal, as it’s obviously in use where other terms aren’t. So I think it’s a local but not global maxima.

    And I think it’s a local maxima because software is where the action is now, in this kind of social experimentation. In fact, I think that danah’s complaint, “I feel as though the term allows us to emphasize the technology instead of the behavior that it supports” is one of the things the phrase ‘social software’ has going for it.

    Technologists have been looking at social interaction for decades, usually through one of two lenses — either ignoring the users as hopelessly irrational (something with more than a grain of truth in it, as committed groups have emotional, not just intellectual, motivations), or by trying to shoehorn social applications into single-user paradigms, ignoring the fact that groups produce interactions that individuals and pairs don’t (e.g. flaming, trolling, etc.)

    So in this case, emphasizing to technologists the way software can both embody and alter social patterns is the right answer, since it creates a sense of possibility in bringing new techniques to old patterns to see what the interaction is like.

    And the misundertanding of the need for, and use of, that phrase by technologists seems to me to be at the heart of danah’s complaint. When she asks “Why are we acting like giddy children who just found a new toy?”, it’s because we’re giddy children who have just found a new toy.

    The fact that sociologists have been talking about these kind of things for decades hasn’t created a lot of value in the way social tools are designed, since there isn’t much of a habit among sociologists of talking in ways or venues that matter to developers (though there are notable exceptions, of course, including danah herself.)

    I spent the summer reading academic journals, principally Small Groups and Group Dynamics, and I can tell you that if you are minded to actually change the way groups interact, there’s more insight in the Flickr and interfaces than in combined publishing history of those journals.

    The difficulty of pattern fit is a fairer cop, since the domain of social software has fuzzy edges (of necessity, in my view.) For example, I say pairwise communications, such as SMS, falls out of the pattern, while MMOs are in, since the group effects, not the pairwise ones, are the hard ones to deal with. But this is a judgment call; others differ.

    And as for the focus on YASNSes, wikis, weblogs, etc, yes, the tech community suffers from neophilia, but then we would, wouldn’t we? If we thought old things were as good as new things, we wouldn’t invent new things. This creates some loss, but also considerable gain, and our tribe, almost by definition, is made up of people who think the gains from focusing on the new things outweigh the losses. (Not all of us are pure neophiles, though — my first post here was to a piece of writing done in 1970…)

    Complaining that we shouldn’t be delighted that new things are happening reminds me of Dave Farber’s comment that Napster was nothing new, because the original internet treated all machines as peers. Everything since — scale, the rise of the PC, the spread of audio tools, the horror of unstable IP addresses — seemed to him to be mere details. And yet Napster was a big deal — it, and not the original IP-bound tools, changed the world in the direction of both decentralization and what Tom Coates calls the New Musical Functionality.

    And social software feels like that to me now. We’ve had social network maps since the 1930s, and the 6 Degrees pattern since the 1960s, but we’ve only had networking services since 1996, and only had working ones since 2002. Bass-Station, Meetup, Flickr,, Fotowiki, LiveJournal, dodgeball, Audioscrobbler, these are new things, and they play well with others (unlike Lotus Notes et al, which wrecked the earlier term groupware), so we are not just getting new tools but are getting combinatorial complexity, as with the spread of the tagging pattern from feature to infrastructure.

    danah’s term, computer-mediated social interaction, is more descriptive, but probably less galvanizing for tool-builders, and places the edge problems in the technical domain — users don’t regard phones as computers, for example. Just plain “mediated group interaction” is probably even better (with my admitted bias towards triplets and away from pairs), but say that to someone who builds software and see if their eyes light up or glaze over.

    Allen probably credits me too much with popularizing the term (it was, ironically, David Winer and Shelley Powers who did most to spread it, by denouncing me and the horse I rode in on, back in 2002.) However, inasmuch as I have spread it, my principal goal hasn’t been accuracy, but inspiration. If you can get people schooled in single-user interaction to get excited about understanding and coding for the inherently social possibilities of software, you get cool new stuff to play with. And that beats uninspiring accuracy any day…

    Comments (8) + TrackBacks (0) | Category: social software

    October 6, 2004

    Blog Explosion and Insider's Club: Brothers in cluelessnessEmail This EntryPrint This Article

    Posted by Clay Shirky

    A couple years ago, when I was looking for some concise way to define Social Software, one of the definitions I used was “stuff worth spamming”, on the theory that any open group channel worth its salt would attract that form of system gaming.

    Behold Blog Explosion, the first blog spam co-operative. With Blog Explosion, you can now sign up to generate worthless traffic to other weblogs, in return for their generating worthless traffic to you:

    The concept is very simple. You read other blog sites and they in return visit your blog. Blogexplosion is the internet’s first blog exchange where thousands of bloggers visit each other’s blogs in order to receive tons of blog traffic. Imagine how many other people out there could be adding your blog to their blogroller and how many people would be reading your blog every day with this sort of attention. It’s free to use!

    And offers more proof, as if any were needed, that fantasizing about weblogs has become a broad cultural obsession, as the article Take the inside track to the insider’s club, demonstrates:

    Injecting yourself into the inside ranks of any subculture, from coin collecting to Java software programming, was once an arduous, seemingly impossible task, requiring years of experience, flights to far-off conventions, and lots of schmoozing with insiders. No more. Now anyone can assume the position of insider — one of those in-the-know types who is up on the latest news, is acquainted with all the major players, and is viewed as a personage of some esteem within a discreet arena. From e-mail to Weblogs, the online world opens up avenues to cozy up to experts, make a mark in your avocation or profession, and be viewed, in your own right, as someone who matters.

    It ends with the exhortation ” And the ultimate act of insiderdom? Create a Weblog. Do it, devote your life to it, and you will soon be a star.”

    I can’t tell whether to feel happy or sad that I’ve sat through this movie so many times that I can mouth the words, but seeing the idea of web rings and that old “Now you can have direct access to world leaders — through e-mail!” meme run through the “Now with new Blogs!” treatment does suggest we’ve entered the phase where first-mover advantages are being sold to Nth movers, where N is large. Next stop, exposes airing the disappointment of people who started a blog and worked on it all week and still didn’t become famous.

    Just like the web before it, the people selling ‘it’s the easy path to a big audience’ are not the inventors of the pattern but the people who understood how things worked when the crowd was small, and begin selling those virtues to the people whose very entrance into the system pushes it out of communal equilibrium.

    Comments (1) + TrackBacks (0) | Category: social software

    September 30, 2004

    The Seven Two Pieces Social Software Must HaveEmail This EntryPrint This Article

    Posted by Clay Shirky

    Last year Matt Webb at Interconnected posted On Social Software, which was then picked up by Matt Phillips at drupal, who posted Incentives for online software: the 7 pieces social software must have…. Both Webb and Phillips’s pieces were riffing on Stewart Butterfield’s earlier post on the same subject. The list of attributes as posted at drupal was Identity, Presence, Relationships, Conversations, Groups, Reputation, and Sharing. [Updated 9/30, 19:40 EDT to reflect Matt Webb’s work.]

    I just went through the list for this semester’s Social Software class at ITP, and re-aranged it, because the list is too big to be a subset of all social software (very few systems have formal support for Reputation or Relationships, for example), but much too small to be a superset of all interesting features (a potentially infinite list.)

    I think there are in fact only two attributes — Groups and Conversations — which are on the ‘necessary and sufficient’ list (though I have expanded the latter to Conversations or Shared Awareness, for reasons described below.) I doubt there are other elements as fundamental as these two, or, put another way, software that supports these two elements is social, even if it supports none of the others. (Wikis actually come quite close to this theoretical minimum, for reasons also discussed below.)

    Some of the remaining attributes are “technological signature” questions. These are not about essence so much as characterization — what kind of software is it? What are its core technological capabilities? I have four attributes that fall into this category, having added two to the drupal list: Identity and Roles, Presence, Naming and Addressing, and Valence. I think you can learn important things about any piece of social software by examining these four attributes. There are probably others.

    Finally, there are three leftovers from the original seven. These are essentially optional characteristics, which only pertain to a subset of social software and which were, I believe, wrongly included in the original list out of an excitement about recent innovations. The inessential characteristics included on the drupal list are Sharing, Relationships, and Reputation. Others are of course possible: Profiles? FOAF networking? etc.

    My version follows.

    ...continue reading.

    Comments (4) + TrackBacks (0) | Category: social software

    September 28, 2004

    Wiki+ patterns: TiddlyWiki and Web CollaboratorEmail This EntryPrint This Article

    Posted by Clay Shirky

    We’ve written a number of times over the last two years on various attempts to fuse the use patterns of wikis and weblogs — they are appealingly different in their uses but complementary in the way their users value them.

    Here are three recent entries in that category:

    TiddlyWiki, a wiki-patterned note-taking app written by jeremy Ruston and designed to run entirely in the browser. No set-up at all, with the acknowledged skeleton in the closet that it can’t save things easily without access to a server. PhPtiddlyWiki is Patrick Curry’s attempt to fix that omission by tying a TiddlyWiki to a MySQL db through PHP.

    The fascinating thing to me about TiddlyWiki isn’t so much the lowered set-up costs as the way a user page is built up during a session. Content creation is pure wiki, down to using CamelCase to specify new bits of content. The units of content themselves, however, are pure blog — post rather than page oriented, with multiple posts per page. And the user interaction is, I think, unique. Clicking on a post moves that post into the blog-standard central column, placing it underneath any posts the user previously clicked on, so the page is not reverse-crhonological by date but reverse-chronological by user interaction — page depth as history (though oddly it switches modes when you are clicking on content within a post in the center column — there the content opens under, rather than on top of, the most recent item.

    Anyway, a lot of words to explain what is a very natural feeling but novel pattern of interaction — well worth a look.

    The third item is Web Collaborator, a wiki/discussion board fusion, designed as a free collaborative space. The bet there seems to be that the free-formness of the wiki can be imporved upon by making two categories of social behaviors on wikis explicit in the tool. First, they make a distinction between discussion about and creation of shared content, and provide a discussion board for the former. Second, the provide a contact list of people on the project, with some nesting of editorial controls.

    I’ve just watched student groups in this semester’s Social Software class use a wiki to coordinate group work, and most groups did both — listed their contacts as the frist thing they put on the wiki page, and shuttled, sometimes uncomfortably, between conversation and shared editing modes. And Liz will be pleased to see that it passes her “Is it ugly?” test for wikidom generally.

    However, the discussion board on WebCollaborator seems badly designed, suffering from tUI issues — too much white space makes comments standalone rather than conversational — to a disconnect with the mission — the conversation is all macro, and I couldn’t see any way to tie a particular discussion to a particular piece of shared content, making it hard to focus the group on proposed edits or changes.

    The overall pattern is interesting, however, and they seem to have come with an appreciation for the wiki form and a desire to make a few simple changes that support existing social patterns, rather than a wholesale makeover.

    Comments (0) + TrackBacks (0) | Category: social software

    MSFT releases FelxWiki as Open Source projectEmail This EntryPrint This Article

    Posted by Clay Shirky

    We wrote about Microsoft’s FlexWiki project last December. Now eWeek is reporting that Microsoft is releasing FlexWiki code under an Open Source license. (Code is available on Source Forge, though it indicates that is is extensions to FlexWiki — I am not sure when or where the full codebase will be released..)

    Comments (0) + TrackBacks (0) | Category: social software

    September 27, 2004

    Ethan Zuckerman on bias in WikipediaEmail This EntryPrint This Article

    Posted by Clay Shirky

    Good Ethan Zuckerman post on systemic bias in the Wikipedia, and on a proposal called CROSSBOW (Committee Regarding Overcoming Serious Systemic Bias On Wikipedia), to address the problem:

    Amazing though it is, Wikipedia is not flawless. It’s got a problem common to almost all peer production projects: people work on what they want to work on. (This “problem” is probably the secret sauce that makes peer production projects work… which is what makes it such a difficult problem to tackle.) Most of the people who work on Wikipedia are white, male technocrats from the US and Europe. They’re especially knowledgeable about certain subjects - technology, science fiction, libertarianism, life in the US/Europe - and tend to write about these subjects. As a result, the resource tends to be extremely deep on technical topics and shallow in other areas. Nigeria’s brilliant author, Chinua Achebe gets a 1582 byte “stub” of an article, while the GSM mobile phone standard gets 16,500 bytes of main entry, with dozens of related articles.

    It is the hallmark of working open source projects that criticism tends to lead to better code rather than just more arguing; the Wikipedia seems to have that same pattern down.

    Comments (2) + TrackBacks (0) | Category: social software

    September 24, 2004

    Social sharing service tutorialEmail This EntryPrint This Article

    Posted by Clay Shirky

    Interesting guide to building social services featuring “implicit social discovery”, the pattern behind, Flikr, and Webjay.

    The service we build will let you:

    1. Publish observations or ‘stuff’ onto a website.
    2. Categorize it variety of ways.
    3. Pivot on yours or others observations to discover other related topics or persons.

    Our work will be modelled on newly emerging services including, Flickr and Webjay . The code itself will be a rewrite based on what I’ve learned from developing Thingster and BooksWeLike over the last year.

    Social content services have a strong emphasis on implicit social discovery. Users use these services to organize their own content for later recollection. But since the services are public, other users can peek into the collective space, and discover similar items, topics or persons. We’re going to look for opportunities in this project to stress the ‘synthesis’ aspect of social discovery; to escape from the pattern of curated collections managed and presented by one person.

    It’s cool to see features become patterns become platforms, and I love the ‘implicit social discovery’ label.

    Comments (3) + TrackBacks (0) | Category: social software

    September 15, 2004

    SIPShare: P2P SIP-based filesharing from EarthlinkEmail This EntryPrint This Article

    Posted by Clay Shirky

    Acronym-laden craziness! Earthlink has released SIPShare, a proof-of-concept P2P filesharing network that uses Session Initiation Protocol (SIP), a tool originally designed for voice and video communications, to set up file sharing networks.

    EarthLink believes an open Internet is a good Internet. An open Internet means users have full end-to-end connectivity to say to each other whatever it is they say, be that voice, video, or other data exchanges, without the help of mediating servers in the middle whenever possible. We believe that if peer-to-peer flourishes, the Internet flourishes. SIPshare helps spread the word that SIP is more than a powerful voice over IP enabler —- much more. SIP is a protocol that enables peer-to-peer in a standards-based way. The emerging ubiquity of SIP as a general session-initiation enabler provides a rare opportunity to offer users all manner of P2P applications over a common protocol, instead of inventing a new protocol for each new P2P application that comes along.

    Written in Java, with a BSD-style license, so it should be extensible in a way that Skype, Grouper, et al are not. Will be interesting to see if anyone uses this as a base for file-sharing + group communications.

    Comments (0) + TrackBacks (0) | Category: social software

    Stowe Boyd on Pay-for-Play in the YASNS worldEmail This EntryPrint This Article

    Posted by Clay Shirky

    Over at Get Real, Stowe notices that Ryze is starting to charge users for access to parts of the overall network, and considers the issues that pay-for-play in social networks raises:

    It’s not that [Pay for Play] is shady, since members theoretically know what they are getting into when they sign up (leaving aside the issue of changing the policies after the fact), but it starts to raise questions:

    * People want to be networked and meet others with whom to do business, so it makes sense to be listed in the ‘yellow pages’ of the future, which is what these services seem to be tending toward. But if it is a ‘yellow pages’ model, shouldn’t people pay to be listed?
    * If it is, on the other hand, a telephone exchange model, certainly the ones making the call (making the search) should pay.
    * If it is a dating service model, people want to get hooked up with people meeting their profiled interests and (theoretically) no one else, and therefore, the service should be managing things so that unwanted contact does not happen.

    So, it looks like we are evolving some scary, blendo model of business, here. I am free to join, but I don’t have the rights of the paying members who can (in some circumstances) see me when I can’t see them. This inequality is troubling, but parallels other fee-for-rights movements, like paid travel lanes in public highways. But since, in principal, I want to be contacted in some circumstances this should be ok, right? Well, only so long as I am never spammed, and it seems likely that those paying for the paid memberships are more likely to be using the service to sell, sell, sell.

    Comments (0) + TrackBacks (0) | Category: social software

    September 14, 2004

    CFP on Virtual CommunitiesEmail This EntryPrint This Article

    Posted by Clay Shirky

    ACMs SIGGROUP has a call for papers on novel approaches to virtual community (Due date: Jan 15, 2005), with the charming title of Less of You, More of Us: The Political Economy of Power in Virtual Communities, explicitly trying to counter some of the research biases present in work to date:

    For example, there are a large number of researchers inquiring into the recent blogging phenomenon, but I have heard many explicitly exclude technologies/communities such as with his 3.8 million users (1.7 active), and discount the value of teenage bloggers, who are mostly female (67% of Livejournal users). Because researchers tend to cover familiar territories, we encourage authors to explore alternatives. Our issue will provide researchers with the opportunity to expose the readership to a wider sense of virtual community and what is going on at the edges of the event horizon.

    For my money, of course, the most important research bias to undo is the bias that regards the use of social software as mainly leading to virtual communities, with real-world ties between participants being regarded as an unusual occurrence. Most communities regarded as ‘virtual’ have at least some good old-fashioned face-to-face interaction among some of the members, and as Meetup et al have showed us, that trend increases as the density of internet users grows.

    Nevertheless, it looks like an interesting CFP, and their interest in non-traditional source of insight could open the door for some much needed conversation between academics and practitioners.

    Comments (0) + TrackBacks (0) | Category: social software

    September 12, 2004

    Grouper: Groove meets WASTE...Email This EntryPrint This Article

    Posted by Clay Shirky

    Grouper, a new entrant in the category of small-group P2P apps. (The N^2 problem is only a problem if N is large…) It’s the usual mix of “communications plus file sharing for small groups” that we know from Groove, Bad Blue, and WASTE, but it claims (Windows only, so I can’t test it at home) to have put a lot of effort into ease of use. It advertises thumbnailing of pics, streaming of files from remote users, and no adware or spyware.

    If any M2M users have tried it, we’d love comments or pointers to other posts.

    Comments (4) + TrackBacks (0) | Category: social software

    September 9, 2004

    Felten on WikipediaEmail This EntryPrint This Article

    Posted by Clay Shirky

    Continuing the examination of the value of the Wikipedia, Ed Felten compares Wikipedia and Britannica

    Overall verdict: Wikipedia’s advantage is in having more, longer, and more current entries. If it weren’t for the Microsoft-case entry, Wikipedia would have been the winner hands down. Britannica’s advantage is in having lower variance in the quality of its entries.

    The thing I love about this post is that it turns the original complaint on its head. The Syracusan Critique is that the wikipedia can’t be good because it isn’t authoritative, which has a precursor assumption that Collin Brooke brilliantly glosses as “Authority/trustworthiness/reputation/credibility is something that pre-exists the research.” Falstodt the Syracusan presumes that if you cannot point to a pre-existing authority, the content itself is inherently less valuable (his phrase damning the Wikipedia is “without any credentials”.) Felten, by contrast, assumes that the value of content can be derived, rather than presumed, and begins by comparing individual articles, a process that cuts through trivial dismissals, and starts the real, and hard, work of talking about actual strengths and weaknesses.

    Comments (0) + TrackBacks (0) | Category: social software

    Progressive trust and Intimacy gradientsEmail This EntryPrint This Article

    Posted by Clay Shirky

    Two interesting posts at Life With Alacrity. First, thoughts on the growth of progressive trust in real human relations, and what it means for technology:

    Computer trust rarely works the way that human trust does. It starts with mathematical proofs—that such and such a mathematical algorithm is extremely difficult, thus something built on it must also be difficult. These are then built on top of each other until a system is created. It often seeks a level of “perfect trust” that is rarely required by human trust.

    One of the reasons why I chose to back the then nascent SSL (Secure Sockets Layer) Standard back in 1992-3, was that I felt that it much better mapped to the progressive trust model, and thus to human trust, then did its competitors.

    At the time, the SET standard was backed by all the major players—Visa, Mastercard, Microsoft, etc. […] But SSL starts out very simple—first it just connects two parties, then it establishes simple confidentiality between them. If one party wants more confidentiality, they can upgrade to a stronger algorithm. Then one party can request a credential from the other, or both can.

    Then a post, Intimacy Gradient, on architectural patterns that may have relevance to the design of social software:

    Refuge and prospect come from the landscape architect Jay Appleton. Prospect is a place where we can see others, and refuge is a place were we can retreat and conceal ourselves. A specific prediction of his theory is that people prefer the edges of a space more then the middle. Often prospect and refuge are in conflict, as a prospect tends to be expansive and bright whereas a refuge is small and dark, but there are cases where they are combined in one place; this is why we value private homes with a spectacular view so much, and why we pay so much to stay at scenic retreats. So what are the edges of our social spaces? Are there ways that we can signal either prospect and refuge?

    Comments (1) + TrackBacks (0) | Category: social software

    Educause on wikis in the academyEmail This EntryPrint This Article

    Posted by Clay Shirky

    Good Educause post on wikis in the academy. It includes a general overview of wikis that will be familiar to anyone reading M2M, but also some specific observations about wikis in academic settings:

    Indeed, an instructor could structure and regulate interaction to such an extent that the wiki is effectively transformed into a stripped-down course management system. But doing so risks diluting the special qualities that make wikis worth using in the first place, with the result being, in the words of Heather James, “pumped-up PowerPoint.” James has described the experience of using wikis in her teaching as her “brilliant failure.” She regrets that she “changed the tool, but did not change the practice,” and failed to account for the “great potential in this tool to be completely disruptive (in a good way) to the classroom setting.” With the benefit of hindsight, she concludes that for wikis to fulfill their promise, “the participants need to be in control of the content—you have to give it over fully.”26 This process involves not just adjusting the technical configuration and delivery; it involves challenging the social norms and practices of the course as well.

    Update: Ross also quotes this piece, in his discussion on anonymity and privacy in wikis.

    Comments (0) + TrackBacks (0) | Category: social software

    Joel On Software on social interfacesEmail This EntryPrint This Article

    Posted by Clay Shirky

    Great Joel Spolsky post on getting social interfaces right, with great observations about the social appropriation of small UI effects:

    Usenet clients have this big-R command which is used to reply to a message while quoting the original message with those elegant >’s in the left column. And the early newsreaders were not threaded, so if you wanted to respond to someone’s point coherently, you had to quote them using the big-R feature. This led to a particularly Usenet style of responding to an argument: the line-by-line nitpick. It’s fun for the nitpicker but never worth reading. (By the way, the political bloggers, newcomers to the Internet, have reinvented this technique, thinking they were discovering something fun and new, and called it fisking, for reasons I won’t go into. Don’t worry, it’s not dirty.) Even though human beings had been debating for centuries, a tiny feature of a software product produced a whole new style of debating.

    Comments (0) + TrackBacks (0) | Category: social software

    August 25, 2004

    FolksonomyEmail This EntryPrint This Article

    Posted by Clay Shirky

    Folksonomy, a new term for socially created, typically flat name-spaces of the ilk, coined by Thomas Vander Wal.

    In commentary on Atomiq, Gene Smith, who generally likes the idea, lists some disadvantages of folksonomies:
    On the other hand, I can see a few reasons why a folksonomy would be less than ideal in a lot of cases:
    * None of the current implementations have synonym control (e.g. “selfportrait” and “me” are distinct Flickr tags, as are “mac” and “macintosh” on
    * Also, there’s a certain lack of precision involved in using simple one-word tags—like which Lance are we talking about? (Though this is great for discovery, e.g. hot or Edmonton)
    * And, of course, there’s no heirarchy and the content types (bookmarks, photos) are fairly simple.

    A lot of this parallels the discussion around the continuing development and use of I am in the “Wenn ich Ontology höre … entsichere ich meinen Browning” camp, so I think Smith’s points are not so much absolute disadvantages as choices.

    Synonym control is not as wonderful as is often supposed, because synonyms often aren’t. Even closely related terms like movies, films, flicks, and cinema cannot be trivally collapsed into a single word without loss of meaning, and of social context. (You’d rather have a Drain-O® colonic than spend an evening with people who care about cinema.) So the question of controlled vocabularies has a lot to do with the value gained vs. lost in such a collapse. I am predicting that, as with the earlier arc of knowledge management, the question of meaningful markup is going to move away from canonical and a priori to contextual and a posteriori value.

    Lack of precision is a problem, though a function of user behavior, not the tags themselves. allows both heirarchical tags, of the weapon/lance form, as well as compounds, as with SocialSoftware. So the issue isn’t one of software but of user behavior. As David pointed out, users are becoming savvier about 2+ word searches, and I expect folksonomies to begin using tags as container categories or compounds with increasing frequency.

    No heirarchy I have a hard time as seeing as inherently problematic — heirarchy is good for creating non-overlapping but all-inclusive buckets. In a file-system world-view, both of those are desirable characteristics, but in a web world-view, where objects have handles rather than containment paths, neither characteristic is necessary. Thus multiple tags “skateboarding tricks movie” allows for much of the subtlety but few of the restrictions of heirarchy. If heirarchy was a good way to organize links, Yahoo would be king of the hill and Google an also-ran service.

    There is a loss in folksonomies, of course, but also gain, so the question is one of relative value. Given the surprising feedback loop — community creates folksonomy, which helps the community spot its own concerns, which leads them to invest more in folksonomies — I expect the value of communal categorization to continue to grow.

    Comments (5) + TrackBacks (2) | Category: social software

    August 21, 2004

    Multiply, spam, and economic incentivesEmail This EntryPrint This Article

    Posted by Clay Shirky

    Stowe, reading my earlier Multiply rant, responds saying Multiply isn’t spam, and says that we need a statement of purpose for social networks to adhere to.

    I’m more pessimistic than he; I believe that Multiply join messages are spam. Now spam has the “I know it when I see it” problem, so to talk carefully about it requires a specified definition. Here’s mine — spam is unsolicited mail, sent without regard to the particular identity of the recipient, and outside the context of an existing relationship.

    Anyone sending me mail because I am on a list I haven’t asked to be on; without having a reason to think that I, in particular, would want this mail; and without us already knowing one another, is spamming me. In particular, ads sent to me as a member of a category, no matter how targeted, count, in this definition, as spam. You could be advertising a new brand of gin specially brewed for Brooklyn-dwelling Python hackers who like bagpipe music and that mail would still be spam.

    If you adopt this definition, even just for the sake of argument, it’s pretty clear that Multiply fails the first and second tests. I did not ask for mail from them, and they are not sending me mail because they know me — they simply have my address on a list furnished by my friends. (IAQH.*) I think where Stowe and I may disagree is in point #3: do I have an existing relationship with the sender of the mail?

    ...continue reading.

    Comments (8) + TrackBacks (0) | Category: social software

    August 20, 2004

    Multiply and social spam: time for a boycottEmail This EntryPrint This Article

    Posted by Clay Shirky

    I’ll go David’s complaint about Multiply one better. Multiply are spammers, and should be treated as such, as should every other service that uses their tactics.

    Imagine that you were offered pills that promised to expand the length and diameter of your fingers, and you wanted some. (Maybe you’re a concert pianist or something — I actually don’t want to know the sordid details…) Now imagine that you could get the first month’s pills for free, if you just uploaded your entire address book and gave the pill manufacturer permission to make the same offer to everyone listed there, in email sent out under your name. Would you do it? Because that is what you are doing if you use Multiply.

    Here is what is happening: anyone launching a new YASNS has to work much harder to get users than in the old days, because the concept is so well established now. Furthermore, the existence of a social profile elsewhere means nothing to Multiply. Therefore they have every incentive to spam non-users mercilessly, because if they can wear them down until they join, great, and if they never join, who cares?

    But Multiply are not ordinary spammers, since they have the email addresses of your friends, and permission to use them. When Jenna NoLastname writes me saying “Want to meet you!”, my spam filter handles it, but when mail from my friend Schmendrick J. Subramanian shows up, my spam filter lets it through, because I’ve known Schmend since Back In The Day.

    ...continue reading.

    Comments (10) + TrackBacks (0) | Category: social software

    August 17, 2004

    The Great Scam: ReactionsEmail This EntryPrint This Article

    Posted by Clay Shirky

    I pointed to The Great Scam [new cached link] over the weekend, a first-person narrative of a scam perpetrated by Nightfreeze in the online game EVE. (Before I go into what caught my eye about it, I want to rectify an omission in my earlier post. The Great Scam contains derogatory references to women and minorities. I should have put a warning in the original pointer; my apologies.)

    The piece is most interesting not as a story but as an artifact — the author does not set out to explain to much as describe the events in question, and in doing so, ends up documenting several areas of behavior that may be of interest to M2M readers:

    • The use of out-of-band communications tools
    • Issues of identity and presentation
    • Questions of constitutional legitimacy

    ...continue reading.

    Comments (4) + TrackBacks (0) | Category: social software

    i-Neighbors: Local social capitalEmail This EntryPrint This Article

    Posted by Clay Shirky

    i-Neighbors, a service to generalize local social networks.

    Form the About page:

    I-neighbors was created by a team of researchers at the Massachusetts Institute of Technology (MIT). These services were designed to encourage neighborhood participation and to help people form local social ties. We believe that the Internet can help people connect to their local communities and to create neighborhoods that are safer, better informed, more trusting, and better equipped to deal with local issues. I-neighbors helps communities build “neighborhood social capital” by providing a place for neighbors to find each other, share information and work together to solve local problems.

    The proposed transformation here is similar to the change from one-off hosting of mailing lists to Yahoo Groups — instead of a set of one-off services, a neighborhood can use an existing template to start a whole related set of services — netowrking, photos, local reviews — all at once.

    The core intuition is that ‘neighborhood’ is a concept that maps well to such services. Current services that have strong geographic components include Meetup, UrbanBaby and Craigslist — the first two rely on strong affiliational ties, with geography as a filter, rather than vice-versa, and Craigs assumes that cities are units, and that people have different ranges for different functions — I’ll travel all the way across town to interview for a job, but not to go to a garage sale. It will be interesting to see what the ‘neighborhood first/ then other filters’ model produces.

    The lessons from UpMyStreet, a similar service in the UK that launched several years ago, are a bit mixed: the UK Postcode system is far more granular than that of the US, allowing them much more refined geo-location, but the place has also become a dumping grounds for racist and anit-immigrant feeling. Bob Putnam (he of Bolwing Alone) is doing some work on neighborhood social capital, and finds that high social capital correlates strongly with ethnic homogeneity — it will also be interesting to see, if i-Neighbors gets enough use, how that dynamic plays out here.

    UPDATE: danah posted about i-Neighbors as well, with interesting questions about the relations between race and neighborhood.

    Comments (4) + TrackBacks (0) | Category: social software

    XFN RelationshipsEmail This EntryPrint This Article

    Posted by Clay Shirky

    The madness of the age — making human relations explicit — continues in the form of XFN’s relationship profiles. Here, as a sample, is the entire range of possible romantic categories:

    - muse - Someone who brings you inspiration. No inverse.
    - crush - Someone you have a crush on. No inverse.
    - date - Someone you are dating. Symmetric. Not transitive.
    - sweetheart - Someone with whom you are intimate and at least somewhat committed, typically exclusively. Symmetric. Not transitive.

    Muse, crush, date, sweetheart. The whole of romantic or sexual feeling in not just four words but those four?

    The odd thing about about efforts like this is not merely the lack of completeness. Attempting to get to completeness would be admitting defeat, since the goal is simplification. The odd thing is that even the few proposals there are are obviously wrong.

    “Date” is not the word for someone you are dating, and everyone knows it except the authors of this list. You can be my date to the prom without it ever being an ongoing thing; meanwhile, someone you are dating is never referred to as your date, but as your boyfriend of girlfriend. Ditto ‘sweetheart’, where the assumption is definite on intimacy and lukewarm on committment, when usage would indicate the opposite balance. And so on.

    The best part, though, is the rationale:

    There were a whole pile of love, romance, and sexually oriented terms we considered and discarded. Some were rejected on the grounds they were unnecessary—for example, polyamorous individuals can indicate their other partners using values already defined (having two links marked sweetheart or spouse, for example). Others were left out because they did not fit with the desire to keep XFN simple. The current set seems to us to accurately capture a sufficiently detailed range of romantic feelings without becoming overwhelming.

    You really can’t make this stuff up. “We left a bunch of stuff out because when you try to model all the ways people really talk about attraction and intimacy and attachment, it just seems messy.” The thought that maybe the domain they are trying to model is messy seems never to have crossed their minds.


    A special note is merited for the omission of a term to describe a person to whom one is engaged. The terms “fiancé” and “fianceé” are gender-specific, which was a problem. We also decided that describing engagement should be left out since it is intended as a transitional state of affairs, as a prelude to marriage (and thus the value spouse, which is a less intentionally temporary relationship).

    Earth to XFN: Most romantic relations are temporary. Fiance is a more formal state of affairs than sweetheart. Despite the fact that it doesn’t fit with your model, it is treated as a real category by actual people. Throwing out real-world behavior because it doesn’t fit your model is supposed to make you question your model, no?

    It’s as if the creation of a list is meant to seem complete, because lists are discrete, and domains to be modelled are also supposed to be discrete…

    Comments (7) + TrackBacks (0) | Category: social software

    August 16, 2004

    Social Origin of Good IdeasEmail This EntryPrint This Article

    Posted by Clay Shirky

    Ronald Burt, who created the ‘social holes’ network measure (find out where the connections between groups aren’t, and look for value in bridging, roughly), wrote a paper last year on the Social Origin of Good Ideas (PDF):

    A theme in the above work is that information, beliefs and behaviors are more homogenous within than between groups. People focus on activities inside their own group, which creates holes in the information flow of information across structural holes. People with contacts in separate groups broker the flow of information across structural holes. Brokerage is social capital in that brokers have a competitive advantage in creating value with projects that integrate otherwise separate ways of thinking or behaving.

    Much of the paper is focused on sturcutral holes in business settings, arguing that brokers create much fo the value we associate with innovation.

    On a related note, here’s a presentation danah co-wrote on social holes in email. The animation of the visualization, referenced in the presentation, is here

    Comments (0) + TrackBacks (0) | Category: social software

    August 15, 2004

    Must Read: The Great ScamEmail This EntryPrint This Article

    Posted by Clay Shirky

    Terrific account of scamming other players in EVE, a massively multiplayer game set in space. It’s got everything — innocent fun, bitter disillusionment, vows of revenge, close calls, a dastardly plan, a network of mostly invented collaborators, and an ending that make the whole thing more astonishing still.

    This is one of the great first-person narratives of game participation, and touches on several themes we care about here. I’ll write about it later, but for now, I won’t bother commenting, or even quoting from it. It’s long, but it deserves to be read in full.

    It’s at, but may still be slashdotted, so check as well. If anyone knows of an alternate and more persistant URL, lemme know.

    Comments (4) + TrackBacks (0) | Category: social software

    August 12, 2004

    Duncan Watts on Collective IntelligenceEmail This EntryPrint This Article

    Posted by Clay Shirky

    Great Duncan Watts piece on the dangers of centralized intelligence, his argument being that while centrally controlled organizations can respond well to situations they’ve forseen, only decentralized but coordinated groups can respond to unexpected catastrophe. (Duncan is the guy who worked out the Small Worlds pattern, providing intellectual backstop to Milgram’s Six Degrees work, so he knows whereof he speaks on the subject of decentralized networks.)

    He covers the Japanese auto industry’s recovery after an earthquake, which he also describes in his Six Degrees book, but adds this more recent example:

    Perhaps the most striking example of informal knowledge helping to solve what would appear to be a purely technical problem occurred in a particular company that lost all its personnel associated with maintaining its data storage systems. The data itself had been preserved in remote backup servers but could not be retrieved because not one person who knew the passwords had survived. The solution to this potentially devastating (and completely unforeseeable) combination of circumstances was astonishing, not because it required any technical wizardry or imposing leadership, but because it did not. To access the database, a group of the remaining employees gathered together, and in what must have been an unbearably wrenching session, recalled everything they knew about their colleagues: the names of their children; where they went on holidays; what foods they liked; even their personal idiosyncrasies. And they managed to guess the passwords.

    Comments (7) + TrackBacks (0) | Category: social software

    YASNS Watch: What up with Multiply?Email This EntryPrint This Article

    Posted by Clay Shirky

    Reminiscent of the Zerodegrees spamming incident, I’m getting spammed by requests to join Multiply. Is this happening to anyone else?

    As for the service itself, so far, it looks like a standard YASNS, with an emphasis on broadcast messages, in the manner of Orkut in the bad old days:

    Multiply can also be used to compose and send messages, similar to e-mail but much more powerful. With e-mail your audience is limited to the specific people in your address book. Multiply messages - or multi-messages, as we call them - can reach the entire network of people you are connected to through mutual friends.

    Channeling danah, note the rhetoric of ‘powerful’ messaging here. Power exists in differentials, and here the power being advertised is clear