The battle for Wikipedias soul

Updated: Mar 17 2008, 06:41am hrs
It is the biggest encyclopedia in history and the most successful example of user-generated content on the internet, with over 9 million articles in 250 languages contributed by volunteers collaborating online. But Wikipedia is facing an identity crisis as it is torn between two alternative futures. It can either strive to encompass every aspect of human knowledge, no matter how trivial; or it can adopt a more stringent editorial policy and ban articles on trivial subjects. These two conflicting visions are at the heart of a bitter struggle inside Wikipedia between inclusionists, who believe that applying strict editorial criteria will dampen contributors enthusiasm for the project, and deletionists who argue that Wikipedia should be more cautious and selective about its entries.

Consider the fictional characters of Pokemon, the Japanese game franchise with a huge global following, for example. Almost 500 of them have biographies on the English-language version of Wikipedia, with a level of detail that many real characters would envy. But search for biographies of the leaders of the solidarity movement in Poland, and you would find no more than a dozenand they are rather poorly edited.

Inclusionists believe that the disparity between Pokemon and Solidarity biographies would disappear by itself, if only Wikipedia loosened its relatively tight editorial control and allowed anyone to add articles about almost anything. They argue that since Wikipedia exists online, it should not have the space constraints of a physical encyclopedia imposed upon it artificially.

Surely there is no harm, they argue, in including articles about characters from television programmes who only appear in a single episode, say After all, since most people access Wikipedia pages via search, the inclusion of articles on niche topics will not inconvenience them. The ideal Wikipedia of the inclusionists would feature as many articles on as many subjects as its contributors were able to produce, as long as they were of interest to more than just a few users.

Deletionists believe that Wikipedia will be more successful if it maintains a certain relevance and quality threshold for its entries. So their ideal Wikipedia might contain biographies of the five most important leaders of Solidarity, say, and the five most important Pokemon characters, but any more than that would dilute Wikipedias quality and compromise the brand. The presence of so many articles on trivial subjects, they argue, makes it less likely that Wikipedia will be taken seriously, so articles dealing with trivial subjects should be deleted.

In practice, deciding what is trivial and what is important is not easy. How do you draw editorial distinctions between an article entitled List of nicknames used by George W Bush (status: kept) and one about vice-presidents who have shot people (status: deleted) Or how about Natasha Demkina: Russian girl who claims to have X-ray vision (status: kept) and The role of clowns in modern society (status: deleted)

To measure a subjects worthiness for inclusion (or notability, in the jargon of Wikipedians), all kinds of rules have been devised. So an article in an international journal counts more than a mention in a local newspaper; 10 matches on Google is better than one match; and so on. These rules are used to devise official policies on particular subjects, such as the notability of pornographic stars or diplomats.

Jimmy Wales, the founder of Wikipedia, has himself fallen foul of these tricky notability criteria. Last summer he created a short entry about a restaurant in South Africa where he had dined. The entry was promptly nominated for deletion, since the restaurant had a poor Google profile and was therefore considered not notable enough. After a lot of controversy and media coverage, the entry was kept, but the episode prompted many questions about the adequacy of the editorial process.

As things stand, decisions whether to keep or delete articles are made after deliberations by Wikipedias most ardent editors and administrators. Imagine you have just created a new entry, consisting of a few words. If a member of the Wikipedia elite believes that your submission fails to meet Wikipedias notability criteria, it may be nominated for speedy deletionin other words, removed right awayor regular deletion, which means the entry is removed after five days if nobody objects. If your article is selected for deletion, you may choose to contest the decision, in which case, you may be asked to provide further information.

Debates about the merits of articles often drag on for weeks, draining energy and taking up far more space than the entries themselves. Such deliberations involve volleys of arcane internal acronyms and references to obscure policies and guidelines, such as WP:APT (Avoid Peacock Termsterms that merely promote the subject, without giving real information) and WP:MOSMAC (a set of guidelines for Wikipedia articles discussing the Republic of Macedonia and the Province of Macedonia, Greece). Covert alliances and intrigues are common. Sometimes editors resort to a practice called sock puppetry, in which one person creates lots of accounts and pretends to be several different people in a debate so as to create the illusion of support for a particular position.

The result is that novices can quickly get lost in Wikipedias Kafkaesque bureaucracy. According to one estimate from 2006, entries about governance and editorial policies are one of the fastest-growing areas of the site and represent around one-quarter of its content. In some ways, this is a sign of Wikipedias maturity and importance: a project of this scale needs rules to govern how it works. But the proliferation of rules, and the fact that select Wikipedians have learnt how to handle them to win arguments, now represents a danger, says Andrew Lih, a former deletionist who is now an inclusionist, and who is writing a book about Wikipedia.

Mr Lih and other inclusionists worry that this deters people from contributing to Wikipedia, and that the welcoming environment of Wikipedias early days is giving way to hostility and infighting. There is already some evidence that the growth rate of Wikipedias article-base is slowing. Unofficial data from October 2007 suggests that users activity on the site is falling, when measured by the number of times an article is edited and the number of edits per month.

It may be that Wikipedians have already taken care of the low-hanging fruit, having compiled articles on the most obvious topics. But there is a limit to how much information a group of predominantly non-specialist volunteers, armed with a search engine, can create and edit. Producing articles about specialist subjects such as Solidarity activists, as opposed to Pokemon characters, requires expert knowledge from contributors and editors. If the information is not available elsewhere on the web, its notability cannot be assessed using Google.

To create a new article on Wikipedia and be sure that it will survive, you need to be able to write a deletionist-proof entry and ensure that you have enough online backing (such as Google matches) to convince the increasingly picky Wikipedia people of its importance. This raises the threshold for writing articles so high that very few people actually do it. Many who are excited about contributing to the site end up on the Missing Wikipedians pagea constantly updated list of those who have decided to stop contributing. It serves as a reminder that frustration at having work removed prompts many people to abandon the project.

Google has recently announced its own entry into the field, in the form of an encyclopedia-like project called Knol that will allow anybody to create entries on topics of their choice, with a voting system that means the best rise to the top. Tellingly, this approach is based on individualism rather than collaboration (Google will share ad revenues with the authors). No doubt it will produce its own arguments and unexpected consequences. But even if it does not turn out to be the Wikipedia-killer that some people imagine, it may push Wikipedia to rethink its editorial stance.

The Economist Newspaper Limited 2008