The Role of Experts In Wikipedia 266
Hugh Pickens writes "Episteme, a magazine about the social dimensions of knowledge, has a special issue on the epistemology of mass collaboration, with many of the articles focusing on Wikipedia. One of the most interesting articles is by Lawrence M. Sanger on the special role of experts in the age of Wikipedia. Sanger says the main reason that Wikipedia's articles are as good as they are is that they are edited by knowledgeable people to whom deference is paid, although voluntarily, but that some articles suffer precisely because there are so many aggressive people who 'guard' articles and drive off others (PDF), including people more expert than they are. 'Without granting experts any authority to overrule such people, there is no reason to think that Wikipedia'a articles are on a vector toward continual improvement,' writes Sanger. Wikipedia's success cannot be explained by its radical egalitarianism or its rejection of expert involvement, but instead by its freedom, openness, and bottom-up management and there is no doubt that many experts would, if left to their own devices, dismantle the openness that drives the success of Wikipedia. 'But the failure to take seriously the suggestion of any role of experts can only be considered a failure of imagination,' writes Sanger. 'One need only ask what an open, bottom-up system with a role for expert decision-making would be like.' The rest of the articles on the epistemology of mass collaboration are available online, free for now." Sanger was one of the founders of Wikipedia, and of its failed predecessor Nupedia, who left the fold because of differences over the question of the proper role of experts. Sanger forked Wikipedia to found Citizendium, which we have discussed on several occasions. After 2-1/2 years, Citizendium has a few tenths of a
percent as many articles as Wikipedia.
Got a better way to do things? (Score:5, Insightful)
Do we have any good reason to trust Sanger as anything other than a provocateur? What is the meat of the analysis? That open editing and cooperation is what explains wikipedia's success? I'll agree with him there. And that control of articles or processes by internal "experts" is damaging to that open editing and cooperation? I'll take two, please. He's the big problem.
We don't really know how to make a reasonably reliable, open and comprehensive encyclopedia without some deference to "local fiefdoms". We just don't. People don't contribute for money or fame. They don't have marching orders on which articles to keep free from vandalism or improve to featured status. They control their own production. Where that is the case they will bring themselves to edit on subjects they like and edit those articles in order to bring the distribution of coverage to their liking. We have to allow a little of that because it is those people who keep it from being a nuthouse. Those people spend 20-30 hours on wikipedia a week. They watch recent changes to keep subtle vandalism out. They fight back against civil POV pushers. They are an absolute necessity.
To they come with drawbacks? Hell yes. There are probably thousands of people who have avoiding wikipedia as editors because their first edits were reverted--even though they might have been productive. I find lots of those reversions and usually don't get a cooperative attitude from editors when I call them on it. Those people make subtle cultural distinctions (I like this and not that). Those people form cliques and cabals. Those people make processes and bureaucracy.
But I don't have a better way of organizing all of that free labor. Does Larry? Do you?
I usually read the discussion or history (Score:4, Insightful)
to get where the controversy in the articles/subjects are, so as not to be led astray by any one current revision. I don't get the big deal about doing that extra bit of work.
Expert FAIL (Score:5, Insightful)
The purpose of Wikipedia is to approach consensus, not truth. The purpose of experts is to forward their own agenda ;)
People who think either Wikipedia or experts are interested in the truth are likely to be confused.
Re:Expert FAIL (Score:5, Insightful)
The purpose of Wikipedia is to approach consensus, not truth.
I guess. This kind of critique gets pretty old. The whole point of moving away from "THE TRUTH" was to suggest that no one editing on wikipedia has access to "THE TRUTH". I'm not an expert. You're not an expert. Sure, we probably have our areas of expertise, but they aren't verifiable in a pseudonymous editing environment. In the absence of that verification, we have to trust references, not people. If I say "believe me, this is THE TRUTH", the right response is "Wikipedia isn't interested in THE TRUTH, do you have a source for that."
/., their blog, or their cat and declare that Wikipedia is only interested in groupthink. Lots of time groupthink does grip wikipedia--just like any other organization. People see comments from editors they know and trust and respond accordingly. New people are often distrusted. These aren't features unique (or even uniquely salient) to wikipedia. They are features found in any community, large or small. Conflating the existence of groupthink with some underlying community desire> to govern through groupthink is inaccurate.
It's cute to twist that around, but neither you nor I are Steven Colbert. We won't make it anywhere near as funny. To misread it to think "Oh, wikipedia is only interested in groupthink" is to miss the point. Lots of so called experts come on wikipedia and demand that people listen to them on the basis of their alleged expertise. When people (rightly) refuse to listen to them, those people storm off to
Really, we don't mind the truth, so long as it has a little blue superscripted number after it.
It isn't "experts" that are needed... (Score:5, Insightful)
You may complain about the quality of the comments on slashdot, but compare it to somewhere without any karma system. (this article [sfgate.com] sums up the problem with pure anonymity, and quite humorously so) Slashdot's system is not perfect, but it is a start in the right direction. I wonder how much wikipedia could be improved with a really good system. For instance, people with low karma would have their changes not show up immediately by default, or would be flagged as questionable, or what have you. People who didn't have a history of posting "good" stuff would tend to have few eyeballs ever see their stuff. There is a ton that could be done. It's tough to make it ungameable, but not impossible.
Experts are few and want to be paid. (Score:5, Insightful)
Even leaving out the political issues, Experts are few ,and when well known, consider charging a lot for their work and would probably only devote time to getting published in a scholarly journal rather than some random website.
It would be like running an open source project where the only people who are allowed to work on it are those people who hold a PHD or are certified to have 10 years experience programming with a major corporation.
Re:Expert FAIL (Score:4, Insightful)
Re:It isn't "experts" that are needed... (Score:1, Insightful)
" wonder how much wikipedia could be improved with a really good system. For instance, people with low karma would have their changes not show up immediately by default, or would be flagged as questionable, or what have you. People who didn't have a history of posting "good" stuff would tend to have few eyeballs ever see their stuff. "
I'm not sure a karma system would work, it would skew heavily in the favor "I agree so I will mod you up" kind of system that slashdot has. A lot of comments "insightful" and "informative" are rated by mods who don't have a clue but "it sounds insightful/informative, I agree with what you are saying, even though I might not have a clue"
Not only that, like DIGG you could 'game' the karma system. Kind of how sellers on ebay leave each other feedback.
Re:Got a better way to do things? (Score:5, Insightful)
Yeah there really isn't a much better way. I ran into this recently on an article on Wikipedia on the Super Nintendo release date. I was a video game reviewer in the early 90's and have a letter signed by Nintendo's PR firm stating I was getting a review system before launch (which was a pretty typical thing for them to do). The letter was dated the same date as when the Wikipedia article said the system was actually released.
Given this evidence I scanned the letter and posted it to let them know their date was off. Their response was that they couldn't use the letter as proof the date was wrong because they only used published sources of information. Unfortunately the only published sources they had were a handful of websites currently online that had the wrong date written down (no doubt copied from each other).
At first I was taken aback by this as it was a bit odd that they would turn down physical evidence, but after thinking about it, it was obvious they didn't know me from Adam and can't just take people's word for things at face value, otherwise people could "prove" whatever they wanted. Those kinds of check and balances probably produce entries that aren't always perfect, but it's a lot better than the alternative in my mind.
Re:Got a better way to do things? (Score:4, Insightful)
Well, unless you count the fact that collectively Wikipedia's articles have quite obviously been on a vector toward continual improvement since Wikipedia started.
This is not obvious at all. Let alone "quite obvious". Indeed, it is false.
Re:Got a better way to do things? (Score:3, Insightful)
You can "prove" things via publishing that letter in a book. The people with money still successfully control information through Wikipedia. What a waste.
The reasons are actually quite simple. (Score:5, Insightful)
Those with the most time on their hands wins.
Re:How can Wikipedia be taken seriously... (Score:5, Insightful)
It is just a different kind of writing than an original paper. People who make it out to be anti-academic or sneaky miss the boat entirely.
Re:Got a better way to do things? (Score:5, Insightful)
We love to talk shit about Wikipedia here on /,; talk about how Knol is going to beat it or how Citizendium is better or how you wouldn't use it as a source (duh). But when push comes to shove, do we have any good competing models of how an online encyclopedia should be made?
Well, I don't recall seeing full support here on /. for Knol or Citizendium. There seemed to be people on both sides of the aisle getting their voices heard. Many people attacked Wikipedia for its shortcomings, which you yourself agree exist. So, too, did many people attack the upstarts, with the standard arguments that come out whenever we discuss project forks or restarts here. Finally, there were people who had reasonable criticism for the upstart projects themselves.
However, Wikipedia itself has been extensively criticized throughout its history; famously, its ability to accumulate knowledge and remove vandalism works "only in reality, not in theory." So on that note, how can we say that we know that those systems are inherently that much worse? Many would argue, as you yourself hint, that Wikipedia's supremacy is eminent in its dominance, and in its success. Yes, Wikipedia has orders of magnitude more content than its competitors. Please consider, though, the possibility that this is simply the result of first mover advantage and network effects.
Wikipedia came on to the scene to find a fresh niche to fill. Earlier sites existed with similar goals to describe and categorize life (see: everything2 [slashdot.org] and h2g2 [bbc.co.uk]), but Wikipedia had a slightly different defined goal (be "the free encyclopedia") and software to ease the processes of collaboration and presentation. It took off like a shot and quickly established itself before it really had competition.
Today, Wikipedia has somewhere around 2.8 million articles and a dedicated community. It seems immediately obvious to me that it's impossible today to compete with Wikipedia from square one; any competitor would need to fork the project or have their own equally impressive database. A database that size needs a huge support structure, not only in infrastructure but also in terms of volunteers/workers to police content. It's not possible to get those things all at once. There might be enough people out there who would be interested in helping a different project, but there's no way to get in touch with all of them. New talent either joins up with Wikipedia, or becomes disinterested when they don't fit with that group.
Wikipedia got where it is because it was good enough at the right time. It does a lot right. It does some things wrong. It's not perfect. But then, I don't have to tell /. about technically inferior products dominating the marketplace due to familiarity...
Re:Got a better way to do things? (Score:5, Insightful)
An improvement towards what though? Most articles have settled down to reflect the viewpoint of the people that watch them. If you agree with that viewpoint, that's an improvement. If you don't it's not and you give up citing, editing or reading them.
Re:Expert FAIL (Score:5, Insightful)
nonsense. Wikipedia is meant to contain fact.
https://secure.wikimedia.org/wikipedia/en/wiki/Wikipedia:Verifiability [wikimedia.org]
"The threshold for inclusion in Wikipedia is verifiability, not truth"
I'm not surprised that the people who whine the most about Wikipedia don't understand what it's meant to do.
Re:Got a better way to do things? (Score:2, Insightful)
Except wikipedia doesn't believe it belongs on wikipedia, it is why it is taken down. If you want to start a wiki about various games and include all the infinite detail and trivia about it, you can. You can even get it linked in the external links. There is a certain subset of the population of editors who can't seem to tell the difference between an encyclopedia and everything every written, said, thought and just made up about a subject ever. That being said the 10,000 sub articles on pokemon needs to be nuked from orbit. Articles are meant to provide all the information the average person would find useful about a subject. Someone who is not a world of warcraft player, which is something 99.998% of the population of the world, doesn't remotely care what spells some paladin gets access to at level 27. It isn't remotely useful to their understanding of the subject. Articles are usually made by players, but not for players. Try being objective.
Re:Got a better way to do things? (Score:5, Insightful)
Which is why they just completed a six million dollar fund raising campaign. With cheap disk space, and cheap bandwidth, and volunteers doing the work... where is the money going?
Setting that aside, the problem isn't the cost of disk space or bandwidth - the problem is the unseen cost of maintenance. Every article on Wikipedia requires some portion of an editors time to maintain accuracy, completeness, coherency, etc..., and to clean up behind vandals. And there are only so many editors at any given time. Too many articles means rot accumulates in the corners and moves inward. Too many articles means too many stubs that remain untouched. Too many articles means an increasing number of articles that say the same thing from different points of views.
And frankly, based on a daily random sampling of articles, Wikipedia seems to be losing the battle.
Re:Got a better way to do things? (Score:4, Insightful)
Those kinds of check and balances probably produce entries that aren't always perfect, but it's a lot better than the alternative in my mind.
This is also why articles are deleted on wikipedia, the editors are biased by their cultural knowledge. If its believed to be correct by their peers, then an alternate and possibly correct view cant be published due to the editors belonging to the same peer group.
This is really difficult area when it comes to politics, groups/clubs, companies, history, ethnic, gender, minority, etc. You only get one side. Thus the reason it cant be used in colleges, its mass agreement by a very common peer group. And a good portion of the editors are very close in its peer groups.
Re:Got a better way to do things? (Score:4, Insightful)
An improvement towards what though? Most articles have settled down to reflect the viewpoint of the people that watch them. If you agree with that viewpoint, that's an improvement. If you don't it's not and you give up citing, editing or reading them.
And for non-controversial subjects, that generally results in a pretty good article.
For controversial subjects, if the article is bad it's normally because policies aren't being followed: for example, the 'Apollo Moon Hoax' article (whatever the title may be today as it gets renamed regularly) has been a disaster zone for years, but that's primarily because it's an enormous violation of WP:UNDUE.
Re:Expert FAIL (Score:4, Insightful)
The purpose of Wikipedia is to approach consensus, not truth.
And one way is to shut out the people who don't agree.
Re:Expert FAIL (Score:2, Insightful)
Lots of so called experts come on wikipedia and demand that people listen to them on the basis of their alleged expertise. When people (rightly) refuse to listen to them, those people storm off to /., their blog, or their cat and declare that Wikipedia is only interested in groupthink.
If they were really such experts, they should capable of citing adequate evidence to back up their claims.
No. (Score:4, Insightful)
Problem is not lack of experts... (Score:4, Insightful)
Re:Expert FAIL (Score:1, Insightful)
Re:Book references (Score:5, Insightful)
The end result was the same - my edits were rejected because they didn't match the commonly, and easily, available sources.
Well, it was more complex than the date - it was essentially an entire rewrite of half a dozen articles. On each, on the talk page, I noted my sources - and the same person (who had written all the articles and had most of the edits on them) promptly reverted my changes. (This was prior to 'citation needed'.) So I went to the talk page and explained that not only had I worked on this weapon in the Navy, but I had researched them for nearly twenty years and wished to share that research - and was promptly hit with the "no original research" hammer. (Never mind that any comprehensive historical article on the Wikipedia contains the same level of original research.)
Between that and several other examples of asshattery, I realized I didn't have time to fight one fool with too much time on his hands, let alone half a dozen or so. I left Wikipedia and have never been back.
Re:The Wikipedia is getting better (Score:5, Insightful)
Yeah, all those things have been rolled into one meta game - WP:TAA. (Wikipedia:Toss Around Acronyms) The end result the is the same - the win goes to he who has the most time on his hands.
Yeah, that's part of WP:NRETISRG. (Wikipedia:Not Really Enforceable Though It Sounds Real Good.)
Re:Got a better way to do things? (Score:5, Insightful)
Indeed, because I had a related problem. A series of related articles I wished to edit had considerable problems. I worked on the item described in one of the articles while I was in the Navy, I had the unclassified manuals at one elbow, at the other elbow I had a stack of expensive reference books... All were trumped because a handful of websites all referenced the same handful of coffee table books - and disagreed with me.
Those stories are probably legion. I've had problems having facts corrected on Wikipedia articles where I was the primary source (one about me in person, one about a project I lead) - and could easily prove so.
But to Wikipedia, if three newspapers all quote the same source that got, say, my birthdate wrong, that is "more reliable" than me sending in a scanned, signed and confirmed by a notary, copy of my birth certificate. In fact, that would be entirely discarded as a "primary source".
And that's why Wikipedia usually gets it right on common and readily available articles, and has a 50/50 on articles about obscure topics (other than nerdy geek stuff).
Re:Got a better way to do things? (Score:3, Insightful)
Well, unless you count the fact that collectively Wikipedia's articles have quite obviously been on a vector toward continual improvement since Wikipedia started.
Citation Needed.
Re:Got a better way to do things? (Score:3, Insightful)
Wikipedia, not being a living being, is utterly unable to believe anything. Some deletionist scum, on the other hand, can only get it up by exercising petty power by deciding what article stays and what goes. It's the closest the ever get to wielding power over life and death (I hope).
This is why I no longer contribute to Wikipedia and resist the impulse to correct any mistakes I notice: the reward is having my work deleted or reverted by some antisocial cretin who got kicked out from the Neo-Nazi party due to his excessive authoritarianism and has no other outlets for his resulting frustrations.
Mod me flamebait, it doesn't change the truth.
Today's featured article is about Thomas Cranmer, a archbishop of Canterbury in the early 1500's. The average person is unlikely to find that information at all useful for any purpose.
Someone who isn't a historian, which is something 99.998% of the population of the world, doesn't remotely care about who led a subset of English church half a millennium ago.
It isn't remotely useful, period.
Re:Got a better way to do things? (Score:4, Insightful)
Indeed, because I had a related problem. A series of related articles I wished to edit had considerable problems. I worked on the item described in one of the articles while I was in the Navy, I had the unclassified manuals at one elbow, at the other elbow I had a stack of expensive reference books... All were trumped because a handful of websites all referenced the same handful of coffee table books - and disagreed with me.
The real problem here is that the best sources are not online. On wikipedia an online reference is worth so incredibly much more because it's verifiable - not as truth but as something others can read. Seriously, if you quote page 234 of the operations manual, how many would ever check that? How many even have the book or would hunt it down at a library? Most likely, none of the them know if you're bullshitting them or not.
I know it's the long way around but you need to find or make credible online sources that reference the offline sources, then make the argument that your online sources are better than the ones currently used. Otherwise you'd see a lot of book spamming on wikipedia, where all the answers are in the spammer's book. Because of the delay from addition to fact checking plus the costs, it'd be an impossible problem. It's fair enough to reference famous books like Porter's book on Porter's theory but in general it would only undermine wikipedia.
"Expert" != "shadowy propagandist" (Score:5, Insightful)
The purpose of experts is to forward their own agenda
I'm not sure how tongue-in-cheek you're trying to be here, so if you are, then this is directed at the people who really believe this.
This is honestly one of the biggest and (to me) scariest and most incomprehensible problems with the American consciousness right now: the belief that not only do they, the average Americans, know better than the experts in their fields, but that those experts are, to a man, interested solely or primarily in putting forward their own "agenda"—which is necessarily something other than "educate people" or "show the truth".
What reasonable basis is there for believing that everyone who's highly educated is somehow trying to subvert society to some nefarious end??
Dan Aris
Re:Got a better way to do things? (Score:4, Insightful)
I didn't say I have a solution.
But when it is this obvious, you can't close your eyes and go "lalalala", pretending that there isn't a problem here.
Great post, and more thoughts (Score:3, Insightful)
Well, good summary, I very much agree!
I like to think of myself as one of those careful, involved editors who spends my free time "tending" and improving articles on my major areas of interest and expertise.
What I've never understood about Sanger's argument is... what is it in Wikipedia's current model that /prevents/ experts from using their expertise?
What makes an expert?
I'd guess that an expert knows many reliable sources of information that they can use as references in their edits, to correct inaccuracies and add new material. Wikipedia encourages that!
I also expect that experts have superior writing skills and can summarize information in their area of expertise effectively. Wikipedia encourages that too!
Finally, I expect that experts are used to defending their conclusions and evidence in a rational but skeptical setting, so they should be well-prepared to convince their fellow editors when disputes over content and emphasis arise (as is inevitable!). Wikipedia encourages that kind of debate too.
So... what exactly ARE the aspects of Wikipedia that hinder expert contributors? It's an honest question by me. Can anyone point out any specific issues or cases they're aware of?
Re:Got a better way to do things? (Score:3, Insightful)
It is not community who decides to delete a page or reverse an edit, it's an individual editor. The tendency of Wikipedia editors to take some article as their "own" and guard it jealously is infamous, and repeatedly referred in this very discussion.
Like I said: I've had my work deleted or reverted once too many times to bother anymore. It's not an issue of pet page but a general feeling of "fuck this shit, I'm outta here."
Actually, Hitchcock did just that. Not that it matters; we aren't talking about prose, we are talking about an encyclopaedia, a collection of information. An encyclopaedia is made better by adding information to it. You aren't supposed to read through it all, as you are in a novel, but only the bits that interest you.
So now the criterion has changed from "not interested in the subject", as it was for WoW paladins, to "interested in the subject". Funny how that goes.
No, you said that someone who isn't a WoW player isn't interested in the abilities of WoW paladin class of certain level. Well, you're absolutely right: they won't be. I'm not, for example. However, using that as a criterion for deleting said details is foolish, since doing it consistently means deleting pretty much all information about anything; after all, few people are interested in the details of something which doesn't interest them even in general.
No. Two things which might interest some people, but for most people couldn't care less. In fact I'd say that there are likely more people interested in the details of WoW gameplay than the details of the history of English church.
"Notable" is almost as ingenius as "interstate commerce" in that it can mean exactly what the editor wants it to mean. And just as the interstate commerce clause, it too is used to justify arbitrary abuse of authority. Or, as the saying goes: "Power corrupts, absolute power corrupts absolutely, and petty power corrupts all out of proportion to actual power."