Stories
Slash Boxes
Comments

News for nerds, stuff that matters

Slashdot Log In

Log In

Create Account  |  Retrieve Password

Free Online Scientific Repository Hits Milestone

Posted by CmdrTaco on Wed Oct 08, 2008 09:24 AM
from the a-whole-lotta-smart-stuff dept.
ocean_soul writes "Last week the free and open access repository for scientific (mainly physics but also math, computer sciences...) papers arXiv got past 500,000 different papers, not counting older versions of the same article. Especially for physicists, it is the number-one resource for the latest scientific results. Most researchers publish their papers on arXiv before they are published in a 'normal' journal. A famous example is Grisha Perelman, who published his award-winning paper exclusively on arXiv."
+ -
story

Related Stories

This discussion has been archived. No new comments can be posted.
The Fine Print: The following comments are owned by whoever posted them. We are not responsible for them in any way.
 Full
 Abbreviated
 Hidden
More
Loading... please wait.
  • by eldavojohn (898314) * <my/.username@@@gmail.com> on Wednesday October 08 2008, @09:25AM (#25299367) Homepage Journal
    When I was a freshman at the University of Minnesota, a professor instructed us to use Arxiv as a resource (I think Citeseer was another but paled in comparison). A large part of my undergrad and grad school days were spent perusing Arxiv and sometimes implementing ideas I had read in the Computer Science section. My hard drive became strained by the sheer number of PDF/PS files in my user directory. My room was littered with papers printed off to read on the bus or at work. My base knowledge of computer science I owe to my professors, most of the things beyond that came from Arxiv.

    I owe a lot of my knowledge to that site. Here's to another 50,000 papers, Arxiv. And another and another and another ...

    Also, the Arxiv Physics blog [arxivblog.com] is a regular favorite in my Liferea news feed account.
    • Perhaps Arxiv works for the hard sciences, but for the social sciences and humanities giving people access to an online repository of papers doesn't necessarily mean that they can easily stay up to date with the field. I get the impression that a lot of current thought in the community of my field (linguistics) is passed on through relatively private e-mail lists and informal discussions at conferences, and might not be written down and published for years.
      • If it's so informal, I have to wonder how Sociology and Literary Criticism stay up-to-date. [xkcd.com]?
      • That's true for biology and is undoubtedly true for physics as well. The bar is high for papers, a lot of your results by themselves prove nothing but strongly indicate things, confirming or denying your ideas. You can "know" something years before you can say it's true, and although that can be misleading, a lot of times your colleagues will be very interested in it.

        Still, an online repository of papers is good for somewhat current stuff, full details, and getting information faster in many cases.

      • Ah, so you're working in the oral tradition, then?

      • perhaps that's simply an issue of convention. i don't see why linguistic papers couldn't be written and published in a similar fashion. is there no way to distill the private e-mail lists and informal conference discussions (transcripts) in a formal academic paper? seems like an open scientific/academic repository would benefit all disciplines, just as mailing lists and conference discussions do.

        i'm really happy to hear of Arxiv's success (i only heard about the site a month or two ago). this type of open e

      • by bjorniac (836863) on Wednesday October 08 2008, @11:17AM (#25301015)

        Well, this is the problem of perception a lot of people have - that scientists are the anti-social ones. Scientists cannot work in a vacuum - we need communications with one another, interactions and a knowledge of other work to get on with our own work. You build off other people's work, use the things recently discovered to move your own work forward, so you need to have constant fast communications of the latest discoveries. Good physicists are always talking to one another, asking about work done, clarifying points and collaborating - just check out how many of those papers have multiple authors, often at separate institutions.

        Compare this to a social science/humanity subject where sitting in your ivory tower is basically encouraged, with publications of great single-authored treatises seemingly the only output. They don't need to talk to one another and many are outright hostile to any discussion of their work.

        Disclosure: I'm a physicist with an SO in the humanities. The differences in our experiences are incredible - people in my department like each other and work together.

    • by vrmlguy (120854) <samwyse @ g m a il.com> on Wednesday October 08 2008, @09:43AM (#25299613) Homepage Journal

      My room was littered with papers printed off to read on the bus or at work.

      A good reason to buy an Amazon Kindle/Apple iPhone/Sony Reader.

  • by Anonymous Coward on Wednesday October 08 2008, @09:27AM (#25299389)

    i'll beat all the cynical punch savvy posters to the punch!

    that comma is in the wrong place, i see 50,0000. I guess they need another article on properly writing numbers.

  • is that a typo for 50,000 or 500,000?
    • Re:50,0000? (Score:5, Funny)

      by vrmlguy (120854) <samwyse @ g m a il.com> on Wednesday October 08 2008, @09:39AM (#25299567) Homepage Journal

      It's half-a-million. CmdrTaco doesn't deal with such large numbers very often.

    • Presumably 50,000 math papers. The remainder are a large but poorly numerically defined set of "new math" papers.

    • Re: (Score:3, Informative)

      Well, according to TFsite, "3 Oct 2008: arXiv passes half-million article milestone", so that would be 5 * 10^5.
    • Actually, in Chinese you sometimes see the comma separator between groups of four digits. In Chinese there is a word for 10,000 (pronounced mun in Cantonese) which is a more "natural" unit for many Chinese speakers than thousands, so this number (500,000) would be spoken as ng-sup mun (fifty ten-thousands), and so written in figures 50,0000.

      But the Western style of breaking into three digit groups is more common these days.

      According to Wikipedia [wikipedia.org] this is also seen in Japan, and India has a rather eccentric

  • by mbone (558574) on Wednesday October 08 2008, @09:41AM (#25299587)

    Here are some in fields I follow :

    In astrophysics, almost all new papers appear first in Arxiv.

    In planetary physics, some but by no means all papers appear in Arxiv.

    In geophysics, basically no papers appear in Arxiv.

    I don't know why there are these differences, but there it is.

  • by Anonymous Coward on Wednesday October 08 2008, @09:42AM (#25299589)

    If it's a science publication, should it have hit a kilometer-stone instead of a milestone?

  • by MosesJones (55544) on Wednesday October 08 2008, @09:43AM (#25299619) Homepage

    But the question we are all asking ourselves is

    Who got the first post?

    The answer is Exact Black String Solutions in Three Dimensions [arxiv.org] by James H. Horne and Gary T. Horowitz

    Slightly better than the "Fkrst Pist" attempts on Slashdot!

  • How significant (Score:3, Insightful)

    by igotmybfg (525391) <slashdot.danielthompson@net> on Wednesday October 08 2008, @09:44AM (#25299621) Homepage
    Because quantity == quality...
    • Considering they were started in 1991 and have now only gotten to 500,000, this is significant.

    • int quality = 0;
      int quantity = 0;

      if (quality == quantity)
      {cout "Yes it does. :)";}
    • Because quantity == quality...

      I realize that you were being snarky, but you accidentally hit on a corner of the truth. The real value of the ArXiV is indeed its quantity of results, mixed with the ease of access. The traditional journals typically restrict access to their output -- unless you are at a subscribing institution, it costs $15-$50 to access a single article from a single traditional scientific journal (depending on publisher). At professional institutes and universities, which typically have online subscriptions to journals, it is possible to surf through the Literature (depending on field, back about 10-15 years) and find recent relevant knowledge extremely quickly. If you aren't at an institution that subscribes, you're SOL. ArXiV fixes that - if you publish your article both in a journal and in the ArXiV, most indexing services will notice that it is the same, and suddenly everyone on the planet has unrestricted access. That's a no-brainer for an author.

      The way that professional scientists (like me -- I am a solar astrophysicist) access the Literature has changed drastically in the last ten years. My office has about 12 linear feet of Xeroxed journal articles in three-ring binders, but I practically never refer to them. It's far faster and more convenient to access (say) the entire archives of Astrophysical Journal online than to go "grep dead trees" at the library. Citation indices such as ADS (Google for adsabs) hyperlink both references and citations, so that I can search through 50 articles relevant to a topic in less time than it used to take to look up one article and Xerox it for reading outside the library.

      Old-style pay-to-read journals get in the way of that rapid access - for example, I have rarely cited articles in Astronomy and Astrophysics, because it's a pain in my ass to download them. Until recently, my institute didn't subscribe, so I had to either pay on a per-article basis (which adds up if you are skimming for the one relevant article in a dozen possibilities), or travel to the local university to get the paper I wanted. This is a very common problem: even large universities generally don't subscribe to all the relevant journals in a given field, because web subscriptions cost thousands to tens of thousands of dollars per year per journal!

      For everyone not fortunate enough to have a computer account at a large institute that can actually afford to subscribe to dozens of journals, ArXiV is the best way to access a large volume of the literature. Hence, articles posted to the ArXiV get cited more. That makes authors want to post to the ArXiV as a matter of course. It's a virtuous circle.

      So, er, yes, quantity is quality in this case -- ArXiV was canny and/or lucky enough to get a critical mass of good work, and the quantity is the driving force that keeps the whole thing going.

  • by ruin20 (1242396) on Wednesday October 08 2008, @09:44AM (#25299643)
    ...convince the scientific journal community that they should open their standards and let articles published in their journals to be republished by the author else where.

    I'm not going to pretend 50,000 is a lot, but the fact it's 50,000 and growing should make them worry. I hope the celebration of this milestone will help accelerate it's growth so we see 100,000 sooner than later. The quicker pay-for-access science disappears the better for all of us.

  • by Guysmiley777 (880063) on Wednesday October 08 2008, @10:16AM (#25300031)
    Wow, that's a lot of ten-thousands of papers!
  • In other news... (Score:3, Interesting)

    by bakuun (976228) on Wednesday October 08 2008, @10:43AM (#25300481)
    PubMed Central, the central repository for open access Life Sciences research articles, is pushing on 1.3 million articles. These repositories is a wet dream of text mining researchers.
  • by moosesocks (264553) on Wednesday October 08 2008, @11:42AM (#25301381) Homepage

    We really need to begin compiling our scientific knowledge into a hyperlinked wiki/database of sorts.

    Wikipedia's great for basic stuff, though there's still gobs of information (much of which is in the public domain) that's inexplicably confined to books and journals.

    Hyperlinks (and extended data sets) should be *standard* for all journal articles these days, given that we have the technology to do so. There's no reason that the arXiv needs to remain as a repository for dead-tree PDFs.

    • Re: (Score:3, Informative)

      At some level, hyperlinks (at least) are standard. They're called "references" and were the closest thing to a hyperlink before the intertubes were invented. Several free services (ADS is one: http://adsabs.harvard.edu/ [harvard.edu] have spiders that walk the literature and create genuine URL-style links between articles. ArXiV is advancing custom along that path, by making many journal articles available for linking to anyone free of charge.

      Extended data sets are coming. Astrophysical Journal allows online publicat

  • XXX.LANL.GOV (Score:3, Interesting)

    by Lawrence_Bird (67278) on Wednesday October 08 2008, @02:42PM (#25304531) Homepage

    was the original .. with the skull/crossbones icon. Now its all too easy and happy looking.

    • by Gromius (677157) on Wednesday October 08 2008, @09:43AM (#25299615)
      Likewise, every particle physicist also puts his paper there before they are published (my three are all there). While it is great as a source of open information, one thing to bear in mind is that it is not peer reviewed, *anybody* can stick *anything* there. This is the major reason why we still unfortunately need paper journals. We need somebody to read it and say yes this follows basic scientific procedures and to the best of his/her knowledge there are no mistakes. Because theres a fairly low signal to noise on arXiv and whats there is not guaranteed at all to be of proper scientific merit and correctness.
      • Re: (Score:2, Informative)

        by Anonymous Coward

        one thing to bear in mind is that it is not peer reviewed, *anybody* can stick *anything* there.

        This is true. However, they do have a group of moderators which recategorizes what they think are "merely mediocre, speculative, or erroneous articles". See http://front.math.ucdavis.edu/ifaq#nonsense

        Of course, this is not the same as peer-review, but at least it's something.

        • They could implement some sort of karma, moderation, and meta-moderation system, which would be even better than peer review, right?
        • ah didnt realise they had moderators, thanks for the correction. So there is atleast one layer of quaility control to keep the crazies out. When I submitted my papers I just clicked upload and didnt realise that somebody reviewed it. Glad nobody objected to mine :)
      • Re: (Score:2, Informative)

        Allyn Jacskon (editor of Notices of the AMS) has published an interesting article about the impact of preprint servers on mathematics: http://www.ams.org/notices/200201/fea-preprints.pdf [ams.org]
        In it he writes:

        As an experiment, Greg Kuperberg looked at the publication status of the first 100 papers in theoretical high energy physics posted to the arXiv in December 1998. He found that 81 had appeared in journals, 11 were conference proceedings or invited lectures, and 2 were Ph.D. theses. "Thus at least 94 of the 100 have been blessed by some form of peer review," he concludes.

        • by Gromius (677157) on Wednesday October 08 2008, @11:00AM (#25300751)
          But that was 1998 where a) the general population was just getting online and b) pretty much only scientists knew about arXiv. There is a lot of peer reviewed stuff on there (every paper submitted to a journal tends to be submitted) but as more less mainstream scientists have access, you regretably get more noise. Looking at Oct 2007 for hep-th and assuming that it would be mentioned in the summary is its published or going to be published (and trust me people mention this...), out of the first 25, 12 are published in a journal and or conference proceedings. So less than 50% were blessed by some form of peer review. And its the other 50% tend to be the most sensational :)

          Note I still think its very valuable for to have a place where non-peer reviewed material can be uploaded as well as peer reviewed but if its not peer-reviewed its a lot more likely to be incorrect somehow and the reader needs to be aware of that.
      • Re: (Score:2, Informative)

        it is not peer reviewed, *anybody* can stick *anything* there.

        I think they've changed things a little bit over time. It does seem like anyone is able to register an account, which would allow them to start submitting papers. But looking at the help pages, I see this on an endorsement system: "Effective January 17, 2004, arXiv.org began requiring some users to be endorsed by another user before submitting their first paper to a category or subject class." They note that this isn't peer review, but it "will verify that arXiv contributors belong [to] the scientific c

        • Re:Paper must die (Score:4, Insightful)

          by Gromius (677157) on Wednesday October 08 2008, @10:50AM (#25300617)
          Oh exactly. I think the future is a peer reviewed online journal. I think arXiv provides a very valuable service as is for the distribution of knowledge. Right now it has a copy of basically every particle physics paper published and I assume this is true for some other fields too. Many times I grab the arXiv copy over the journal copy as its more convient. So all the journal does is basically place a peer reviewed stamp of approval on the online arXiv paper and this could easily be replaced with a online journal in the future.

          I am strongly against journal sub fees as I believe which that the knowledge contained in scientific papers (doubly so for public funded ones) should be availible to all and not only accessable to people willing to pay the high cost of a journal subscription fee. CERN is pushing open journals for that very reason and that may evolve into a respected online peer reviewed journal which will compliment arXiv nicely.
    • "It seems that a lot of people follow their field by reading pre-prints posted to arXiv. Isn't this kind of dangerous, considering the lack of peer-review?"

      Peer review is great for some things, but just ask Galileo how 'peer review' worked for him. 7 years in a prison as a part of the inquisition. I do realize, that today scientific breakthroughs are treated a little differently today, unless you're talking about Genetic Engineering, which has it's own set of inquisition style prohibitions.

      but yeah even oth

      • Peer review is great for some things, but just ask Galileo how 'peer review' worked for him. 7 years in a prison as a part of the inquisition. I do realize, that today scientific breakthroughs are treate

        Just a note, Galileo's trial by the inquisition was not a problem of peer reviewing: it wasn't that he couldn't get his work published; it was what happened after it was published.

    • Re: (Score:3, Informative)

      Here's how it works (for me at least):

      First you write a paper - this is the hard part. Then you can submit it to Arxiv - usually done at the same time as submission to a journal, though some choose to wait for any initial backlash/corrections before doing this. Arxiv normally publishes it the next working day with no peer review (8pm EST the night before) for all to see online. Meenwhile your journal is still looking for peer reviewers. No journal in physics can now ask to be the sole source for any article