Want to read Slashdot from your mobile device? Point it at m.slashdot.org and keep reading!

 


Forgot your password?
Close
typodupeerror
×
The Internet Java Networking Programming

TinyDisk, A File System on Someone Else's Web App 188

Posted by ScuttleMonkey from the all-your-byte-are-belong-to-us dept.
Psy writes "I attended Phreaknic this weekend where Acidus released TinyDisk, a shared file system that runs on top of TinyURL or his own implementation NanoURL. TinyDisk compresses a file, encrypts it, and dices it into clusters. Each cluster is submitted to TinyURL as if it were a url. This clusters can be read back out of the database, making TinyDisk a global file system anyone can use. There are safeguards in the default config to prevent people from dumping gigs of MP3s into TinyURL. While file-system-on-web-applications are nothing new (GMail file system anyone?) this hack shows how easy it is to accidentally design a web application insecurely despite the default PHP protections. See his presentation for more info"
This discussion has been archived. No new comments can be posted.

TinyDisk, A File System on Someone Else's Web App More

TinyDisk, A File System on Someone Else's Web App

Comments Filter:

  • Nifty hack (Score:5, Interesting)

    by FidelCatsro ( 861135 ) * <fidelcatsro&gmail,com> on Tuesday October 25, 2005 @08:06AM (#13870615) Journal
    This does seem to be a bit of an abuse of the service. whilst it does not contravene the TinyURL terms of use in any real way
    Terms of use:
    TinyURL was created as a free service to make posting long URLs easier. Using it for spamming or illegal purposes is forbidden and any such use will result in the TinyURL being disabled and you will be reported to all ISPs involved and to the proper governmental agencies. This service is provided without warranty of any kind.
    , it just seems rather untoward .

    Nifty little program all the same and a nice hack ,
    Having it running on his NanoURL implementation locally , could allow for a cool little service . Though there are better ways to offer web based file systems in the real world .

    He does state in the FAQ that its not intended to pollute TinURL in any way .. but this is what some people will use it for (and I doubt the restrictions will be hard to remove).

    Perhaps it will give TinyURL a nudge to tighten up their security though .

    • It's simple. (Score:3, Insightful)

      by TheSpoom ( 715771 ) *
      If you want your online app to not be used by scripts such as this, implement a CAPTCHA [captcha.net]. Sure, people could still use it if they wanted to input a bunch of letters for every single chunk of their file...

      • Re:It's simple. (Score:5, Insightful)

        by Smidge204 ( 605297 ) on Tuesday October 25, 2005 @08:40AM (#13870753) Journal
        Or better yet, validate all entered URLs by attempting to establish a connection to the server. If the URL is invalid then kick it back.

        You wouldn't even need to do this with every URL added to the system. Spot-checking every 1 in 10 URLs or so will go a long way to preventing any sort of abuse.
        =Smidge=

        • Re:It's simple. (Score:3, Informative)

          by FidelCatsro ( 861135 ) *
          That is a great idea . It would also double as a feature , incase anyone accidentally mistypes a URL or misses a chunk at the end during a copy/paste

        • Re:It's simple. (Score:3, Interesting)

          by Chi-RAV ( 541181 )
          not really. as the author of the hack already proposed, you can add hosts of sites that have base64 encoded urls which means that you can get http://www.bleh.org/topic.php?= [bleh.org] and then the prog will filter out the part for use in the decrypting... surely if you use big enough sites (amazon, google, again as the author proposed) you can circumvent this. The point of this "hack" isnt really to show how to break tinyurl but to create a different way of approaching networked file systems, using only HTTP POST/G

        • You wouldn't even need to do this with every URL added to the system. Spot-checking every 1 in 10 URLs or so will go a long way to preventing any sort of abuse.

          Actually, that would only make things worse, as the filesystem code would simply have to resubmit 10% of queries, doing nothing but drive the load up further. Assuming, of course, that the trick suggested by the AC where all the data is stored in the query string is not used.

        • Won't validating each URL add to tinyURL's bandwidth costs? I think it would multiply their bandwidth usage many, many times over.
          • Well, if people start trying to use TinyURL as a file system, it may be more cost-effective to do the validation than blindly accept the long URL. TinyURL could decide how much bandwidth to devote to checking URLs based on the payoff from eliminating a bad one.

      • Re:It's simple. (Score:4, Informative)

        by nahdude812 ( 88157 ) on Tuesday October 25, 2005 @12:43PM (#13872714) Homepage
        Visual captchas present a significant usability problem for blind users, and audio captchas have proven much easier to defeat in general than visual captchas (some of which are actually quite easily defeated).

        Further, even the best visual captchas are easily overridden if the attacker is motivated enough; a common means to perform this action is to get other humans to voluntarily solve the captchas as they are encountered by offering, eg, free porn.

        Basically, captchas aren't really the solution to preventing bots (there are no good solutions for this), they only deter casual botters.

    • Re:Nifty hack, or antisocial behavior? (Score:5, Insightful)

      by Anonymous Coward on Tuesday October 25, 2005 @08:49AM (#13870804)
      It is a nifty hack, but let's not kid ourselves and pretend this is anything new, or that it's even a good thing.

      At its core, Tinyurl is just a write-once database. You add data and get back a key/pointer to said data. As with typical databases, the size of the pointer is logarithmic in the size of the input (* number of keys stored, not bytes; however, the number of bytes/key is bounded under some constant, so it's effectively the number of bytes).

      This gives us a logarithmic compression scheme, where our compression ratio (N-logN)/N approaches 100% as N gets large. This kind of "infinite compression" is what makes the method attractive: you put in say a kilobyte of data and get out a (currently) 5 byte key. All you have to do is keep an index of the keys.

      TinyDisk doesn't seem to do this, but you could then turn around and store the index as a key. Take 1000/5 = 200 keys and get back one key. Lather. Rinse. Repeat. In the end, you have a single key that points to the backup of your mp3 collection, all in one TinyUrl! Not too shabby. After all, it's free storage, right? Wrong. Someone ends up paying for the infinite compression. In this case, it's Tinyurl. If this kid had stopped to think for a few minutes before publishing his hack, he would have realized that he's actually doing a malicious, antisocial thing. I suspect there will be a dozen copycats in the wild before the end of the day.

      Farewell TinyUrl, we knew ye well.
      • Only corporations should act malicious and antisocial? That doesn't seem very fair. Why are you picking on the little sociopaths? At least it's possible to do something about them. Once you get a large corporation interested in a new form of plundering, it's all over.

        I don't really see this as abuse as much as the fundamental flaw in providing free services. If the cost to support the service is higher than the cost to the recipient, it's just a matter of time before someone finds a way to cut their co
      • In the end, you have a single key that points to the backup of your mp3 collection, all in one TinyUrl! Not too shabby.

        Sure, but I think it's a pretty dumb idea because of the large overhead (in time and data) of actually retrieving that data.. http request and response, encoding, etc. And the fact that tinyurl will (rightly) kick your ass off the service once he's on to you.

        • Perhaps you should have read the sentence immediately following the one you quoted:

          After all, it's free storage, right? Wrong.

          He wasn't suggesting it was a good idea to do it - he was giving a sample mindset of someone who would use TinyDisk to do stupid/malicious things.

  • Problems ahead? ;) (Score:5, Funny)

    by sznupi ( 719324 ) on Tuesday October 25, 2005 @08:08AM (#13870620) Homepage
    TinyURL might become not so tiny anymore...

  • From what I understand (Score:4, Informative)

    by FST ( 766202 ) on Tuesday October 25, 2005 @08:09AM (#13870626) Journal
    I saw this a few hours ago, and from what I understand the process goes as follows:

    1- Open a meta file
    2- Retrieve and concatenate all the clusters from TinyURL in the order
    specified in the meta file.
    2- Base64 decode the file
    3- Decrypt the file with the algorithm and key in the meta file
    4- Decompress the file with the algorithm in the meta file.
    5 - Verify the file size given in the meta file is correct for the
    decoded/decrypted/decompressed file
    6- Verify the checksum with the algorithm and value in the meta file matches
    for the decoded/decrypted/decompressed file
    7- Set the filename of the decoded/decrypted/decompressed file to the
    filename specified in the meta file.

    Hope that helps somebody :)

    • Solution for which problem? (Score:4, Interesting)

      by 't is DjiM ( 801555 ) on Tuesday October 25, 2005 @08:26AM (#13870692)
      That's pretty much what I understood.

      I adore the ingenuity (correct spelling?) of the hack but... I can't really find a problem this hack is a solution for.

      As a way to distribute files, it's probably too slow. The pro's I see here: the file is not stored as one single file but it's stored as a distributed file (a set of Base-64 encoded clusters), making removal of the file hard. On the other hand, if one single segment drops out, the file will be destroyed (except if some redundancy exists, of which I did not find evidence).

      If you want to send attachments in an e-mail, this is a very complicated way to do it. Every receiver must have the decoder program to re-assemble the file.

      Moreover, if tinyURL builds in a check to see whether the submitted URL exists (not just some 404 page), the whole concept would probably break.

      Anyways, very clever hack!
      • Simple, guerrilla distribution of files that would be illegal but not immoral - DeCSS, anyone?

        Also, it could be used for distributing small text files containing reports from warzones and other heavy-censored countries. EFF should have a blast on this one.
        • Simple, guerrilla distribution of files that would be illegal but not immoral - DeCSS, anyone?

          Yes yes, and thermonuclear weapons were only invented to deter the Soviets.

          Sorry, but moral questions DO come into this.

  • Insecure? Really? (Score:5, Insightful)

    by Afty0r ( 263037 ) on Tuesday October 25, 2005 @08:10AM (#13870632) Homepage
    this hack shows how easy it is to accidentally design a web application insecurely despite the default PHP protections.
    The design of these TinyURL style applications is insecure in the same way as a concrete wall is insecure because someone could spray paint on it.

    Insecure? Rancid tabloid hyperbole more like.

  • NanoURL review (Score:5, Funny)

    by Anonymous Coward on Tuesday October 25, 2005 @08:11AM (#13870634)
    I like NanoURL but it scratches real easily.

  • I'm sure this won't be abused (Score:3, Interesting)

    by Dekortage ( 697532 ) on Tuesday October 25, 2005 @08:11AM (#13870635) Homepage

    Pretty soon you'll see someone trying to use this as their backup system for 30gb of pr0n. Will large files kill TinyURL? What kind of latency is this going to introduce? If nothing else, this might constitute a DoS attack on TinyURL.com (which would be illegal.

    It's still interesting work.

    • It's still interesting work.

      Even more interesting would be something which encrypts your files and spreads them around in various free storage media (slashdot trolls?) in such a way that they can not be easily correlated with each other.

      Cramming all this stuff into tinyurl is bound to be noticed, but if it is a couple of dozen bytes here and there it might be possible to store lots of stuff with a reasonable degree of safety.

  • Default PHP protections? (Score:4, Funny)

    by Egregius ( 842820 ) on Tuesday October 25, 2005 @08:12AM (#13870638)
    Bwahahahahaha.

  • Is it (Score:2, Funny)

    by manojar ( 875389 )
    is it another way to backup more porn/mp3s online?

  • TinyDisk's inspiration (Score:3, Interesting)

    by Anonymous Coward on Tuesday October 25, 2005 @08:21AM (#13870672)
    Gmail Filesystem [jones.name]

  • You could do this with blogs or any CMS (Score:5, Insightful)

    by tezza ( 539307 ) on Tuesday October 25, 2005 @08:27AM (#13870696)
    You could put this in an unapproved pending queue for Wikipedia, comments on Joel Spolsky's blog or wherever.

    But overall 'WHY?' must be the question? Al Quaeda or The Real IRA? They still have their old working communication channels. Also who needs space like this? Space of this amount could be made redundant and available by using GoogleMail, Yahoo and Hotmail in synchrony. If none of those are available, presumably you'd have it on USB key as well.

  • Furthur Compression (Score:5, Interesting)

    by hoshino ( 790390 ) on Tuesday October 25, 2005 @08:30AM (#13870706) Homepage
    I noticed that the whole of Alice in Wonderland is compressed to just 20 clusters and each cluster is represented by the five-letter keys used by TinyURL. So is it not possible, using the same method, to reduce the entire metafile (which is merely a textfile of less than 1kB) into a single-line URL? Then you can have the program retrieve the metafile from the URL and the actual file from the metafile. So instead of sending people a metafile, you can just copy and paste them one line of URL.
  • Encryption, clusters, nano.. Why don't they encode the tiny clusters to DivX while they're at it and embed it in the silences in movies... :)
    This sounds like a very cool conference, are they going to distribute a conference program in pdf format, or is Phreaknic too underground for that, and require you to get it off torrent ??

  • What does PHP have to do with it? (Score:5, Informative)

    by miknight ( 642270 ) on Tuesday October 25, 2005 @08:41AM (#13870758) Journal
    Sure, well all know PHP is far from synonymous with security, but this seems to be a case of exploiting a web app using only the mechanics intentionally made available, just in a novel way. Seems like some unfounded (though not necessarily undeserved) PHP bashing.

    • The underlying message is that web application development is inherently difficult to secure, despite PHP's valiant attempts to protect programmers from themselves. This is the opposite of PHP bashing. It's PHP apologetics.

      I disagree with the article's premise. It seems to me the same sort of mindset that attributes to "pilot error" aviation incidents that would better be attributed to poorly designed instrumentation.

      • But TinyURL is equivalent to the instrumentation designer, while PHP is equivalent to the components from which the instrumentation is made. So the article is right.

        Security is layered. PHP components can be secure, but in providing a general purpose language, anything can be built from those components, including insecure web applications.

        To create a programming language or environment from which you cannot build an insecure application would require seriously compromising the flexibility which gives the l
        • To correct myself: I'm talking about the submitter's commentary, not the cited article. TinyURL does not enter into the aviation accident analogy, because TinyURL has not been demonstrated insecure here. By design TinyURL allows anonymous entry of information that becomes publicly readable. They are correct in waiting until there's an actual problem before building anything to "defend" against this filesystem hack.

          In my analogy the programmer is the pilot and the programming language is the instrumenta

  • The end of TinyURL. (Score:2, Insightful)

    by tmroyster ( 309750 )
    The end of TinyURL is in sight. Yes, this is (probably) a clever hack.
    But this is a misuse of a really useful service.

    When TinyURL's administrator has to either go out and buy his
    second 2Terabyte disk array in a week or shutdown, which do
    you think he will pick?

    • He'll just restrict traffic by IP. Any legit user will not be making more than one request per second.

  • Video/Overview of Acidus's presentation (Score:4, Informative)

    by Anonymous Coward on Tuesday October 25, 2005 @09:06AM (#13870917)

    Here [wilpig.org] is a video of Acidus's presentation. If you haven't seen him present before (At Hope, O'Reilly's E-Tech, Toorcon, Phreaknic, Interz0ne, etc, etc) he puts on a good show.

    The presentation was called: Layer 7 Fun: Extending web applications in interesting ways. He discusses how traditional web applications work -vs- "new" web ppas that use AJAX. He talks about writing extensions to web apps using an API supplied (ala Housingmaps.com, or chicagocrime.org). Finally he talks about writing an extension to a web app where you don't have access to an API. TinyDisk was a case study for writes these so-called "non-sanctioned" extensions. He has a funny little slide he goes back to about how to properly implement a web app (which TinyRUL fails to do). Things like "don't wallow users to uploaded arbitrary amounts of data directly into your database."

    Funny Stuff. His upcoming talk at Shmoocon [shmoocon.org] seems pretty cool too.

  • Book names - Recommended Reading (Score:4, Informative)

    by eltoyoboyo ( 750015 ) on Tuesday October 25, 2005 @09:10AM (#13870936) Journal
    In the Recommended reading section this is stated:

    There are definitive works in certain fields that online guides and HOWTOs cannot even approach in terms of detail or quality. It's a class of books that are so familiar people refer to them by nicknames instead of by full title.

    Well maybe so, but I did not know them all, and in the interest of helping people along the path here they are:

    Books like:
    K&R,     The C Programming Language by Brian W. Kernighan and Dennis M. Ritchie
    The Dinosaur Book, Operating System Concepts by Abraham Silberschatz
    Knuth's never-ending story, The Art of Computer Programming, but Donald Knuth
    The White Book, Introduction To Algorithms by Thomas H. Cormen, Charles E. Leiserson, Ronald L. Rivest, and Cliff Stein
    P&H, Computer Organization and Design The Hardware/Software Interface David Patterson John Hennessy
    The Illustrated's. TCP/IP Illustrated Series (The Illustrated's) - W. Richard Stevens
    The Rainbow series. U.S. DOD Computer Security Series
  • Sorry to be Mr. Obvious this morning, but I take issue with submitter's conclusion that TinyDisk illustrates a security issue on the part of tinyurl.com. It rather illustrates the ease of creating a leachable web app that resource pirates can abuse. Yes, I have a negative opinion of those using such a creative hack against others who provide services to the general public in good faith.

  • Why not go further? (Score:4, Interesting)

    by SamSim ( 630795 ) on Tuesday October 25, 2005 @09:16AM (#13870974) Homepage Journal

    Take the list of cluster URLs [msblabs.org]. Concatenate them into a single URL. Submit it again. Thus compressing literally ANY file to five characters.

    At least, as long as the possibility space of five-character URLs isn't exhausted. It's very much first come, first served.

    • Let us not forget, we are NOT COMPRESSING ANYTHING.

      We are simply addressing it. By your definition, a filesystem path (e.g. /usr/home/SamSim/foo.txt) is compressing the contents of foo.txt to the length of the path.
    • You are aware that RFC valid URL's have a maximum length..right?

      I can't be arsed to look it up but off the top of my head I think it's =256 characters.
      • I didn't think there was a maximum specified length, and I've certainly seen longer URLs that 256 characters in working use. MSIE, for example, can handle URLs up to 2048 characters long. But it doesn't matter how big the file is or how long the maximum URL is. Suppose your file is four gigs. Split that up into 2097152 chunks of 2048 characters each, and reduce them all to five characters each, and you have a 10-meg file. Split that into 5120 chunks of 2048 characters and reduce them all again, and you have
        • From RFC2616 (HTTP/1.0):

          The HTTP protocol does not place any a priori limit on the length of
          a URI. Servers MUST be able to handle the URI of any resource they
          serve, and SHOULD be able to handle URIs of unbounded length if they
          provide GET-based forms that could generate such URIs. A server
          SHOULD return 414 (Request-URI Too Long) status if a URI is longer
          than the server can handl

  • Great Idea! (Score:4, Insightful)

    by Se7enLC ( 714730 ) on Tuesday October 25, 2005 @09:22AM (#13871014) Homepage Journal
    I was looking for another way to store files online in such a way as to make them:
    • Difficult to access
    • Unreliable
    • Split into many different pieces
    • Under somebody else's control that has motivation to delete them

    I guess once this goes down, I'll have to go back to posting UUencoded files in peoples blogs.

  • Juggling With Packets (Score:4, Insightful)

    by pizza_milkshake ( 580452 ) on Tuesday October 25, 2005 @09:23AM (#13871029)
    looks like an implementation of Michael Zalewski's Juggling With Packets [coredump.cx] concept, the storing of data in buffers of publicly available services for use as a filesystem.
  • Spread it across a bunch of servers, and as soon as you have enough parts rebuild the file. Maybe use a meta search engine to find the parts.

  • What does PHP have to do with it (Score:3, Interesting)

    by PktLoss ( 647983 ) on Tuesday October 25, 2005 @09:54AM (#13871302) Homepage Journal
    Yes, I RTFA, and looked at how things work, the fact that PHP is being used is immaterial.

    The basic functionality of TinyURL, NanoURL or any other service is to accept a string (presumably a URI) and return a shorter string that will serve as a pointer to it. If you want your application to accomplish that it doesn't matter what it was written in, people can store things other than URLs in your database. The protections against this sort of use/abuse suggested in the article are also language independent.

  • Greatest FAQ answer ever. (Score:5, Funny)

    by jdludlow ( 316515 ) on Tuesday October 25, 2005 @10:02AM (#13871373)

    From the TinyDisk FAQ:

    Q: This damn thing doesn't work on large files! #@%& You!
    A: Did you not read the manual? Man I wish I could punch you in the face over TCP/IP! Change the config file's MaxSize line. By default the limit is 2 megs.

  • google fuck (Score:3, Interesting)

    by cjsteele ( 27556 ) * <coreyjsteele AT yahoo DOT com> on Tuesday October 25, 2005 @10:44AM (#13871740) Homepage
    I had a similar idea a number of years back basically masquerading uuencoded files inside of bogus html files that get crawled by Google's caching bot... the idea being that if you knew the names of the files you could query the cache and retrieve the UUEncoded bits.

    At the time, no one else had written about such things. I just never got around to automating the process, so it never really materialized. Maybe some brave and time-rich soul would like to give it a go?
  • If you use OS X, it takes a little bit of work. I download Java 1.5 and installed it. Then you have to set your path variable in terminal to get it use the correct Java (the website didn't mention this, and the new Java Configuration Panel thing didn't make it work for the terminal). Anyway, after doing all that, I was able to make this file:

    # TinyDiskFile - used to retrieve a file that has been stored in TinyUrl.com!
    #
    # By Acidus - Most Significant Bit Labs - Acidus@msblabs.org
    Version: 1.0
    Filename: hello.j

Slashdot Top Deals

Solutions are obvious if one only has the optical power to observe them over the horizon. -- K.A. Arsdall

Close