Please create an account to participate in the Slashdot moderation system

 


Forgot your password?
Close
typodupeerror
×
Technology

Swarmcast GPLed 103

Posted by CmdrTaco from the worthwhile-concepts dept.
miguel writes "OpenCola has just released SwarmCast which is a very interesting mechanism for distributing software. At Ximian we are looking into integrating this into Red Carpet to accelerate software downloads by using their sharing software. The demo of their product is pretty amazing." Very clever: essentially it creates a peered network so larger files can be shuttled around faster. Each client can serve a small piece of data to other clients so that a massive centralized data center isn't necessary. Now the cola on the other hand...
This discussion has been archived. No new comments can be posted.

Swarmcast GPLed More

Swarmcast GPLed

Comments Filter:
  • I've had no success getting their software to work in Opera v5.11 for Windows. Anybody else gotten it to work?
  • The "legitimate use of p2p" thing was not in reference to Napster, but this [slashdot.org] slashdot story, in which the EFF is attempting to find examples to defend p2p as a technology from those who believe that allowing individual clients to share information directly encourages nothing but piracy. It is thus more likely to be used in defense of Freenet and Gnutella, which (unlike Napster) were created without any overt overtones of piracy (well, I'm not certain wrt Gnutella), to demonstrate that the creation of a p2p system does not necessarily demonstrate intend to promote violation of copyright.
  • First, I must say that red-carpet is a very impressive piece of software installation for newbies. BUT...

    I am terribly afraid of this trend of making EVERYTHING too easy to do for the people who do not understand HOW or WHAT they are doing in fact.

    So now, as if it wasn't enough that red-carpet requires root access for it to work BEFORE really needing root previledges, you will have software distributed in pieces in that shareing mechanism?

    Make usage easy on people, but please, do not exagerate on giving power to those who cannot handle it. Next thing we'll know, viruses for linux will start spreading because of all the guys who have no idea they should avoid using root as much as possible.

    I love gnome as a desktop, but primates, please stop and consider... do not open precedents please.

    First of all, cut root usage to the bare minimum in red-carpet (it currently isn't).

    Apart from that, thanks for the neatly tied gnome packages.

    Hugs, Cyke
  • I beleive people are thinking this is some kind of "one way napster". I don't belive that's what they guys at OpenCola mean it to be.

    I think the hackers at OpenCola want to create a product FOR large media streamers. The fact of the matter is that when something really big comes out for download, like a trailer to a new movie, a large portion of the internet endusers will want to download the same file(s). Instead of each joe or jane sitting at his/her computer, waiting for the content to download at 4kbps, the hosting company can place up 3 or 4 decent servers (peferably in diffrent areas of the nation) to be the first 3 or 4 nodes of this download.

    The best part about this settup is that the servers will recive their prapotion of the download, and that's it. another really cool feature that could be implemented (And it possibly could be, I didn't read the code or faq) like automatic selection of servers with good ping/transfer-rates/hops, while dropping poor connections to make sure I'm not reciving packets from accross the contenent.

    Questions I have: Can a packet be restarted if it's droped? Can a faster server take over a slower server's packet? Is it possible to have streaming media? Can there be a diffrent client software, that's not necessarialy Java? (not saying I don't like the java client, but not everybody will.)

    All in all, I like the consept. Large companies would spend big bucks to have their heavaly downloaded files swarmcasted, even the content providers and hosting companies will like this... if it works, and it catches on.

    Good job FreeSoda-- er, OpenCola. Now, Don't give up! Take ideas and suggestions from your peers (us) and keep polishing the code untill it shines!

    Pathway
  • I will repost my note to the developer maling list as you asked, but as the anonymous coward did state: This is an open forum. This is as good a place as any to answer these questions.

    None the less, I'm off to send off my post.

    Pathway
  • Who's Justin? My name's Matt...

    Ooops. Sm:)e.
  • New questions:

    Why not have this as a plug-in? Keep the Java client (more compatibilty, the better... plus competing products), but have a Plug-in that integrates with the browser? This will increase the popularity of the software, as people are used to using plugins, but not used to installing seprate java clients.

    What kind of requirements does the server need? Is it a simple http or ftp server? Or is there a diffrent format for the download?

    Can a modem/dsl/cable handle several streams at once? How many, in your experiance?

    Is there a log of where I got each segment somewhere? Is this log on my local machine, or is it logged on the web, or on the provider?

    If I was a large company wanting to host a large file, how much would/could it cost? (Obviously, it could be free, but I want to know how much somebody would charge to provide such a service...)

    Are there plans at OpenCola to make a Napster style distributed network using this technology? Does OpenCola know of any other software that plans to?

    Okay, that enough new questions for now...

    Pathway

  • It isn't true that Mojo Nation [mojonation.net] is "not focussed on performance". I'm one of the Mojo Nation hackers, and we care about performance. It is true, however, that Mojo Nation is pretty complex, providing both data transfer (using a "swarm" like technique), and data storage, and a queriable search engine. The end result is something like a distributed, non-deletable World Wide Web. (Sort of like Freenet plus persistent data, or the earlier concept of Ross Anderson's "Eternity Service".)

    Performance isn't that great on Mojo Nation right now, but it is good enough, in my experience, for daily use.

    I'm pretty excited about the Swarmcast open source release, both because I think Swarmcast is a cool app in itself, and because I can now start taking ideas and code from Swarmcast to put into Mojo Nation, and vice versa.

    In the long run, both Mojo Nation and Swarmcast will improve because of this sharing, as will other related open source projects like Freenet [sourceforge.net] and Free Haven [freehaven.net].

    Regards,

    Zooko

    P.S. I've been talking with Justin Chapweske, the Swarmcast, lead, on irc.openprojects.net, and he's already pointed out a potential bug that we need to avoid in future versions of the Evil Geniuses Transport Protocol...

  • Hm -- it is my impression that data is more persistent in MN due to the IDA splitting and heterogenous block-buying policies (i.e. some block servers rarely buy new blocks at all), plus MN's random-block-dropping policy.

    In any case it is a goal of Mojo Nation that the data be persistent, whereas if I understand correctly, Freenet doesn't make persistence a goal. I hope I haven't incorrectly labelled Freenet's goals here.

    Technically, data falls out of Freenet and Mojo Nation in the same way -- due to no servers wanting a copy -- but Mojo Nation contains an essential element which hopefully makes it possible maintain persistent blocks even when you are the only one who cares enough to maintain them: exchangeable network karma in the form of Mojo Tokens.

    As the cypherpunks who investigated Ross Anderson's Eternity Service in the 1990's concluded, you can't have persistent data unless you have some way to motivate other people to hold on to copies that they personally are not interested in. Hopefully Mojo Tokens provide that motivation -- by holding on to blocks that they personally have no interest in (and very likely that they can't decrypt anyway), they earn network karma which they can use to store and retrieve data which they are interested in.

    If I understand correctly, based on some comments by Ian Clarke, Freenet is happy with blocks that only a few people care about being dropped in favor of blocks that more people care about. That is a good design in many ways, but I hope that exchangeable Mojo Tokens will make it so that blocks are retained as long as anyone on the network wants those blocks to be retained, instead of as long as the servers who currently hold them want them to be retained.

    Slashdot is a terrible medium for a real conversation, so I'll post this to the p2p hackers list...

    Regards,

    Zooko

  • You just have to trust the main server's md5 signatures, and never end up trusting the peer servers.

    There is a known weakness in MD5 which, while it would not allow a malicious peer to replace arbitrary content, would allow them to send you corrupt content that you could not detect by checking the MD5 signature. This could perhaps serve as a denial of service. The other response to your article states that they are actually using SHA-1 signatures, which I think have no such known weaknesses.

  • s/replace/substitute/
    s/you could not detect by checking the/would have the same/
    s/article/comment/
  • Thing is, I don't see why anyone would use this. In reality, the transfer rates aren't as good as a single fast dedicated server

    True. If you have an adequate dedicated server with plenty of bandwidth, it will be hard to top with swarmcast.

    The benifit comes in terms of cost. Those beefy servers cost a shitload of money in terms of colocation and bandwidth.

    A small, unfunded content shop may want to stream animations but cannot afford the cost of a central server to do it.

    If they use swarmcast, however, the streaming is accomplished by utilizing bandwidth of everyone participating. No expensive server is needed, and performance is at least decent.

    Real world uses for this technology are still lacking, so we shall see how swarmcast gets adopted IRL.

  • Here's what it took for me to try the Swarmcast beta:

    1. locate, download, and install a Java2 runtime
    2. download and install Java Web Start
    3. click through a prompt that warned "this software wants to do anything on your system"
    4. it also didn't work with Mozilla (probably a Mozilla bug), but others report that it's mostly Netscape 4.x and IE specific

    This is alot to discourage someone from even trying it out.

    Swarmcast is a great idea and I was impressed by how well it worked once I set it up. However, it needs either a simplified intallation process or to come bundled with your OS/environment. Otherwise, the swarm will be limited to those geeks who bother to go through the installation.

    That Ximian is considering incorporating it into Red Carpet is a nice step, but that's a rather narrow application. It does nothing for my browsing downloads. A nicer step would be incorporating Swarmcast into GNOME Transfer Manager [sourceforge.net] or something similar. Ditto that for Windows and Mac.

    I really do love Java, but from a pragmatic point of view almost noone has Java2 and Java Web Start installed. I think at most people will tolerate a single downloaded installer that "does the right thing" and works from thence forth.

  • I dont think that content distribution at high speeds is valid in regards to isolating P2P uses.

    What is a better choice for the EFF to use is distributed load balancing for large files for file transfer. It has nothing to do with high speed distribution, that can be accomplished by getting a T3 or OC line. However, distributed large file transferring is similar in nature to a distributed CPU utilization system from my perspective.

    I think the EFF is looking for uses where P2P is more or less the only solution (or only feasible solution) for a particular problem. I see this definitely fitting that niche. Load balanced, and redundant if I read the site correctly this is definitely valid. Lets hope the EFF picks it up too.

  • The Weakest Link [twlonline.co.uk], an absolutely terrible TV show.

  • Would it not be possible to checksum the entire file? That should give you confidence when a file is not corrupted.

    I'm not sure how you would deal with corrupted files, however. Any thoughts?

    --Ben

  • Wow, I'm massively impressed by the cluefullness of your response, perhaps you could repost it to our developer mailing list [sourceforge.net] and we can answer some of your questions in more detail.
  • You are basically correct, we are using SHA-1. I should put up a security FAQ.

    -Justin Chapweske, Lead Swarmcast Developer
  • Cool, I'll make sure the FAQ gets cleared up.

    Thanks!

    -Justin Chapweske, Lead Swarmcast Developer

  • Re:Ok, but... (Score:3)

    by Orasis ( 23315 ) on Wednesday May 23, 2001 @02:30PM (#202375)
    Swarmcast is neither a fragile chain structure, nor a hierarchy, it is a many-to-many 'swarm' structure where peers send and recieve data from many peers in parallel. The use of Forward Error Correction allows us to have a potentially huge number of unique packets in the mesh where only a small subset of those packets are needed to recreate the original content. This allows the peers to swap data back and forth in a fairly random fashion to provide a high level of resiliance against changing network conditions, very high throughput, and rapid scalability.

    -Justin Chapweske, Lead Swarmcast Developer

  • Re:What's with this gateway thing? (Score:5)

    by Orasis ( 23315 ) on Wednesday May 23, 2001 @02:19PM (#202376)
    Everything you need to both serve and download content is released under the GPL. Besides, its peer-to-peer so there really isn't that much of a "server" concept. The gateway is mostly for content management and permissions, the kind of stuff that companies pay money for so that I can keep my job and write more open source code. -Justin Chapweske, Lead Swarmcast Developer
  • No.

    With mojonation, I've never seen the performance that a solution like this should provide. Mojonation's not focused on performance, and Swarmcast is.

    Mojonation's a lot more complex, too. With Mojonation, there's a searchable, virtual repository of files. Every host on the network is a peer.

    With Swarmcast, only hosts downloading a given file are peers, and files are simply linked at web sites. (Personally, I think the proxy server approach is niftier, though.)
  • There's more tech info here: http://opencola.org/projects/swarmcast/swarm_tech1 .shtml [opencola.org]

  • Zooko, I didn't mean that you don't care about performance. I know that you folks are trying to make things faster. I read it in the changelogs. But Swarmcast is focused (not "focussed", btw) on performance. It's the only purpose of Swarmcast.

    They've aimed at something a lot smaller than what you folks are doing, but they've succeeded. You guys have more ambition-- it's not done yet, but when it is, it will be glorious.

  • Gott in Himmel! (Score:3)

    by Webmonger ( 24302 ) on Wednesday May 23, 2001 @02:29PM (#202380) Homepage
    Man, you don't usually see download rates like that from anything except akamai. Problem was, the data got corrupted.

    And no, it's not like a bucket brigade. It's more like building a house with more than one bricklayer.

    Re: slow hosts-- if I understand correctly, all swarmcast hosts maximize their available bandwidth, automatically balancing the download.

    Each node downloads packets one at a time. If a host is slow, it won't grab as many packets, and someone else will.

    If machines A and B are downloading, and A is 5x as fast as B, then in the same time, A will download 5 packets and B will download 1. If there are 12 packets in the file, A will download 10, and B will download 2.
  • Thing is, I don't see why anyone would use this. In reality, the transfer rates aren't as good as a single fast dedicated server (I can easily get 75-80KB/sec on this line), so there's really no gain on the client end from using this.

    You have an account on /. and don't see the benefit of distributing downloading of large files? When was the last time a big file got posted here and the server didn't bog down, no matter how big pipe they had? Only with a big mirror network would you have similar redundancy.

    For the record, my download began at 100k or so and then worked it's way up to a bit over 200k. I live in Sweden and it's not uncommon that downloads from across the "pond" max out at 50k. Only when downloading from servers in Sweden or nearby do I get over 500k. So I could definitely use this. ;-)

    Personally I thought the install procedure went very nicely. Simple click-click-click things that even "Joe Sixpack" could handle. I would agree that downloading an extra 6 Mb might seem odd to the casual surfer though.

    And although I'm currently letting it run along serving people (14 currently it seems like) and I don't have any problem with this it would be nice to have more data in the form of in/out trafic and such. And naturally, although the FAQ claim that "the participants IP's are not shown" it shouldn't be much problem "correcting" that for someone that is inquisitive as the source is available. Or by using a sniffer or just a personal firewall.

    And sure, it still won't be used all that much until newer versions are available. (The GUI was a tad buggy on my machine.) But the concept is great and obviously it works. The next goal would be to get it into Protozilla or so, neh? ;-)

  • I don't know if anyone else had this problem; but I'll shoot it out anyways.

    After downloading the demo file I let the program run along in the taskbar. However as I tried to reload and post stuff to /. I continously got "Connection refused" answers. I didn't try any other servers before shutting down the program and retrying though. (After that post and such went fine again.)

    Anyone else experienced this? I'm using Mozilla .9 BTW.
  • Agreed, IF (and only if) you actually pay for the data transfered to/from you. Personally I'm hooked up with a permanent connection right now, and since I pay a flat fee for this I have no problem sharing files with other people. At least as long as it doesn't hog capacity which I want to do something else with.

    There are some places where you generally have to pay for the data you send and receive though, and I agree that Swarmcast is then not all that nice. But at least I think that this is a temporary "solution" for teleco's and ISP's. When we all have broadband [sic] connections then utilities like Swarmcast will be the only realistic way of distributing data across the network.

    Now that banner ads have been demostrated to not work perhaps this (or a future Freenet version) will replace it. What other method is there to stop "slashdotting" from being the norm rather than exception?
  • A secure hash function is the solution for validity. Get a trusted source to give you something like a mojonation sharemap [mojohackers.org] or a freenet content hash key [netunify.com] which point you to the file you want by telling you its hash. Use whatever mechanism is provided by the system to query for a particular file, which the provider identifies by its hash. Download it, and re-compute the hash. If they don't match, throw it away (and mark the source as less trustworthy). If they do match, it's either exactly what the trusted source specified, or someone managed to get a hash collision (fairly unlikely with a secure hash).

    A series of high-speed mirrors requires a lot of (expensive) fat pipes. Getting away from that requirement is the whole point behind swarm distribution.
  • You may find MojoNation [mojohackers.org] a bit more to your liking; MN brokers transact with one another in a crypto e-cash currency called "Mojo". Using others' bandwidth costs Mojo, providing bandwidth to others earns you Mojo.

  • Re:Possible Security Issues (Score:4)

    by Pemdas ( 33265 ) on Wednesday May 23, 2001 @02:03PM (#202386) Journal
    I don't think any of these are real issues:

    First of all, what type of security is going to be implemented to prevent hacks. It seems that it would be pretty easy to shore up a single server, or even several in a single datacenter, but it would be a daunting task to protect thousands of machines spread thoughout the world against hackers.

    So long as the swarmcast client software doesn't have any holes, how is this an issue? If it does have holes, yes, this is a big deal, but if you've got a machine on the net with other security holes, that's an entirely different problem.

    Secondly, what type of redundancy is going to be built into this system. Again, if a file is going to be served from a single centralised machine, it should ideally be fairly reliable, with multiple connections, and RAID to ensure continuous uptime. However, if you're serving tiny pieces of a file from thousands of boxen, it seems to me that if even one of those fragments doesn't make it all the way downstream, the whole file would be worthless. Has Swarmcast done anything to prevent this from happening.

    This is a pretty trivial problem to solve. There's still got to be a central server somewhere that tells you where to look for the peer bits. I don't know the details of their setup, but it certainly seems trivial to implement MD5-checking per chunk, so you can tell if there are bad chunks in your download. You just have to trust the main server's md5 signatures, and never end up trusting the peer servers.

    This applies to the first point, too. If a serving machine is hacked, it's not a problem beyond that box because the real security comes from the MD5 key, or equivalent. It may be possible for a hacked server to serve bad code, but, given reasonable client design, it should be impossible for this bad data to actually be used by another machine. (Or, at least, should be impossible without big flashing lights and sirens screaming "You fool, don't do that!" :)

  • Wait for a situation like the day the next Lord of the Rings trailer is released.

    Say, 10,000 active nodes at once? (And I'm being conservative.)

    It should hum nicely, then.
    --
    Charles E. Hill
  • Have you ever heard the old saying "too many cooks spoil the stew?"

    ')
  • Anne: You are the weakest link, g'bye.

    Ok, what is this, some kind of new "all your base" meme-storm?

  • Re:Possible Security Issues (Score:4)

    by interiot ( 50685 ) on Wednesday May 23, 2001 @02:09PM (#202390) Homepage
    Erm, I'll bite.

    Secondly, what type of redundancy is going to be built into this system. ...if you're serving tiny pieces of a file from thousands of boxen, it seems to me that if even one of those fragments doesn't make it all the way downstream, the whole file would be worthless. Has Swarmcast done anything to prevent this from happening.

    One of the main benefits of distributed anything is that there's an incredible amount of redundancy because every node is willing to contribute a little towards achieving your goal.

    Are you trolling?

    First of all, what type of security is going to be implemented to prevent hacks.

    Against what? To prevent against file modification, checksums can be distributed from a single point since they're much smaller. All other security problems will be the same as any other P2P system.
    --

  • You're presuming a rigid split between Providers and Consumers.

    Imagine a world where we're all a little of both. By sharing our network, instead of turning chunks of bandwidth into money, we can make our network more robust and more efficient (if not always faster...). We can learn that everyone's a Provider, and everyone's a Consumer.

    La-la-la.
  • True peer to peer is a terrible way to distribute anything important. It becomes harder to prove the validity of anything. I'm sure md5 could help, but a series of high speed mirrors would work better.

    The courts would laugh at this. It is distributing content that is legal to be distributed. That is the issue with napster, not peer to peer file sharing. In fact I used to get all my mp3 in a client/server setup.
  • Yup, we're submitting Swarmcast as an example of "good" P2P to the EFF. We're pretty tight with them; Cindy Cohn, the EFF's Legal Director & Senior Staff Attorney, is on our Technical Advisory Board.

  • Maybe its time these developers list what exactly their software is being used for, and who is using it, to promote it, as opposed to waiting for groups like RIAA, MPAA to cry foul over them. Doing so would provide a nice argument, such as the ones EFF was looking for earlier.

    As for the brief mention on security I browsed through, personally I don't see it as a big deal provided you know how to set perms, and or can configured some form of SSL behind it or something similar, perhaps make some rules on your firewall or IDS to ensure nothing gets broken along the way.

    I can't wait to see how groups will react to cDc's Peekabooty, thats sure to be a kick in the ass for groups like RIAA, MPAA.
  • Why not distribute the pieces on Freenet? Using SSKs you can be sure that each piece was inserted by the original poster. Maybe this would make a cool client for Freenet, though...
  • My big beef with Swarmcast is that you don't have the option of downloading a JRE-less version. I already have JRE 1.3, and therefore don't need that extra meg of stuff to download.
  • The UI/user experience needs a lot of help. On windows, it should load as an activex-type control- the more seamless the better. And if things could be handled by the browser, as if there were no Swarmcast component - I think this has the potential to kick butt...
    --
    OliverWillis.Com [oliverwillis.com]
  • Everything you need to both serve and download content is released under the GPL.
    Thanks for clearing that up. You might want to have your webmaster reword the File Registration FAQ [swarmcast.com] answer number 4, which basically says you must register a file (accepting the terms of the gateway agreement) to be able to "Swarmcast-enable" it. I had interpreted that as "you must use our gateway and agree to our conditions to use Swarmcast". Sorry for any confusion.
    --

  • What's with this gateway thing? (Score:3)

    by Eric Seppanen ( 79060 ) on Wednesday May 23, 2001 @01:50PM (#202399)
    What's with the "swarmcast gateway" that you have to use? If all we're getting is a GPLed client to an ultra-proprietary [swarmcast.com] secret server, forgive me for not getting too exited.

    Still a cool idea, but if OpenCola wants everybody to put eggs in their basket, maybe somebody should release a GPLed server too...
    --

  • Re:First Impressions (Score:3)

    by nlh ( 80031 ) on Wednesday May 23, 2001 @06:52PM (#202400) Homepage
    Well, my first problem is that the FTP server for downloading the JWS file seems to be /.'ed, so I can't get that and can't install SwarmCast, which would have solved the problem in the first place. Hmph...ironic, in a Catch-22 kinda way, isn't it? ;)

    nlh

  • License (Score:5)

    by AirLace ( 86148 ) on Wednesday May 23, 2001 @01:49PM (#202401)
    Sure, their client may be under the GPL but some of the terms on their license agreement are quite unpleasant:


    (a) Reverse assemble, reverse compile, or reverse engineer the Gateway (or any component or portion thereof), or otherwise attempt to discover any underlying Proprietary Information (defined below) of the Gateway;


    Isn't reverse engineering explicitly granted by law?


    (b) Sublicense, rent, sell, lease or otherwise transfer the Gateway (or any portion thereof) to any third party;

    (c) Remove or alter any marks or designations indicating the ownership of copyrights, trademarks or other intellectual property rights of any party contained in the Gateway;


    In fact I'd go so far as to say these conditions wouldn't feel out of place in a Microsoft EULA.
  • which everyone* watches anyway


    * well.. ok.. not everyone
    --
    Azrael - The Angel of Death
    posted with: Mozilla (0.9+)
  • I imagine a very similar thing.. but instead of a line of people, a mob of people surrounding a person tied to a post. If each person throws just one brick...
    --
    Azrael - The Angel of Death
    posted with: Mozilla (0.9+)
  • Uh, since when was peer to peer technology not legitimate? We've been using it for years in local networking and games and so forth.

  • Well, I tried it out and when I was the only one downloading I was getting 74KB/s. With each person joining, my speed dropped, until with 8 people on I was only getting 11KB/s.

    It seems as though the supposed advantage never had a chance to kick in...
  • It'd be a fun academic exercise to implement something like this in Sun's jxta environment...
  • I've got a cable modem, and my mileage definately varied from yours. Normally, my best download speed is about 150k/sec. I was peaking at 250k/sec and averaging about 200k/sec by the end of the downlaod. That's a pretty significant increase for me. That best download speed of 150k/sec is not very normal and usually peters off on larger downloads, to about 130k/sec.

    If more people start using the server, I'll be happy to use it.

    Now, that said, I'm sitting here watching my cable modem go nuts and I've got about 40 established connections. I can't really gauge what the throughput is, but it sure looks to be significant.

    I leave my machine on all day when I'm at work, though, so that doesn't really matter to me. Activate it before I go to work, build up karma, and then turn it off when I'm at home and not using it. That's cool..

  • this sort of peer-to-peer microcaching is just the sort of thing zodiac networks [zodiacnetworks.com] (a mountain view-based outfit founded by ex-netscapers mike homer + wade hennessey) is trying to accomplish. i'm pretty interested in the details and statistics of such an approach, and more so now that there is an open-source competitor.

    as a side note, i just underwent seven interviews with them only to be turned away because my "lack of windows experience would be too much of a hindrance". heh.

    what do any of you think about the differences between the two suites of software? zodiac's infrastructure and money will certainly encourage people to use its software, but going GPL arguably has the same effect.

  • The concept of Peer-to-Peer file distribution is not at all new. The interesting contribution here, if there is one, is if it catches on as a practical system.

    And just to add my contribution to the "someone else thought of it first" bandwagon, see my dissertation, available online here [crypto.com], which I believe anticipated what we now call p2p by at least five years. (Ironically, of course, this file is still available only via plain old http, from my not at all p2p centralized server...)

  • Wasn't someone compiling a list of valid uses for P2P? This certainly sounds like one to me.
  • The EFF is seeking help in this area of finding legitimate uses of peer to peer technologies...

    Maybe this might be a very good argument in court... content distribution at high speeds.
  • AFAIK, the origin server is always willing to send you more blocks. So if there are few (or no) other Swarmcast peers currently running, you just download most of the file from the origin server.
  • The cola is pretty good. I'm not going to give up Coke, though.
  • Data isn't any more persistent in Mojo Nation than in Freenet; they're both caches. Unless you meant something else...
  • From the FAQ [swarmcast.com] :
    Swarmcast breaks a file into packets, which are encoded using Forward Error Correction (FEC), a mathematical technique that Swarmcast uses to make it easy to reconstruct the file. The encoded packets are then distributed randomly to the computers that have requested the file. These computers become nodes in the mesh (a temporary network) that Swarmcast creates for this download.

    Each node receives only a portion of the original packets. But each node is also aware of some of the other nodes receiving packets. Even as a node receives a packet, it also rebroadcasts it to other nodes, so packets are rapidly swapped back and forth.

    As each packet is received, the receiving node checks to see if it is useful in rebuilding the file; if it is, Swarmcast decodes that packet. As soon as a receiving node receives sufficient useful packets, it reconstructs the file. Thanks to FEC, a Swarmcast download is a bit like playing poker when every card in the deck is a wild card: as soon as you've received five cards, you can build a winning hand.

    So there seems to be some amount of redundancy built in, provided enough servers are running at a time. When I tried it out, there was just one provider, but at 500Kbps.

  • Same reason, really: You share packets, so
    that others will share packets with you.
    Ideally, Swarmcast should be a background
    service that just makes sure that when you
    want a file, it comes as fast as your little
    pipe can swallow it. It will be, in time.
  • You don't need a browser to use swarmcast.
    Run it from a commandline once you have javaws
    installed.
  • Before you go off and start including this GPL'd code in your new and useful tools, you should be aware of several pitfalls involved in writing new programs using GPL'd code:

    Using GPL'd code may put you into severe financial jeopardy. Make sure to check with your local Microsoft representative to make sure you have paid the appropriate "GPL tax". Remember: this applies even if you don't own any Microsoft software.

    There are stringent limitations on the ways you can redistribute GPL'd code. If you redistribute any GPL'd tool in whole or in part, you must put a link to Midori's home page in the About box of your program.

    The GPL is "viral", and can cause your program to become GPL'd too. However, instead of including the code directly in your new program, use a C trick called "linking". To do this, all of the GPL'd code you wish to use must be in a "library". If it isn't, take all the parts you need and compile them into a library, then distribute that with your program.

    If you follow these tips, you can make great use of other people's hard work-- without running the risk of having to share any of your own. Good luck!

  • I got some at the p2p conference in SF a while back, it tastes great. Between Pepsi and Coca-Cola it's closer to coke or RC Cola. If you are a cola-addict you will be able to tell it apart from the name brand, but still quite good. As far as home-made cola, that's an entirely different subject...

  • Has anyone tried the cola from OpenCola? (http://www.thinkgeek.com/stuff/things/37f3.html).

    It sounds like an interesting idea, open sourced beverages - but does it taste good?
    --
  • First of all, what type of security is going to be implemented to prevent hacks. It seems that it would be pretty easy to shore up a single server, or even several in a single datacenter, but it would be a daunting task to protect thousands of machines spread thoughout the world against hackers.
    Well, that's an easy one. One centralized server which tells client what file to get, AND also tells md5 sum of the file & possibly also fragments (512k each, maybe).

    Has Swarmcast done anything to prevent this from happening.
    Now, to get an answer to that, you should read the article..

    --
  • When musiccity canned their opennap servers a few weeks ago, they released a new client (morpheus) built for a new p2p network that does the same thing... downloads are very fast! I think the idea of distributed file serving is really cool.
  • I don't run any servers (including Napster) on my machines right now, because of bandwidth, really, but also because I am afraid that my ISP (just one of the many which are now all part of Earthlink) will one day enforce its ToS re: servers. I'm online enough hours that I have drawn their tightlipped little notes before about it, and they hint that they think I'm running a server, so ... anyhow, keep my nose clean.

    This clearly functions as a server, albeit one which it seems would probably *reduce* overall congestion (am I right on that?).

    Anyone have constuctive / instructive experiences re: ToS with servers and dialup ISPs? This is one of the first things I've seen which tempts me to risk it;)

    Cheers,

    simon

    p.s. oh, and the cola? It's not that bad. Sweet, but not disgustingly so like "Big Red" and certain others. Quite drinkable, and no stranger than Coke or coffee would be to someone who's tasted neither one.
  • "chain is only as strong as its weakest link"

    Anne: Swarmcast, where does your knoweledge begin
    *Swarmcast sends TCP Packet
    Anne: You are the weakest link, g'bye.

    -Compenguin

  • So if each person is only sharing a part of the file, would the RIAA have a case if this turned into another MP3 trading technology? I suppose the RIAA could plead that since the entire file would be assembled on the end-user's computer it is still a copyright violation... but still, is this a loophole?

  • Saving content providors money? (Score:3)

    by _ganja_ ( 179968 ) on Wednesday May 23, 2001 @04:18PM (#202426) Homepage
    No thanks, if a content providor wants to provide files, they can pay for the bandwidth. If I want to download that file, that's my choice and thus I pay for the bandwidth it takes to download it at my end. However, just becasue I decide to download the file doesn't automatically mean I want to serve this file to anyone and everyone else.

    This saves the content providor bandwidth at my expense. Bandwidth is, of course, not free and somebody has to pay for it. Ximian take note, if you don't want to pay for the bandwidth involved in being in the software distribution business, get out of that business. For Joe Average that installs Red Carpet and doesn't understand that it now includes this new feature, is going to be mighty pissed when he gets his bandwidth bill only to discover that he's been serving 10 gigabytes to strangers due to this feature.

  • now we can slash dot more servers more easily and more quickly.

    ;-)

    And tie up more band width.

    Although I acknowledge that this is the opposite of what is intended by the system.

    maybe they ought to have a few distros on the system so we can help them test it out.

    Check out the Vinny the Vampire [eplugz.com] comic strip

  • This sounds like a great idea and all, but what incintive do I have for people to use MY bandwith, MY diskspace, etc for others? I know this sounds trollish, but one of the reasons napster was successful is because you could download music from anybody, but you coulnd't prevent people from downloading from you. Therefor you are willing to share your mp3s in return for being able to download some music. Unless you can come up with some reason for people to run the software, I don't see why people would want to run it. (Referring specificly to redcarpet)

    No matter how good you think humanity really is, I sure don't see it- I'm a greedy bastard and I want my bandwith.

  • Possible Security Issues (Score:3)

    by atrowe ( 209484 ) on Wednesday May 23, 2001 @01:45PM (#202429)
    Maybe it's just me, but the concept of a distributed file server just doesn't sound like a very good idea. I have several reservations about this software to which I couldn't find any acceptable answers on Swarmcast's site.

    First of all, what type of security is going to be implemented to prevent hacks. It seems that it would be pretty easy to shore up a single server, or even several in a single datacenter, but it would be a daunting task to protect thousands of machines spread thoughout the world against hackers.

    Secondly, what type of redundancy is going to be built into this system. Again, if a file is going to be served from a single centralised machine, it should ideally be fairly reliable, with multiple connections, and RAID to ensure continuous uptime. However, if you're serving tiny pieces of a file from thousands of boxen, it seems to me that if even one of those fragments doesn't make it all the way downstream, the whole file would be worthless. Has Swarmcast done anything to prevent this from happening.

  • ..Guess it depends on how far you want to move those bricks. That truck still has to be loaded and unloaded - If the bricks are only going 20 feet a nice line of people would work quite well. Not sure how this fits into your filesharing analogy tho'
    --------------------------------------
  • Okay, I downloaded the file (very fast!). Now what the hell do I do with it???? I can't watch quicktime movies on linux!!

    Also, the downloader is very "tricky." When I clicked "close," all that happened was it minimized itself and pretended to be closed--but it was really stealthily serving up data!

    Okay, I thought. So then I tried closing this app with my window manager...no dice! That is rather not nice. I had to do a killall java. But I think this thing is still lurking somewhere (after all, I gave it full root access like a good newbie should to all his programs, like red carpet ;0)

  • Ultra-Wide Bandwidth (timedomain.org) and SwarmCast just might -- let me stress JUST MIGHT -- be the technologies that synergistically propel us closer to the kind of cyberworld many envision. Foundational code distributed p2p via low-energy burts of UWB -- it could be the foundation for a ubiquitous surround-the-earth wireless ethernet much easier than any existing technology. Or I just might -- let me stress JUST MIGHT -- be wrong.
  • Uh, you most certainly could turn off file sharing in Napster. It would pop up a text box saying "it's not nice be so selfish" but that was it.
  • The previous poster makes a vary good point. The system seems to depend on a corum which may not exist. How is corum maintained? This is unclear.

    This is the same problem currently being experienced with Mojo Nation [mojonation.net], a P2P file sharing system which implements a similar method of delivering blocks of data from various locations then re-assempling the requested file.

    Perhaps OpenCola's solution to this problem is explained someware on their website but I looked and didn't see it.

    --CTH

    --
  • Cool concept in theory, but what happens in the real world when you hit congestion, or a client goes offline, or there's some other interruption? Could put a kink in things, a would definately slow things down. By having X clients be necessitated, you leave yourself vulnerable. You know, the whole "chain is only as strong as its weakest link" thing.

    -- Chris

  • This is a good idea. I wish I'd thought of it. But what if a malicious user is running a "custom" version that sends trojans to requesting peers?
  • Let's get Swarmcast goin on a s/390 [slashdot.org] mainframe and see what that does!

  • This is exactly the same peer-to-peer multiple-download that was done a long time ago by KaZaa [kazaa.com].

    In their system it is for downloading media (aka prOn) -- but the updates to their own software are also distributed using this technique.

    While it is true this is never as fast as a single fast server it is a lot better than other p2p systems.

    R.

  • i've posted to /. before saying I would love to host a Mandrake ISO or something in IDLE time. Why not? If someone finds me a cross-platform file sharing service, not free net and not mp3 only, I'll be glad to host something.

  • Hello, thats the wrong license (Score:4)

    by Edgewize ( 262271 ) on Wednesday May 23, 2001 @02:37PM (#202440)

    You've successfully quoted the Sun Java Web Start license. Now please click the word 'license' in Swarmcast's about box and you will see that there are no such reservations or claims.

    You are the weakest link. Goodbye!

  • RANT Better re-read the DMCA. Apparently, reverse engineering is illegal at the moment, no matter how easy it is to break. Better make sure that you don't inadvertently "figure out" how someone's code works! /RANT

  • First Impressions (Score:5)

    by Jade E. 2 ( 313290 ) <[slashdot] [at] [perlstorm.net]> on Wednesday May 23, 2001 @02:19PM (#202442) Homepage
    Well, I just finished downloading their sample file (a 20MB quicktime (uugh) trailer), and the technology seems to work, albeit not as well as a single fast server. When the download started there were 3 participants in the 'active mesh' and I was getting 41KB/sec or so transfer. By the time it was done there were 12 participants, and I finished the download at 55KB/sec. Even after the download finished, the Swarmcast client is sitting in my tray (Yes, this is a windows box) happily transmitting at between 8 and 14KB/sec.

    Thing is, I don't see why anyone would use this. In reality, the transfer rates aren't as good as a single fast dedicated server (I can easily get 75-80KB/sec on this line), so there's really no gain on the client end from using this. On the server end, yeah you're using a lot less bandwidth. This might be useful for open source projects or other products that are downloaded by knowledgable people, but you're average computer luser isn't going to want to download 6MB of Java Runtime/Swarmcast Client just so they can save your company money by getting slower-than-normal downloads. (And I shudder to think what a distributed p2p network comprised mostly of 56k modems would be like, at least at the moment it seems most of the users online are broadband.)

    Anyways, pessimism aside, here's what I'd like to see:

    • Some indication of how much bandwidth you're 'serving'. (My estimate above is based on the stats in my RRAS console, there's no indicator anywhere that you're even sending data.)
    • An option to automatically shut the client down when you're done with the file. This may seem selfihs, but I still know people paying by the hour for 'net access, and until they can be sure they can start a download with this and have their connection go idle when it's done, they're not going to use it.
    • Some sort of statistics (Number of packets received, number served, total clients served, things like that.)

    OK, I'm done for the moment. If they play this right I can see it helping out smaller outfits with knowledgable users, maybe eventually even going mainstream if they can convince people it's worth their while to install. It's definitely sparse on information in it's current form, but hey, it's a beta. It's a good idea, and I wish them luck.

    -Jade E.

  • You have a large file, you want to share it, this is what swarmcast is for. Here's *a* scenario: You made a movie. You want to share it. It becomes popular. Other people like the movie and want it to get around. (After all, they were the first in this cool movie, they want to see it popular). Or, imagine fifty people downloading a 500mb file in one night from your apache on your cable modem. There are many other possible scenarios.
  • Imagine you have a 500mb file. Are you going to share one that people think suck? Can you handle more than one person downloading a 500mb file from your cable modem?

    With swarmcast, the load is distributed. If people like the file, they will stay in the mesh. Just as with gnutella, if people like the files they have, they will share them. (And, get kicked off their ISP because of their piece of shit TOS).

    :)
  • FAQ's 1,2,3 in support answer these things. It should be noted that cable modem users often like to download large files. For instance, a Linux ISO.

    Believe it or not, but very often those very large files are not accessible because too many people are already downloading it. In fact, you don't even see that many movies, for instance, on the internet, as there could be... because of these bandwidth problems.

    Even though, so many cable/dsl users are happy to pay for that extra bandwidth.

    http://www.swarmcast.com/gate/support.html#N77

    1. Download speeds are slow

    There are two possible causes for this problem:

    You are using a dialup connection or a shared (LAN) connection that is busy. Swarmcast ensures that you get maximum usage of your download bandwidth. It can't, however, give you more bandwidth than you already have.
    The mesh isn't large enough to efficiently distribute the download effort. Swarmcast-enabled downloads go faster the more nodes there are in the mesh. If you find the download unacceptably slow, try getting friends to join the mesh with you. If you are the only user in the mesh, you'll do a straight HTTP download. Remember, Swarmcast is usually faster than straight HTTP, but it's never slower.
    2. Installation is inefficient

    For the time being, Swarmcast must be installed and updated using Java WebStart. We are aware of the limitations this imposes, and are developing a more efficient solution.

    3. The Swarmcast installer is too large

    This issue is related to the use of Java WebStart as an installation tool. We are developing a solution.

  • Open link with jnlpfile from the webstart download window.


  • I agree.

    When I'm playing Counter-Strike and it changes to a level which I don't have, it has to be downloaded from the server.(*) Of course the server is in the middle of running a game, so it can only afford 2-5 KB/s of bandwidth. So even though I have a 100KB/s broadband connection, I'm sitting there waiting to play a game while sipping from a straw.

    It seems obvious to me that it would be tons better if I could get the file in pieces from some of the other people playing. When I'm playing CS only ~ 6-10KB/s total bandwidth is used, which leaves me tons of free bandwidth to help serve such things. Of course I would have to be able to manually 'throttle' it and configure a maximum amount of bandwidth to share, but otherwise it would be tons nicer.

    It has also occurred to me a long time ago that web-content could (hell should) be served in the same way, although you would have to configure a maximum amount of data to 'cache', and a maximum amount of your bandwidth to share, and allow you to 'turn off/on' the sharing, and to ask for certain content to be 'permanently' mirrored, and conversely to ask for none/only-requested content to be mirrored.

    No, I don't see us subsidizing the costs of big corporations. I see us using this to allow the little guys to compete with the big guys without having to shell out for the big pipe. Heck, we could even have 'meta directories' maintained by groups/individuals/organizations of sites worth mirroring, and we would just tell our browser to 'use that meta directory' as the list of places to help mirror.

    It does also promote efficiency. We don't need every site in the world to have a super huge pipe, we just need all the pipes in the world to work together to serve content.


    (*) I could exit the game and try and find the level on the web and DL it manually, but with all the game-sites going under and fileplanet.com being maxed out nearly 100% of the time, it takes 10x as long to get a new/uncommon level myself as it does to sip on a straw for it.

  • I had thought of the same idea when I was downloading some large files from an overworked FTP site where the demand easily outweighed the supply. The idea is that if you have a lot of information to distribute (like some large media files) but don't have the server capability to serve everyone at once, just require everyone to use a program like Swarmcast. The more popular the data is, the more temporary servers are setup, and the transfer may actually go faster. This is in contrast to the current way of distributing data, as the number of transfers increase the speed that any single user can receive the file decreases.

    In short, if you're running a popular file server off a standard broadband connection, just have everyone use Swarmcast. It'll ease your bandwidth use by a great deal. And if you're distributing your data for free, your users shouldn't have too much of a problem with installing this software and donating a little of their bandwidth.

  • Actually, I'm working with a few guys on a program similar to this, and the technology is a "killer app". You don't need to have X users online in order for all the packets to be available. Essentially, everyone that has downloaded a "swarmcasted" file will have all the packets necessary to produce a complete file. The more people with all the packets, the more routes the packets are able to go through. If only two clients are up, you can DL from two routes simultaneously. If 5, then you can go 5-way, etc etc. This helps when.. say.. each of 5 routes are bottlenecked at 10k/s. You can effectively get a transfer rate of 50k if you simultaneously DL from all 5 routes, or, if you like the traditional way, you can take your pick of which source to DL from and get stuck at 10k/s. Imagine if you could simultaneously DL all packets from all your neighbors for a linux distro (assuming they all have the distro you want) instead of having to go through some central server a thousand miles away with bottlenecks all over. Even if you have a T-3 and they all have DSL's, you effective get a througput of N*M (N = number of neighbors with the distro, M = the max DTR of the DSL modems) The upper limit is either the max combine DTR of all DSL's or the max DTR of your T-3. Of course, that is a best case scenario, but you get the picture. That technology facilitates a minimum transfer rate = the fastest connection with a complete set of packets and a max that is hugely scalable (in theory) to nearly infinity.
  • From the swarmcast website "Since each node is only a member of the mesh for the duration of its file download, the network can be extremely unstable and dynamic. Swarmcast, however, continuously maintains a rapid scaling mode; there are always new consumers joining even as old servers leave. Further, if a node repays to the network the amount of data it consumes (with a bit of buffering to deal with bursts), then each node will be able to download data as absolutely fast as its connection allows, even if the giving back is spread over a much longer period of time than the download took." Hope this answers your point
  • Sorry forgot the link to the site [opencola.org]
  • Not quite, but still the wrong license: He's quoting the license for the Gateway, which is the website that acts as a peer-introduction-service; the Gateway is not GPL (it's a very specific thing, not a very general tool or protocol).

    Don't worry, the Gateway isn't tracking you: Your node anonymously requests a "lease" on a resource and until that lease expires, new mesh members are given your node as a possible entry-point. It's a salve until multicast discovery can do the same thing in pure peer-to-peer mode. Other apps such as Ximian, where the audience is focussed on some website, will probably also use Gateway-like software to stitch the mesh together.

    But don't confuse them: There is precious little SCL code in the Gateway; it is only there to keep a temp cache of REMOTE_HOST headers and link them to known resource SHA hashed IDs.
  • Well, for one thing, the per-block SHA hash of the trojan wouldn't match the original, so the swarm would learn to ignore that source of data. At most, you slowed down a few people imperceptably while they fetched useless blocks from your trojan server.
  • Don't like the sample content? Why not register your own, for example, how about RedHat 7.1?...

    RedHat 7.1 disk one [opencola.com]
    RedHat 7.1 disk two [opencola.com]

    Of course, it's only useful if several people do this all together ;)

  • This is a great idea! It'll let high traffic sites like kernel.org distribute the work out to others. I mean it's nice that they have a 100 Mbit pipe thanks to donated bandwidth, but now they can more optimally use it by allowing users to share portions of files between each other.

    Don't get me wrong, there are plenty of problems with this system also. Security, lowering the bar for newbies and other problems have been brought up in other posts. I won't rehash them here.

    However, this will go a long way towards helping the open source community more easily share files -- without requiring that you beg some private money making companies to donate bandwidth. And that in my mind outweighs some of the negatives.

    I especially like that they forbade sharing any files for which you don't have a copyright. Yes, I'm sure some people were thinking of using this for mp3's, warez, etc, but that would have killed the general acceptance of what is a great way to scale file downloading.

    Just my $ 0.02 worth.

Slashdot Top Deals

Top Ten Things Overheard At The ANSI C Draft Committee Meetings: (1) Gee, I wish we hadn't backed down on 'noalias'.

Close