Forgot your password?
typodupeerror
The Internet Technology

Company Offers Customizable Web Spidering 46

Posted by ScuttleMonkey
from the my-legs-are-longer-than-yours dept.
TechReviewAl writes "A company called 80legs has come up with an interesting new web business model: customized, on-demand web spidering. The company sells access to its spidering system, charging $2 for every million pages crawled, plus a fee of three cents per hour of processing used. The idea is to offer Web startups a way to build their own web indexes without requiring huge server farms. 'Many startups struggle to find the funding needed to build large data centers, but that's not the approach 80legs took to construct its Web crawling infrastructure. The company instead runs its software on a distributed network of personal computers, much like the ones used for projects such as SETI@home. The distributed computing network is put together by Plura Processing, which rents it to 80legs. Plura gets computer users to supply unused processing power in exchange for access to games, donations to charities, and other rewards.'"
This discussion has been archived. No new comments can be posted.

Company Offers Customizable Web Spidering

Comments Filter:

Lisp Users: Due to the holiday next Monday, there will be no garbage collection.

Working...