Want to read Slashdot from your mobile device? Point it at m.slashdot.org and keep reading!

 



Forgot your password?
typodupeerror
×
Canada Communications Bug IT News

Massive Email Crash Hits Canadian ISP Shaw 150

rueger writes "One of Canada's biggest cable/Internet providers has their customers in an outrage. '... after an interruption of Shaw's email services Thursday led to millions of emails being deleted ... About 70 per cent of Shaw's email customers were affected when the company was troubleshooting an unrelated email delay problem and an attempted solution caused incoming emails to be deleted ... Emails were deleted for a 10-hour period between 7:45 a.m. and 6:15 p.m. Thursday, although customers did not learn about the problem until Friday, and only then by calling customer service or accessing an online forum for Shaw Internet subscribers.' To top it off, when Shaw did send out notices about this, they looked so much like every day phishing spam that many people deleted them unread."
This discussion has been archived. No new comments can be posted.

Massive Email Crash Hits Canadian ISP Shaw

Comments Filter:
  • Blame? (Score:3, Funny)

    by Anonymous Coward on Sunday March 10, 2013 @10:04PM (#43134987)

    Who? [youtube.com]

  • WAIT.... "deleted?"

    As in, "spilled the seed on the ground?"

    I can understand if things maybe don't get delivered for a few hours, or maybe a few got munged up somewhere during the repairs, but to blissfully direct the firehose into the abyss that is /dev/null...

    Who pays for that?

    Oh, wait. Canadians are rather accepting of abuse on the part of their phone/cable/broadband suppliers, and the Tories back up the big businesses.

    "Unacceptable! Unacceptable! Mepps, mepps!"

    • by isopropanol ( 1936936 ) on Sunday March 10, 2013 @10:32PM (#43135127) Journal

      Shaw is probably the least abusive of the canadian major telecom companies. I've been a shaw customer for 14 years and this is the only incedent I've had other than lines being blown down in a storm. My wife's email was effected but mine was not. This is a normal (and rare) human error... most of the actual abuse telecom companies dish out is abusive contracts and misleading advertizing like 3-year cellphone contracts and "Optik TV and Internet" ... which is actually satellite and DSL, not FTTH.

      • by slazzy ( 864185 )
        I would say Distributel is less abusive, similar prices, runs over Shaw lines but internet is unlimited - no overage fees. Teksavvy used to be good too, not sure if they still are.
        • Teksavvy seems to be getting worse. I check them every so often, and last time I checked, their AUP now specifically says you can't run servers; I'm pretty sure it didn't used to say that. They don't appear to offer static IPs for residential accounts any more. Their business pricing isn't very competitive. Distributel's AUP looks similar to Teksavvy's and Shaw's. I can't really see much to recommend any of the resellers over just using Shaw directly any more unless you really need the 'unlimited' data. Tha

          • by Mashiki ( 184564 )

            Teksavvy seems to be getting worse. I check them every so often, and last time I checked, their AUP now specifically says you can't run servers; I'm pretty sure it didn't used to say that.

            No it's always said that. Though they really don't enforce it unless people are being abusive, it's more of a CYA clause. They only offer static IP's for DSL customers, that being the nature and problems with the cable plants used by the majority of companies(rogers/shaw/cogeco). Really though everyone got the shaft from the CRTC on the latest round of TPIA agreements, and tek is moving to a new ATPIA system which will cost more. But you'll get more, which is okay. Though they're still fighting the rul

          • by pod ( 1103 )

            Shaw actually has pretty crappy caps, and they DO enforce them. My Telus internet usage has recently, in the last 4-6 months, been removed from their services page, so even if they measure it, they can't enforce caps, because users can't check their own usage. Also, Shaw fairly strictly throttles P2P, and is happy to serve copyright infringement warnings.

            That's not to say either one is particularly great.

            I used to have ETTS from Novus, and that was very nice for a residential service, but it is only offered

            • Shaw only throttles *upstream* P2P, so being a selfish ass, I don't care. I throttle it harder than Shaw does myself. :P

              Shaw's published caps are clearly better than Telus'. I can't speak to enforcement any more, though. I actually used to work in Shaw's AUP team (which was like five people in the Vancouver office) and we'd just do the top X% of worst offenders from overloaded routers and they got an assload of warnings and phone calls before any enforcement, but that was years back (when the caps were actu

            • Nonsense, I finally* got a call from Shaw after my second month of using 1Tb of my 200Gb cap, and they basically said, 'hey, quittit.' No threats, no charges. I have never* received any* copyright warnings either...
      • Re: (Score:2, Insightful)

        by Anonymous Coward

        This is a normal (and rare) human error...

        Making mistakes is human. This is why a competent professional acknowledges that he will make a mistake sooner or later, and designs his activities so that mistakes in execution won't have catastrophic consequences. These guys failed to properly do this.

      • by Anonymous Coward

        Shaw might be less abusive of their customers of their internet service.

        When it comes to other things, they're nasty as hell.

        Barrett Xplore ring any bells for the Canadians here? Shaw taking over Starchoice ring any bells, too?

        I ran my own store and decided to sell satellite service and equipment. Barrett Xplore wouldn't even give me the time of day. Eventually their answer was "FUCK OFF" (not joking, those were the actual words from the sales droid). I took their advice and sold FTA equipment after tha

      • Re: (Score:2, Informative)

        by Anonymous Coward

        I work for TELUS, and can assure you that optik tv is NOT a satellite service. It is an IPTV service being delivered over either ADSL 2+ or VDSL connections (and sometimes fibre in new areas).

        Satellite tv is offered in areas where we don't have the broadband infrastructure to support standard Optik TV. However we refer to that as TELUS satellite TV and not Optik tv.

        I don't believe TELUS ever claimed optik was FTTH. The optik name refers to the fact that it is served by our new fibre network. The "last mile"

      • by Anonymous Coward

        Shaw is probably the least abusive of the canadian major telecom companies. I've been a shaw customer for 14 years and this is the only incedent I've had other than lines being blown down in a storm. My wife's email was effected but mine was not. This is a normal (and rare) human error... most of the actual abuse telecom companies dish out is abusive contracts and misleading advertizing like 3-year cellphone contracts and "Optik TV and Internet" ... which is actually satellite and DSL, not FTTH.

        Less abusive to customer perhaps, not to their employees.
        http://www.cbc.ca/news/canada/british-columbia/story/2013/03/08/bc-shaw-contracts.html

      • "Optik TV and Internet" ... which is actually satellite and DSL

        I'm a very happy Telus "Optik TV and Internet customer", and I assure you it's not satellite - It's IP TV. And yes, my Internet is delivered over a copper pair, but so what? It's fast and reliable - Faster than I was getting from Shaw when I switched.

      • by epine ( 68316 )

        I've also been on Shaw since pretty much continuously since the first month they offered service, and I live in one of the first cities activated. Yes, about 14 years. Haven't used their email or web services since the first year.

        I had one incident with Shaw which was very annoying. An OpenBSD firewall just suddenly stopped working with no change on my part. If just the firewall accessed the internet, it worked normally. But as soon a NAT client relayed traffic through the firewall concurrently, respon

  • Things like this (Score:5, Insightful)

    by suprcvic ( 684521 ) on Sunday March 10, 2013 @10:14PM (#43135043)
    Are one of the reasons I don't use ISP hosted email. Main reason is portability.
    • At least in the USA, then you get nailed for anywhere from 10-40% surcharges for "business-class service" which basically means you pay $400 a year for fixed IP and the "privilege" of running a server on your own broadband line."

      My ISP tries to explain the ridiculous cost boost on "well, you get 'priority' on service calls. You have 10 email addresses and 10MB of free web space on our server!"

      I shut them up one time:

      I've been on "the internet" longer than you've existed as a company. Even at 2am, nobody has

      • You don't need a fixed IP to receive mail. For years I've used a dynamic DNS service and that worked just fine. I've even for a while ran a web site off a dynamic DNS, also worked. Not recommended of course, but it works.

        • by ls671 ( 1122017 )

          Be aware: that was the old days. Nowadays, most need a business class service just to have port 80 and 25 unblocked. You can't run a web server or a mail server on most customer grade connection because the ports are blocked by your ISP.

          I used to do it too. Only risk was that during the IP switch, due to the lag of DNS updates depending mostly on TTL and how server respect it, you may have somebody impersonating your site or grab your mail.

          Risk was very slim, the machine that gets your old IP would have to

          • by pe1chl ( 90186 )

            How terrible!
            And how do you access your own NAS or printer over the net?
            These days, more and more devices come with their built-in webserver that enables the owner to contact home from his smartphone or tablet.
            It is a security nightmare, but that is a different topic.

            In the Netherlands, it is forbidden to filter internet other than for security or customer-opted convenience reasons (e.g. spam filtering).
            Many providers also offer fixed or semi-fixed IP as standard feature.

      • More importantly, you get an IP address from a range that is not on the ISP's dynamic range, which is always going to be on the RBLs, so good luck running your own mail server without a business class line. Personally, I just relay through dyndns, but then you lose the ability to see errors directly from the receiving party. I'll probably get around to getting a business line again eventually mostly because of this.

      • by msauve ( 701917 )
        You can get your own domain for a few bucks a year. You can buy hosting for a few bucks a month. From that, you can effectively have your own server, for maybe $100/year. You don't pay for electricity, maintenance, upgrades, etc., and the bandwidth it uses is free and doesn't interfere with your own use.
    • My university (I graduated long ago) gives a complementary email account to all allumni. Very useful for maintaining a constant address regardless of ISP.

  • Don't worry (Score:5, Funny)

    by the eric conspiracy ( 20178 ) on Sunday March 10, 2013 @10:17PM (#43135055)

    There will be more mail tomorrow.

  • I've hosted my own mail server for about 15 years and I regularly think to myself, 'I'm tired of worrying about hardware and my circuit. Maybe I should let somebody else host it.'

    Then it seems there's always an article like this that clears my head.

    • by gweihir ( 88907 )

      I have primary and secondary on differently hosted vservers with local backups (via procmail) and immediate forwarding to my machine at home. Very little chance of loss and hardware is the provider's to worry about.

    • I've hosted my own mail server for about 15 years and I regularly think to myself, 'I'm tired of worrying about hardware and my circuit. Maybe I should let somebody else host it.'

      Then it seems there's always an article like this that clears my head.

      Why are you continuing to run a set-up that, by your own admission, is a great hassle ameliorated by a once in blue moon event? Seems like wearing a crash helmet whenever outdoors, and justifying it by pointing at an incident in which a pedestrian got clocked by a golf ball and ended up with a brain clot.

  • Cox over the years has had some spectacular email outages and fuckups. To the point where I now use Gmail via IMAP and a private domain via IMAP.
  • /dev/null (Score:5, Funny)

    by linatux ( 63153 ) on Sunday March 10, 2013 @10:29PM (#43135119)

    isn't a holding-bay?

    • by Sipper ( 462582 )

      isn't a holding-bay?

      /dev/null isn't, but sadly the "Trash" folder is.

      A few years ago I was working as an email administrator and got a call to someone's desk that was having a problem with their mail client because some of the folders were too full. One of them was Trash, so I was about to erase messages from the folder when the user paniced; "wait, that's important!"

      For whatever reason, they were using the Trash folder for "real work"
      (Sigh.)

      • by dbIII ( 701233 )
        I had that too - some insane bullshit of using it as a transfer point to sort stuff. I'd set up mail clients to empty Trash on exit, and when the guy that was doing this logged back in it was of course empty, so he came to rant at me in the lunch room to the amusement of all onlookers. Now every few months he rings me up about a full disk, and each time I have to suggest emptying the Trash mail folder and the "Recycle Bin" on his desktop.
        It's not that the person in question is stupid, it's that such peopl
        • by Sipper ( 462582 )

          I had that too - some insane bullshit of using it as a transfer point to sort stuff.

          If I remember correctly (it's been a long time) the user I was dealing with was deleting messages he had dealt with, but got into the habit of referring back to the messages when necessary in the Trash folder. I suggested moving these sort of messages to another locally-stored folder, but the user refused saying "I'm used to working out of my Trash now."

          I'd set up mail clients to empty Trash on exit, and when the guy that was doing this logged back in it was of course empty, so he came to rant at me in the lunch room to the amusement of all onlookers.

          Logical. I considered trying that, but never did.

          Now every few months he rings me up about a full disk, and each time I have to suggest emptying the Trash mail folder and the "Recycle Bin" on his desktop.

          That reminds me... another thing that was going on back then was that a small set of users refused to del

  • by gweihir ( 88907 ) on Sunday March 10, 2013 @10:33PM (#43135131)

    Like, ones that make a backup before messing with critical data? As an elementary precaution known to anybody halfway competent in IT?

    This just demonstrates a massive, massive management screwup, as they allowed unqualified personnel to work on their systems. Save a buck, loose a million.

    • Yeah, messing up is normal. Failing mark a snapshot before becoming with a million emails is incompetence. With a snapshot, human error might have resulted in losing three minutes worth of emails.

      "To err is human, to fuck up the whole system requires root."
      • by gweihir ( 88907 )

        Indeed. And with preventing the server from accepting emails, and a snapshot, there would have been no loss at all. (Emails in the 3 minutes going to the secondary...)

        Those truly incompetent are those not aware that they can make mistakes. Seems management is trying hard to make the engineers more like them.

        • by ls671 ( 1122017 )

          Yeap, the whole mail system is designed from the core so mail should never be lost as I learnt in my young days.

          Managing to loose a single email never mind millions is quite an achievement.

          • by dbIII ( 701233 )
            MS Exchange lowered the bar. Yes I know it's supposed to do a dozen other things but it's MTA was crap for years and still seems to generate a lot of panic on sysadmin mailing lists.
            • by gweihir ( 88907 )

              panic on sysadmin mailing lists

              Hehehe, good old Microsoft. Never transparent, never clear, always good for surprises and gets more obscure than Linux kernel hacking when you have to fix problems MS did not anticipate. In German we call these "Schoenwettersysteme" (translates as "nice-weather only systems"). Toys, not fit for any real-world use.

            • I would never trust exchange as a relay. Everywhere I worked where I had the power to do so, Exchange did not sit in the DMZ, and relayed through proper unix mail servers. I prefer sendmail, because I am familiar with it and know how to properly extend and secure it. Use postfix if you prefer, but again, I'd never trust a Microsoft Exchange mail relay.

          • Yeap, the whole mail system is designed from the core so mail should never be lost as I learnt in my young days.

            There were always ways to lose mails. One obvious way is if a mail server dies then you lose all mail between the last backup and the mailserver dieing. Another is if both the original mail and the bounce suffer a delivery failure but these circumstances were rare afaict. The majority of the time mails were either delivered successfully or bounced to the sender.

            Then spam and virus mails with faked from addresses came along. If you bounce such mails you create backscatter for an unrelated user. If you reject

        • by dbIII ( 701233 )
          There's a lot of incompetence about, especially bullshit such as the secondary being /dev/null itself as some sort of stupid anti-spam bandaid. I was stung by that one when I had the situation where the primary that was accepting mail for a company I was working for was getting congested and the host they were paying the ISP to supply as the secondary had a management imposed policy of just dropping everything. Probably 2/3 of incoming mail during working hours was never delivered in that four month windo
          • by gweihir ( 88907 )

            There's a lot of incompetence about, especially bullshit such as the secondary being /dev/null itself as some sort of stupid anti-spam bandaid.

            How stupid is that? Incredible! The whole reason for secondaries seeing more spam is that some of them do not have spam filters because of incompetent mailadmins. The fix is to either have the secondaries forward to the primaries (when they are back up and storing for some time before that) or to have the same spam filter on the secondaries.

        • You don't even need a secondary. If your SMTP server goes off-line, the senders should retry for up to 4 hours. So you can quite literally unplug a mail server, do what you got to do within 4 hours, plug it back in and no mail wil be lost.
          • by gweihir ( 88907 )

            You are right. I was thinking of IMAP servers for clients sending outbound mail, but they should be separate and a secondary would not help.

            Although the postfix/sendmail default for delivery failure is 2 days, I believe.

    • by Maow ( 620678 )

      Like, ones that make a backup before messing with critical data? As an elementary precaution known to anybody halfway competent in IT?

      This just demonstrates a massive, massive management screwup, as they allowed unqualified personnel to work on their systems. Save a buck, loose a million.

      Speculation: backed up the email server settings, made minor change such that spam matched against "*" wildcard.

      Spam of course gets deleted and not backed up.

      Hours later(!), someone notices overly aggressive spam filter and restores backed-up rules.

      Just prior to xmas, I was without internet for > a week due to some routing issues within Shaw, so it may well be that they have an over abundance of incompetence these days.

      Also, lost->lose. Tight->loose. See how easy it is to make a simple mistake o

      • by gweihir ( 88907 )

        Hmm. Good theory!

        If they were halfway competent, they would store the spam for a few days though, just for this eventuality. Personally, I keep all spam on my own mail servers for a while.

      • by green1 ( 322787 )

        I think you might have summed up the real issue in the part "Hours later(!)" ... I understand mistakes, I've made my share of real doozies... but what's the first thing you do after changing a system? TEST. Sure sometimes you don't manage to test every conceivable way something works, but even the simplest test will notice a complete failure like this (every time I touch my spam rules the first thing I do is send myself an email to make sure it still gets through)
        So while I unfortunately understand a short

    • by Anonymous Coward

      I was just thinking that. I realise that there is a lot of data, but backups are really important. I've lost stuff before (drives dying) and I'm getting quite anal about making backups. I have a website with a crapload of software that I've built, including a backup system (backups the sites, the databases, and the backup software). The test is that its all saved my poor miserable hide, more than once. Accidentally deleting data is bad. Drives crashing is worse. I used to play an old-timer game calle

      • by gweihir ( 88907 )

        Indeed. Everybody with real experience has lost or come close to losing data that they cared about. Those that can learn from experience learn to never be without backups and in particular to never, ever, ever work on life data without backups _before_ it is their job to handle critical data for others.

  • by Anonymous Coward

    From TFA : "The mistake was an “isolated event,” Lakshman said, and promised a detailed review, which would include a discussion about compensation."

    Except it isn't. Few years ago I had a business's domain email hosted with Shaw (was included with the internet service and they provided IMAP), and they lost all of it. They wouldn't return my calls about it, and on the third time I called in a week or so later I was told it would not be recoverable, that there is no backup for their business ema

    • by v1 ( 525388 ) on Sunday March 10, 2013 @11:19PM (#43135281) Homepage Journal

      From TFA : "The mistake was an âoeisolated event,â Lakshman said, and promised a detailed review, which would include a discussion about compensation."

      "A small number of our customers may be experiencing a minor problem with..." is standard big provider boilerplate for "between some and most of our customers are having a major problem with..."

      Until proven otherwise, when I hear "a small number of...", I immediately translate that into "a few have yet to complain to us about..."

  • I've been working with a Vancouver based retailer's email newsletter for years. Around here, Shaw is by far the worst of the bigger email domains on the list for deliverability - at least on any of the big webmail providers, recipients can white-list the email address we use to send out emails. Further, some emails will be completely deleted and not put into the junk folder at all. And emails that are suspected spam, will be deleted after only 7 days - don't go on a 10 day vacation.

    I could go on, but suffic

    • BTW: Gmail provides IMAP and POP access, which is a stumbling block for those who want a desktop email client. I'm not sure about Yahoo or Hotmail.

      I'm sorry, I don't follow your logic. How is providing the option for POP and IMAP -- in addition to webmail -- considered a "stumbling block"?

      • by WoTG ( 610710 )

        Oh, sorry, I meant that in a good way. I meant that a lot of people need a desktop email client, and in the past Hotmail and Yahoo didn't offer that, whereas Gmail has had it for years.

  • by _Shorty-dammit ( 555739 ) on Sunday March 10, 2013 @10:49PM (#43135191)

    Also, gmail exists.

    • by Almost-Retired ( 637760 ) on Monday March 11, 2013 @01:28AM (#43135679) Homepage

      Damn, isn't there anybody here but me who has been locked out of their gmail account for about 2 weeks now? I have not changed a thing in my fetchmailrc or mailfilterrc's, and have been sucking my gmail account dry at 3 minute intervals with fetchmail for damned near 5 years.

      2 weeks ago, both fetchmail and mailfilter started reporting password failures. It worked about 30 minutes a day for 5 or 6 days, but has not worked since the last week of February.

      I call them up, get some yahoo whose command of English sucks dead toads through soda straws, he leaves to go get someone who speaks English, but the next guy isn't a hell of a lot better, and he finally speaks clear enough that he is telling me the account is blocked because my machine is compromised. I object, its a linux box, behind a router running DD-WRT. Doesn't make squat to him, my machine is compromised.

      Seeing as how everything that comes in here has to run the clamav gauntlet, and that this is a linux machine which has not had java enabled anywhere near firefox in months, currently at V-19.0.2, AND that its behind a router running DD-WRT, AND neither chkrootkit nor rkhunter can find anything to complain about, I seriously doubt it has been compromised.

      I had been gradually weaning my mailing list activities, moving them to other servers precisely because of their no dups policy, so that was all the impetus I needed to just move all my subs. I still scan them on schedule just in case they actually get someone who reads english wondering why a fetchmail instance is failing to login, telling fetchmail the password is toast when its the same pw I've been using for years, and its long enough John didn't get it in 6 hours of grinding on it when I last checked with john the ripper.

      Until that happens, screw gmail, and the camel that rode in on them.

      Cheers, Gene

      • by Anonymous Coward

        Did you ask for your money back?

      • by DNS-and-BIND ( 461968 ) on Monday March 11, 2013 @06:29AM (#43136529) Homepage
        You have to log in with the gmail interface and answer a captcha. Then your account's back on.
        • Tell that to google. I have no access by any method. End of discussion. I didn't even call them until after my username and passwd known to be good, was rejected trying to login via FF.

          I don't use webmail. Ever. Its a solution promulgated because they can wrap it up in so damned much advertising that you sometimes can't find the frigging message. Why folks, mostly winders users I suppose, use it, and put up with the hassle of spending 5 minutes to log in using a browser, when that is an automatic func

  • Cmon (Score:2, Funny)

    by Anonymous Coward

    Haven't we all fantasized about just deleting the goddamn queue and going home?

  • http://it.slashdot.org/story/12/07/13/2050234/citys-it-infrastructure-brought-to-its-knees-by-data-center-outage

  • by trawg ( 308495 ) on Monday March 11, 2013 @12:44AM (#43135549) Homepage

    ...if your email is not in at least two physically separate places, you are at risk of losing all of it, forever.

    It's weird Shaw can't restore from a backup - the article is a bit weird on the exact details about what happened and just ends with "the emails were not backed up".

    If your online mail provider does not allow you to access or export your data to your own PC (via IMAP, POP, or whatever) then you should switch to one that does - and start backing up your own email if you want to be more confident that it's going to survive catastrophes.

    • While it's utterly trivial to alias everything incoming (or even outgoing) to another host that's another bit of infrastructure and often seen as an unnecessary expense. Their backups will be system files, whatever is in the mail spool at any given day is beneath their care factor and anything that arrives after the last backup is gone anyway.
      Remember this folks before considering outsourcing, it's not their email so they don't care about it as much as you do. While you may want to keep stuff in two place
  • What? (Score:5, Informative)

    by AdamWill ( 604569 ) on Monday March 11, 2013 @01:31AM (#43135687) Homepage

    "To top it off, when Shaw did send out notices about this, they looked so much like every day phishing spam that many people deleted them unread."

    Erm. No they didn't? I'm looking at one right now and it doesn't look remotely like 'every day phishing spam'. It doesn't offer me anything, threaten me with anything, or ask me to click on anything. It doesn't include any links except to a forum thread, which the text doesn't make any special effort to make you click on. It didn't trigger my mental 'phishing detector' in the slightest.

    I got the email notification late Saturday, two days after the event happened, I guess. That's not a horrible delay. I also saw a bunch of delayed mails come through around that time - 10 or so - and they notified me of the sender and subject line of three mails that were lost, so looks like they managed to recover quite a lot.

    I dunno, I guess I'm not TOTALLY OUTRAGED at this. As another commenter said, you know, admins screw up sometimes. Lord knows I have. The fact that they're at least able to identify the subject lines of all the lost mails makes a big difference; you could get any really vital ones re-sent.

    • Oh, and I should have included that there were zero spelling errors, which pretty much disqualifies it as a phishing attempt on its own. :) The Venn diagram of 'phishers' and 'people in possession of working spell check' seems to have no overlap.

  • by Anonymous Coward

    Well if everybody has their mail servers configured correctly the incoming mail should be flag for redelivery by the sending MTA for at least 2-4 days, so hopefully nothing is lost. I believe Sendmail is 4 days. You would think with so many users Shaw would also have at least secondary MX records for failover. Despite being a horrible protocol email does have it's delivery protections. The problem these days is that everybody *expects* immediacy with a technology that was designed with broken connections i

  • But, if anything happens, It's my own fault. I don't have to trust my ISP to do anything but provide the pipe.

In practice, failures in system development, like unemployment in Russia, happens a lot despite official propaganda to the contrary. -- Paul Licker

Working...