RIM Server Crash Leaves Millions Without BBM 191
Several readers have sent word that "tens of millions of BlackBerry users in Europe, the Middle East and Africa have been unable to receive or send emails and messages through their phones, following an outage at the server systems of parent company Research In Motion." RIM has confirmed that they're aware of the problem and working to restore service. A former RIM employee said to The Guardian, "They didn't start looking at scalability until about 2007, when they had around 8M active devices. The attitude was, 'We're going to grow and grow but making sure our infrastructure can support it isn't a priority.' They have their own clunky infrastructure to do something that you don't really need a clunky infrastructure to do anymore."
The problem with last minute engineering (Score:2)
Is that the last minute is either "next year" or "last week" depending on which side of the disaster you're on.
Re: (Score:2)
That number may, however, have dropped with the second crash.
Like Apple Messenger? (Score:2)
Isn't iOS 5 going to feature a very similar "clunky infrastructure" feature?
Re: (Score:2)
Yes, but I believe you can fallback to SMS with iOS5.
Re: (Score:2)
Yes, but I believe you can fallback to SMS with iOS5.
From an iPod?
iMessage works across iOS devices (iPod, iPhone, iPad) and does not require a cellular plan, so on a large number of devices, there is no SMS to fall back on.
Re: (Score:2)
No. The Message app uses the iMessage protocol if available. If not available, it falls back to SMS/MMS. It's completely transparent/unified. The only indicator which message transport is being used is the color of the send button or conversation bubble. My first reaction was similar (how long until we see an iCloud outage), but for this example I don't think Apple has anything to worry about.
Re:Like Apple Messenger? (Score:4, Informative)
If they had simply used DNS names, they could fail-over on many levels simply by re-pointing DNS.
Maybe they were worried about DNS poisoning or something like that. Still should be a way to remotely update those "service book" addresses though.
Re: (Score:2)
As far as I know, they can push new books to your device so they are probably able to update the existing ones.
Every time you configure a new BES e-mail account, they push a new service book to you.
They probably send you new books every time your device cold-starts (battery pull). I don't know it for sure, but most BB forums recommend fixing missing/incorrect books by doing a battery pull. Of course it could be the usual "did you try turning it off and on again?" voodoo.
Re: (Score:3)
No, it doesn't mean any such thing. I take it you've never heard of CARP [wikipedia.org] or VRRP. [wikipedia.org]
Re: (Score:2)
No, it doesn't mean any such thing. I take it you've never heard of CARP [wikipedia.org] or VRRP. [wikipedia.org]
Or even something as common as the decades-old BGP.
Re: (Score:3)
> if you look in the "service books" of a Blackberry device, you will find a lot of hard-coded IP addresses. This means that RIM is pointing all devices to a single server in a single data center on the end of a single circuit owned by a single provider.
Following this "logic", by using DNS it would have been possible to do what, change the single server in that single data center to another single server in another single data center? Or is it only when you use DNS that it becomes possible to do load-bal
Re: (Score:2)
Yea, one IP does not mean one server.
I work in a DC and we regularly deploy server clusters with virtual IPs. Customers can setup a LBer and any number of servers they want behind it.
Another incorrect Apple Hater (Score:2)
Forgetting that iOS 5 is all cloud-based,
It's not cloud based. It is, to put it in manager-speak "cloud enabled". The technical point behind the distinction is that iOS5 merely always for the cloud to provide a device with MORE data which it then stores locally. It's not cloud based at all, as content and media is locally based but then distributed.
that great new feature on the 4S, the voice assistant, is ALSO all cloud-based.
Really more service based, but yes that requires a data connection. But you ca
Re: (Score:2)
Forgetting that iOS 5 is all cloud-based,
It's not cloud based.
I think it's pretty clear what he meant, that most of the new iOS5 features require access to Apple's servers, so if those servers go down or you can't access them for whatever reason then those features simply will not work, those features are based on 'cloud' technologies and won't work without them.
that great new feature on the 4S, the voice assistant, is ALSO all cloud-based.
Really more service based, but yes that requires a data connection.
No, it's not just a data connection, you need a data connection that can access Apple's 'cloud', so if for whatever reason (routing issue, server outage, etc...) you cannot access Apple's servers you cannot us
Re: (Score:2)
I think it's pretty clear what he meant, that most of the new iOS5 features require access to Apple's servers
But not CONSTANT access. Remember the context, BBM going down - a momentary thing, NOT permanent. Anyone using iCloud would only be inconvenienced, not blocked from doing anything like playing music or whatever, or even syncing from a home computer - same for Siri, you could just make a calendar entry yourself.
Re: (Score:2)
But not CONSTANT access. Remember the context, BBM going down - a momentary thing, NOT permanent.
Whether it's constant or intermittent makes no difference, if access to those servers isn't available when you need it then you can't use those features.
Irrelevant if they do not block activity (Score:2)
Whether it's constant or intermittent makes no difference, if access to those servers isn't available when you need it then you can't use those features.
If Siri is down you just use google or make an appointment yourself.
If iCloud is down you just listen to the music already synced.
The operation of the device does not RELY on them being up any one moment. The benefit that comes from iCloud would be just as good even if the servers operated only 50% of the time.
The point is that Apple is relying much less
Re: (Score:2)
Look, it's AC the incredible Hater Wanker! COme back to embarrass yourself even further? Lets take a peek!!
All the new iOS 5 features require the cloud.
Like notifications or the newer lock screen camera?
FAIL
If Apple's servers go down, you lose all those spiffy new features that Android has had forever.
Same goes for Android. FAIL. Also you may want to look up the definition for "FOREVER" which extends rather a lot more than a few years in the past.
Except Google is kind of good at the whole cloud thing,
Re: (Score:2)
Mostly what comes to mind isn't a cloud crash, but the fact that it took Apple something like 3 attempts to write a fucking alarm clock that worked.
Re: (Score:2)
As a Nokia user, I wouldn't say it works yet: it still doesn't activate if the phone is powered off. Apparently having an RTC chip with alarm is something extraordinary.
Re: (Score:2)
Shhh... That's supposed to be the "one more thing" at the iPhone 5 launch. Don't spoil it!
Re: (Score:2)
Shhh... That's supposed to be the "one more thing" at the iPhone 5 launch. Don't spoil it!
We don't have to worry about stealing Steve Job's thunder anymore,
Re: (Score:2)
Re: (Score:2)
> Why would I want it to work if I powered the phone is off?
Because you still have to go to work, even if you turn off your phone to get a good night of sleep, or if you battery is dead? My 7 year old Sony works that way, and I bet most other phones. If you don't want to get an alarm, don't program an alarm :-)
Re: (Score:2)
Because it's stupid to be wasting all the energy necessary to keep the main CPU and other components running the OS when you only need the alarm. This is specially important when you for some reason can't charge your phone.
If you don't want the alarm clock to work, just don't set it.
Re: (Score:2)
It's very useful when you run out of battery to fully run the device. The alarm will still sound in a reduced environment. That's saved me a couple of times.
Re: (Score:2)
Just because YOU don't want it and don't understand it, it doesn't mean other people won't.
I have one clock only, my cell phone. I want an alarm clock and I want it to work when I turn off the phone (which I do every night!).
You think I'm stupid? I can live with that...
Re: (Score:2)
Name one apple product that has crashed once a week or once a month. Just one.
itunes
piece of crap
Re: (Score:2)
Name one apple product that has crashed once a week or once a month. Just one.
Quicktime
piece of crap
Re: (Score:2)
You can NOT compare Voice Actions/Vlingo and Siri.
That Google shit has no AI and is very limited. Voice Actions have existed for iOS since 3.0 but that's not what Siri is about.
I'm not talking about built-in voice commands. (Which have existed on every mobile phone I've ever owned for the past decade - the iPhone seriously didn't get that for three versions? I knew Apple was behind the curve, but I didn't realize they were THAT far behind. Then again, it took them that long to get copy and paste, didn't it?)
I'm talking about third party apps that provide features similar to those Siri does. For example, there was an app released a year ago that does the same things. It was called "Siri."
That's because Apple didn't create Siri. Siri had been available on the App store since 2010, but was removed with the iPhone 4S announcement. (Which means that people with existing iPhones - which support Siri just fine - are having it removed by Apple to force them to upgrade to a new phone whose only other new feature is a slightly better camera.)
The keyword is INTEGRATION. And polish.
You didn't watch the Siri demo, did you? It doesn't integrate with anything, it's its own separate app. It launches with a UI that looks like nothing else on the phone, and does its own thing. That's the complete opposite of "integration" and the only thing "polished" about it is the shine on the little microphone icon.
Not to mention that apps on Android can easily integrate with other apps - even apps developed after they were. Apps on the iPhone cannot - Apple not only doesn't provide any API to do so, they also forbid anyone from developing one on their own.
Evidently, you did not watch the demo either or you are confusing the Siri app which apple bought out with the Siri feature in iOS 5 build for iPhone 4S. The latter is integrated into the look and feel of iOS 5. It can take over the entire screen but first launches in s similar manner to how the app switcher does taking up the lower portion of the screen with the icons greyed out when you hold down the home button.
Re: (Score:2)
I had cut and paste on my XV6800 WinMo 6.0 phone back in like 2007, via a touch screen.
It's okay if Apple didn't invent everything, you know. They're still cool if you think they are.
Re: (Score:2)
And how exactly did you cut and paste on it? Using the buttons, or using only the touch screen?
Re: (Score:2)
Nevermind, I found the answer. Yes, once you upgraded to 6.1.
Re: (Score:2)
A long press on a selection was the equivalent of a right-click on a Windows machine. Select text, long press, select copy, place cursor in new location, long press, select paste.
The phone used a stylus, but it still used a touch screen, not the buttons (although there were buttons to open the contextual menu, too, which was nice). It technically worked with a finger, too, but I have big fat fingers so I stuck with the stylus for precision.
Re: (Score:2)
exactly who implemented copy and paste on a touch screen before Apple...
And could you cut and paste using only the touch screen on WinCE?
Re: (Score:2)
Here, let me answer for you. You couldn't in WinCE 3.0, or 4.0, nor in Windows Mobile 5.0, or 6.0.
Windows Mobile didn't support copy & paste via a touchscreen until v6.1, which was announced on April 1, 2008, and first appeared on phones in June 2008. Nowhere near your date of 2000
So, I stand corrected, MS did implement it on a touchscreen before Apple. One year before. WinMo 6.1 came out right before iOS 2 shipped with the iPhone 2. iOS 3 didn't ship until June 2009 with cut/copy/paste.
Re: (Score:2)
Re: (Score:2)
No, just BIS. From the article:
Many corporate customers said they had not lost service, suggesting that the problem was with Blackberry's BIS consumer systems, rather than its BES enterprise systems.
What? (Score:5, Insightful)
Okay, why does a disgruntled ex-employee's rant about scalability and infrastructure come into play before we know that scalability or infrastructure was the cause of the break? Seriously, maybe the taco bell dog just chewed through fiber lines in NY and LA while on tour. Could happen to anybody.
Re: (Score:2)
Given the way that RIM has been running itself out of business, I would be surprised if there weren't something to those assertions. It's definitely possible that it's a disgruntled ex-employee looking to bad mouth his former employers, but by the same token, RIM isn't exactly known for having competent management so a suggestion list this is at least plausible.
Re: (Score:2)
Depends on what your definition of competent management. They were very competent, but at maximizing their short-term share price.
Choosing to not properly build your infrastructure and then crushing under the weight of 8M devices is a MUCH better decision than building the proper infrastructure and then watching your competitor get crushed under the weight of 8M devices while you go bankrupt because they beat you to market by two months.
The managers who made the decision to skimp sold their stock ages ago
RIM is dead... (Score:3)
... they just don't know it yet. We have 40 Blackberry's in our company, but we purchased our last one about six months ago. I hope BES dies a painful, painful death.
Android, here we come.
Re: (Score:2)
... they just don't know it yet.
So, you're saying they've been zombied.
That would explain a lot actually.
Re:RIM is dead... (Score:5, Interesting)
Re:RIM is dead... (Score:4, Insightful)
You need 'Enterprise' tools to manage 40 devices ?!? Methinks (and from experience) that you will spend more time and money 'managing' the 'Enterprise' tool than to manage the 40 devices directly.
Re: (Score:2)
Re: (Score:2)
I doubt you have a 'corporation' when you have 40 devices. That probably mean you have a 100 employees company/startup where the overhead of fine tune management of devices is not worth it. Once you reach a couple hundred accounts, then yes maybe, at the level above it just gets in the way.
Re: (Score:2)
You can have a corporation of 1, at least in Canada. But that's beside the point....
A company I recently worked for got burned badly by just ONE corporate user making unlimited overseas long distance calls. If you've got 40 devices out there, you can pretty much bet several of your employees will abuse them. 'Tis simple human nature.
Aside from that, how hard can it be to go into an admin console and configure the limits on a user's device if the tools to do that are provided?
Re: (Score:2)
It takes a whopping 1.5 hours to deploy BES Express. I bet you I could have the server deployed and all of the blackberries configured before you got through configuring your andriods.
Re: (Score:3)
A quick google of 'Android Enterprise Management' returned some potential solutions:
Zenprise [zenprise.com]
Good for Enterprise [good.com]
3LM [pcworld.com]
Those were the top three. I'm sure there's more, including RIM's own Android management solution.
Re: (Score:2)
I saw those, but "being available" isnt the same as "widely known and trusted". They might be solid, and are worth looking at, but something you found on google 5 minutes ago hardly compares to a product thats been out for around a decade and has had its security vetted by the masses.
Re: (Score:2)
how are you going to manage 40 Android devices?
'roid is open source! You write your own mis-management tools. Duh.
Re: (Score:2)
Re: (Score:2)
iOS has the same problem, I manage 40 iPads and will be getting 26 more iPod Touches in that mix, they are a pain in the ass to manage, image and deploy.
Care to expand a little on your frustrations? Sounds more like the task was dumped on you, on top of other previous responsibilities, and because it takes time and you would rather be doing something else, "they are a pain in the ass to manage, image and deploy."
With the right console and software tools, and given the time needed, these devices would seem a dream to other administrators: once deployed, they go out... and never come back. Just how often are you having to support bone fide technical issues,
Re: (Score:2)
Is it even possible to 'image' an iPad?
I like mine and get they're virtually maintenance free, but I can also see where a 'standard' iPad with a set of 5-6 third party apps would be a colossal headache to manage centrally if you had more than a dozen or so devices. If you were talking 100 or more? It'd be a full time job keeping them in iOS updates. I can't imagine what a "new school year" type imaging program would look like.
Re: (Score:2)
Re: (Score:2)
Well, there are a couple of options for handling modern consumer hardware like iPads and Android phones (I'm generalizing and not intending to poke at either in particular):
1. Make it the employee's problem. If employee gets a virus or leaks important info, fire them and say you've solved the problem. This is the cheapest solution if everybody lets you get away with it (which is surprisingly often).
2. Make it the vendor's problem. Look at the feature list and see that it has no viruses advertised and i
Single point of failure... (Score:2)
Thats the problem with blackberry devices depending on RIM for service...
Your mobile operator could fail, but theres more than one operator...
Your own email server could fail, but your in control of this yourself and can take steps to fix it... Plus, it only affects you and not anyone else, you have a choice of email providers and if you run the server yourself its your own fault if it fails.
You are stuck with RIM service if you want a blackberry handset, you don't have a choice unless you switch to a diffe
Re:Single point of failure... (Score:4, Informative)
You are stuck with RIM service if you want a blackberry handset, you don't have a choice unless you switch to a different type of handset.
Complete nonsense. Even without RIM service, it still functions as a phone, internet access, and SMS text messaging. Basically everything you use a smartphone for. You just dont have access to the secure RIM enterprise data and messaging services. Neither does any non BB device, so basically the BB just downgrades to a standard smartphone.
Re: (Score:2)
Re: (Score:2)
Failures happen - plan for it (Score:2)
The problem is not that system failures happen, but that businesses and people don't plan to deal with those failures.
Running your own email server will not prevent crashes. RIM crashes, Google crashes, a bazillion corporate email servers crash.
The trick is not to expect anyone to really deliver 24x7x365 uptime, because no one has ever actually done so. The closest they've come is playing word games with the service contracts and reason for outages so that they can still claim 5-nines uptime, even th
It's as if millions cried out (Score:2)
Re: (Score:3)
...and the Babylon 5 fans come out of the woodwork. As if BlackBerry fans weren't annoying enough to begin with!
Re:It's as if millions cried out (Score:5, Funny)
Re: (Score:2)
Oh right Deep Space 9...my mistake. Truly one of the more moving lines ever uttered by Commander Cisco.
Re: (Score:2)
Re: (Score:2)
Think of it as a honeypot for nerdrage and you won't be far off the mark.
I was curious after I saw somebody else do it a month or two ago (misattributed science fiction quotes). I found it humorous. Mine wasn't as inspired...
http://entertainment.slashdot.org/comments.pl?sid=2459630&cid=37602748 [slashdot.org]
Re: (Score:2)
Banylon 5???! I think you just lost your right to be on Slashdot.
I think he meant Deep Space Nine
The Lost Saucer [wikipedia.org] most likely.
Re: (Score:2)
Re: (Score:2)
cmon dude, it was obi wan in the original Star Wars
http://www.imdb.com/title/tt0076759/quotes?qt=qt0440735 [imdb.com]
Re: (Score:2)
It's as if millions of souls cried out ... and were suddenly silenced.
Blackberry users have no souls. They're all PHBs.
Re: (Score:2)
Blackberry users have no souls. They're all PHBs.
Some of them do actual work, actually. Thats why we get a device with a physical keyboard, a full-days batterylife, and awesome messaging suite, rather than aiming for the GPU performance and whether or not theres an "app for that" yet.
Yes, Im being disparaging of Android (even as I secretly long for a Galaxy S 2-- if only they had a physical keyboard....).
Future Thinking (Score:4)
Re: (Score:2)
In four years I'll be starting a company based on the idea of having a device that stores all your photos, emails, and applications locally so you aren't tied to the cloud.
You're a little late. You have to hand it to Apple yet again, their new iCloud service is mainly used not to _store_ but to _sync_ your apps, music, photos, contacts, calendar events and documents between your devices, where almost everything is then stored locally on each device or computer. Gee whiz, how about that. What a concept, huh? An iCloud outage will just mean you temporarily won't be able to sync things between devices. Most people probably won't even notice if it doesn't last more than a day.
I d
Which you can also do (Score:2)
An even better idea would be to be able to sync things WITHOUT a damn iCloud at all! What a concept!
But you can ALSO do that. Without iCloud you are still perfectly free to sync media and applications onto the device, even if all you have is a computer and an iOS device and no network.
Why is having layers of possibilities not better than the alternative?
Re: (Score:2)
In four years I'll be starting a company based on the idea of having a device that stores all your photos, emails, and applications locally so you aren't tied to the cloud.
Way to completely not understand how BES works. None of your stuff is stored in RIM's "cloud", whether you use BIS, BES, or neither. All BES and BIS do are provide push functionality. BES can go completely down, and my access to previous email is still there, as is my access to my gmail. My photos and apps, notably, are stored locally.
Leave it to RIM. . . (Score:2)
. . . to ruin my chances of snagging a big, beautiful man.
Users fault (Score:2)
> have been unable to receive or send emails and messages through their phones
Maybe the users just don't hold their phones correctly, like it happened with another vendor...
Campaign (Score:3)
Re: (Score:3)
I bet RIM still have a few cards left up the sleeve.
Unfortunately for those who like RIM, the cards up their sleeves are the instruction cards for playing poker.
Re: (Score:2)
As long as they stop buying into the "business phones need to be multimedia whizzes" bullshit, theyll be fine. People who want touch phones arent going to want a blackberry, and those of us who want blackberries dont want fast CPUs, games, and a touch screen. All we want is a messaging device that works and is reliable.
Seriously, Im kind of pissed that I upgraded to BES os 6.0, and it was optimized for touch, but was less usable and less stable. If they can just focus on what theyre GOOD at-- BES, great
Well, at least Slashdot is consistent... (Score:5, Insightful)
Wow, another Slashdot hatchet job on RIM. Heavy into Apple stock, are we?
RIM certainly has issues, and it may not survive. But it seems the Slashdot editorial staff wants to make sure.
Someone is consistent; not Slashdot (Score:2)
Wow, another Slashdot hatchet job on RIM.
In Slashdot's defense, there were already several hatchets in place by the time they got there, and they were pretty obviously self-hatcheted. Slashdot is just pointing out the obvious.
When you have a central point of failure you WILL HAVE FAILURES. It's like saying the sun will rise. I've never liked Blackberries for that reason long before any obvious decline began; I greatly prefer systems that lack such points of failure.
OMG! An email provider is down! (Score:2)
Even when it's Google or RIM, an email server being unavailable isn't what I'd call "news".
Re: (Score:3)
If Blackerry was still popular.
Or if it happened to BlackBerry....with their 70M subscribers.
Re: (Score:3)
And, you know, SQL databases, too, since they scale very well.
BTW, why do you jump to the datastore as the scalability problem? It's not mentioned in the article. For all we know, it could be their internal Cisco routers that are overloaded or their WAN links or their 10-year-old Windows 2000 servers or something else.
Re:NoSQL, Baby! (Score:4, Funny)
He's jumping to the datastore because he can't cunningly fit in a link to his presentation about NoSQL database software if he doesn't.
Re: (Score:2)
Jeez everybody is so suspicious these days. You know, there are some people who don't have to monitize everything they touch. For that matter, all of the NoSQL databases I mentioned are free--nobody is making any coin off of them. My link was harmless.
Re: (Score:2)
Oh, chill, dude. Someone who saw the sarcasm tags rated it a +1, Funny.
Best of luck to you on your presentation.
Re: (Score:2)
The whole point of datastores is scalability.
Re: (Score:2)
In case you haven't heard, relational databases have a dickens of a time scaling horizontally.
The whole point of datastores is scalability.
oh, you are THAT guy ...
http://www.youtube.com/watch?v=b2F-DItXtZs [youtube.com]
Re: (Score:2)
Re: (Score:2)
That takes a lot of IBM printer paper. And the SaturnV supplied its own lift. Nice architectural separation.
Re: (Score:2)
Re: (Score:2)
P-u-s-s-y!
Your statement was one of the most craven I've ever seen expressed on Slashdot.
Slashdot is not beanbag, pal. It's combat.
Maybe you should go here, where you feel less threatened [go.com]
Re: (Score:2)
Eventual consistency is actually known as Tuneable consistency. You pick how fast you want your data replicated. There is no free lunch.
You are referring to CouchDB methinks. Bit player...
No, my presentation will point out how Facebook to this day uses Cassandra for inbox search. Just because you chose a particular datastore does not mean your architect can check her brain at the door.
Relatio
Re: (Score:2)
Finally, the bit about Larry Ellision? That's called humor. You know, to laugh? Ha ha? If you don't find it funny
Re: (Score:2)
Re: (Score:2)
While all of these solutions are scalable they have terrible latency; especially when nodes are failing. RIM probably needs very low latency for it applications.
When the endpoint of its applications is either receiving traffic from, or sending traffic to, a small underpowered device typically at the end of a cellular network connection, I'd have to say that the latency of its inter-datacenter connectivity for its redundant backend data storage should be the least of its concerns.