USPS Server Meltdown 238
m2pc writes "The US Postal Service is experiencing major server issues for its shipping API web services. After spending about an hour debugging my own eCommerce software for a client, I found the problem was with the USPS shipping servers being unavailable. Further research showed that message boards for OS Commerce and other e-Commerce packages are filling with posts from angry users who are experiencing crashing Web store applications and frustrated customers. Developers are scrambling to find interim solutions, from hard-coding fixed price shipping, to 'rolling their own' shipping calculation APIs based on the USPS Fixed Rate Zone Tables, to disabling the USPS option altogether. One user reported yesterday that a call to USPS yielded the response 'we expect it to be down all day.' As of 9:20 AM PST the service is still unavailable."
Why SAS fill eventually fail (Score:2, Interesting)
Lots of free shipping today. (Score:3, Interesting)
Try it. Go to a USofA-based commercial site and check the shipping charges for your purchase.
Most of the sites are offering free shipping. I'm guessing that these two items are related.
UPS is flaky too (Score:4, Interesting)
Re:Crashing Web store applications? (Score:5, Interesting)
Only a really terrible developer would hit a web services API and not code for it to fail. No one should expect a third party service to be up 100%. The apps should fail gracefully. Anyone finding their e-commerce software handling this situation poorly should find another package.
I have developed a fulfillment system using UPS so I have some experience with this. This was a few years ago, but at the time, UPS's system was designed on the assumption that a shopping cart system would contact UPS during the order placement process (or checkout). USPS appears to be very similar. This is incredibly moronic, because it means that if their server doesn't respond or any reason, you simply lose the sale. Problems AFTER the order has been accepted are far less serious - that just means fulfillment is interrupted - you don't lose orders. It would be even better if you could print the labels without having to reach their server, but they don't provide this (UPS's own software, however, can.)
The only reasonable way to do it (obviously?) is to have your own shopping cart software calculate the shipping rate. At the time I did this in 2001, UPS did not offer reasonable data to let you do this. Their rating system is extremely complicated, and the spreadsheets they gave me had clearly been maintained by hand and needed a lot of massage to get them into a reasonably parseable format. Not only that but it was very difficult to get these tables in the first place, requiring multiple escalations through tech reps and ultimately a "god damn it just give him the tables" from our sales rep.
I imagine things might have improved a little since then and I know there are shopping carts that can now do the rate calculations internally. But if any shipping company forces its customers to contact a server to calculate shipping costs, then they are just as retarded as the merchants who use them (for a web checkout).
I know why (Score:2, Interesting)
posting anonymously to cover my ass... We upgraded to MySQL 5.1 last week. We had some major table corruption and no backups. Sadly, no one will be fired or even reprimanded for any of this major cluster fuck.
Re:What can Brown do for you? (Score:1, Interesting)
UPS is slow as hell though. Plus they have delivered to the wrong house (sometimes not even on the same street) enough time to make me leery. I was sad when Newegg switched to UPS instead of FedEx for their saver shipping.
USPS is much faster than UPS (pretty much any service is faster than UPS) but they piss me off by often not even delivering the package and making me go get it at the post office because the mail dude is too lazy to stop of get out of his truck (once the guy drove across my lawn, making huge ruts, just so he could toss out the package at my front door without getting out).
FedEx and DHL have given me the best service and cheaper than UPS.
Re:Sounds like fun. (Score:5, Interesting)
USPS has IT people? Oiy veih I can imagine that job.
A few years ago, I used to be one of them...
Oy Vey, indeed...
Re:Crashing Web store applications? (Score:2, Interesting)
hosted vs downloaded/licensed e-commerce (Score:5, Interesting)
The company I work for provides a hosted e-commerce shopping cart solution, SEO-Cart, which supports the USPS Web Tools. Of course the first call coming in for the day was from a client using USPS and having incorrect shipping prices being calculated for their store.
I went ahead and called USPS and the lady who answered was quite rude and explained to me that they had a Worldwide outage which affected other applications than just their Webtools API, and also that they hire a 3rd party company to handle their Webtools API software. She couldn't provide any other information at all and I told her a company of that size should have some sort of fail over plan in place to prevent them from being down as long as they have been. I was really disappointed in the fact she didn't even ask me for my name, phone number, or company by time the conversation was over, but she was probably being bombarded with phone calls all day.
After figuring out that USPS was completely down, I looked through our fail over code and found the following equations seem to come close to the USPS pricing:
National shipping: [cart-weight]*1.6+3.00
International shipping: [cart-weight]*1.6+15.00
These also include pricing for insurance.
After tweaking the fail over pricing code to this, it seemed that everyone using USPS were happy with the results. We also had to decrease the connection timeout set for the request to the USPS Webtools API which was also slowing things down.
The Webtools API seems to be both up and down today, with some orders having shipping prices directly from USPS and others having the fallback pricing. Either way, hopefully their IT department learns from this and also provide us information as to what exactly went wrong.
On that note, this is a prime example that I use when speaking to prospects about the advantages of using a hosted shopping cart solution rather than a licensed/free download solution. Besides the obvious IT benefits that you get with a good hosted shopping cart solution, hosted shopping cart software is typically a centralized application that can provide quick updates to problems like these. Of course this is assuming that the prospect is serious about their online store and doesn't want to handle technical support themselves.
Re:Crashing Web store applications? (Score:3, Interesting)
Most e-commerce sites, including the ones I've built, have a multi-step process to place an order. If the step for shipping (or tax, etc.) failed, then the system would re-load the page with any appropriate messages and option changes. So, for example, if the third-party payment processing service didn't respond before a timeout the user would be asked to call customer service to complete their order. The sales rep could see the order in its incomplete state and finish it over the phone.
USPS (Score:2, Interesting)
"Neither snow nor rain nor heat nor gloom of night stays these couriers from the swift completion of their appointed rounds"
But the computers will!!!
You do realize that this has never been the motto or creed of the USPS.
It is taken from the courier service of the Persian Empire. See Wikipedia.
--
This is not the sig you are looking for... Move along...
USPS rate API SUCKS (Score:5, Interesting)
Apart from the trouble reported in this /. article which I found occuring on one of the existing sites I wrote yesterday (simply because there were no USPS prices being returned, no error, but took about 30 seconds to work out what happened), USPS simply sucks ass.
Here's why:
Some time ago, they had an API to get rates, it was called RateV2.
Then they "updated", and now have RateV3.
RateV3 is the only specification published.
To get access to the Rate API servers, you must first test your implementation against thier testing servers successfully, when they see that they let you on the production server....
Thier testing servers only work with a limited version of RateV2.
So, in order to use the USPS API, you must:
Write to the now unpublished RateV2.
Test that RateV2 on the test servers.
Ask USPS to allow you to use Production (and get the keys etc) because you have successfully tested.
Write completely new code against RateV3.
Test that RateV3 on the production servers.
And if you try and show the USPS staff the logical problem in this process, they will reply "I can not put you on production servers, until you have done three successful tests on the test servers".
Re:What would USPS do? (Score:3, Interesting)
You laugh, but older contract offices still get their software updates, including price changes, via mail.
On 3.5" floppy.
The terminals still use green-text monitors.
Re:That's what you get.... (Score:1, Interesting)
It doesn't matter either way. My company hosts a USPS-owned server ON SITE which we use for a specific task periodically all day long. Not the same as the the shipping API that's busted but it's somewhat similar in purpose.
Today, our USPS server has had issues. It's down due to a nasty software error. There have been many tears shed today over this and many more to come. It's a rather important little server.
Now, the fact that we host the box ourselves and I can go in our server room and see it and touch it and whatever makes absolutely no difference whatsoever to the problem or the solution.
Had it been hosted by the USPS, that might actually have been better for us because the software problem might not have happened. Dunno.
But I do know, having hardware ON SITE is not helping us at the moment and it's not the perfect solution.
Look at it this way, suppose an error like ours happened to multiple sites -and it could: the USPS releases new software for this box every so often and everybody has to apply it. Suppose one of the updates went bad. Then there would be multiple places that needed a fix all at once.
Whereas with the API problem of today, apparently the issue is centralized and once they fix it, everybody is fixed.
Sure everybody hurts in the meantime but there's only ONE box to fix. The time and resources can be focused on that one problem.