Slashdot Log In
USPS Server Meltdown
Posted by
kdawson
on Tue Dec 09, 2008 03:23 PM
from the neither-snow-nor-rain-nor-storm-of-net dept.
from the neither-snow-nor-rain-nor-storm-of-net dept.
m2pc writes "The US Postal Service is experiencing major server issues for its shipping API web services. After spending about an hour debugging my own eCommerce software for a client, I found the problem was with the USPS shipping servers being unavailable. Further research showed that message boards for OS Commerce and other e-Commerce packages are filling with posts from angry users who are experiencing crashing Web store applications and frustrated customers. Developers are scrambling to find interim solutions, from hard-coding fixed price shipping, to 'rolling their own' shipping calculation APIs based on the USPS Fixed Rate Zone Tables, to disabling the USPS option altogether. One user reported yesterday that a call to USPS yielded the response 'we expect it to be down all day.' As of 9:20 AM PST the service is still unavailable."
Related Stories
This discussion has been archived.
No new comments can be posted.
The Fine Print: The following comments are owned by whoever posted them. We are not responsible for them in any way.
Bailout? (Score:5, Funny)
Re: (Score:3, Funny)
That's what you get.... (Score:5, Insightful)
And that's what you get for writing e-commerce packages that rely on 3rd party sites for basic functionality...
Don't say I didn't tell you so...
Re:That's what you get.... (Score:5, Insightful)
Parent
Re:That's what you get.... (Score:5, Insightful)
Parent
Re: (Score:3, Informative)
Re:That's what you get.... (Score:5, Insightful)
"While in principle I agree with that, what are they supposed to do? They are quoting you a price for a service they don't provide themselves."
Fail gracefully. For example, state that they are unable to price that shipping option at that time. Offer you to accept the option without knowing the price, select another type of shipping, or get an email when they know.
Designing software to gracefully handle external failures is a vital real-world programming skill.
Parent
Re: (Score:3, Insightful)
Wouldn't it be easier to have a background task update the USPS prices, and just use the last working snapshot?
Re:That's what you get.... (Score:4, Informative)
Really, you'd think someone in the apparently non-existent risk management departments would ask "what if the USPS Servers went down?" Hopefully the answer wasn't simply "we're screwed."
In any third party connection for an application I've built, I typically consider the implications of such web service/outside API going down, even if I label it "non vital, don't worry about it" to "have a backup plan". Oh well, no sympathies here
Parent
Re:That's what you get... for not using FedEx (Score:5, Insightful)
To be fair, what sort of "backup" calculation would you have done here, short of reverse-engineering the USPS algorithm for calculating shipping rates?
I'm not usually a rabid free-market libertarian, but this here can be seen as a result of the fact that the USPS isn't really beholden to its customers. Can you imagine FedEx or UPS being afflicted by such an issue? And, if they were, would they blow off inquiries with a glib, "We expect the servers to be down for the rest of the day?" Of course not, because, for FedEx, UPS, DHL, et. al. such an outage directly affects the health of the organization. If people can't calculate shipping rates, they can't ship, and if they don't ship, the company doesn't make money. The close linkage between revenue and working services tends to put more impetus behind keeping things working and making sure that they get fixed quickly if they do happen to go down.
Parent
Re:That's what you get... for not using FedEx (Score:4, Informative)
The same has been true with the USPS since 1970. Their entire budget is financed by people buying stamps and other services. They don't get a dime of taxpayer money.
Parent
Re: (Score:3, Informative)
The same has been true with the USPS since 1970. Their entire budget is financed by people buying stamps and other services. They don't get a dime of taxpayer money.
How did you ever get modded +5 informative with that load of bull?
The total funding for the Postal Service in the administration's 2005 budget is just over $61.7 million. [upi.com]
Appropriation, fiscal year 2004 $65,135,000 [congress.gov]
Appropriations, 1999 $100,195,000 [congress.gov]
etc, etc
That's what you get... for not using Utilities. (Score:3, Insightful)
"Americans are pathetic sometimes -- they expect their government services to do as well as private industry, yet they don't give them the ability to charge what private industry charges. Amtrak is a similar situation, Amtrak is expected to be cash flow positive, yet they are not allowed to own their own tracks, those are owned by the freight companies, whereas their main competitors run on highways that are paid for completely by the taxpayer and gas taxes, or operate out of airports also funded by taxpaye
Re:That's what you get... for not using FedEx (Score:4, Insightful)
Parent
Re:That's what you get... for not using FedEx (Score:4, Informative)
Parent
Re:That's what you get... for not using FedEx (Score:4, Informative)
The zones are clearly marked but determining which zone you are shipping to is not. They are based on distance from the originating ZIP Code so the application must use Postal Explorer to generate the zone chart ahead of time. (For now, it happens to be working.) And in the case of non-flat-rate Priority Mail, there's a huge price difference between zone 4 (zone and weight only) and zone 5 (zone and weight *OR* zone and size).
Parent
Re: (Score:3, Insightful)
try {
}
catch (exception ex){
}
Re:That's what you get.... (Score:4, Insightful)
And that's what you get for writing e-commerce packages that rely on 3rd party sites for basic functionality...
Like a credit card processor? How many web stores have more than one?
Parent
Re:That's what you get.... (Score:5, Funny)
that's why i design all my e-commerce sites to accept cash only. (always makes sure the bills are facing up before you feed them into the floppy drive!)
Parent
Lots of free shipping today. (Score:3, Interesting)
Try it. Go to a USofA-based commercial site and check the shipping charges for your purchase.
Most of the sites are offering free shipping. I'm guessing that these two items are related.
Proof that computers corrupt all!!!! (Score:4, Funny)
"Neither snow nor rain nor heat nor gloom of night stays these couriers from the swift completion of their appointed rounds"
But the computers will!!!
Re:USPS (Score:5, Informative)
Sure, but it would have been easier if you had provided a link at least.
The motto will also continue to be mistaken for the motto of the USPS so long as the USPS does nothing to correct that.
The motto is prominently carved over the U.S. General Post Office in NYC [wikipedia.org]
and the USPS even used it in their own television spot [wikipedia.org] (albeit in an altered form):
Note that the [2] link in the above quoted Wikipedia article takes you to this page [usps.com] on the USPS site quoting the exact text of the tvspot.
Yes, they have no Motto, but considering how much they use the meme, they might as well have one.
Parent
What can Brown do for you? (Score:5, Funny)
Re:What can Brown do for you? (Score:5, Informative)
Parent
Crashing Web store applications? (Score:5, Insightful)
Only a really terrible developer would hit a web services API and not code for it to fail. No one should expect a third party service to be up 100%. The apps should fail gracefully. Anyone finding their e-commerce software handling this situation poorly should find another package.
If a store offers only the USPS delivery method and the web service is down, the user could be directed to call the sales number to place their order. If the store offers other deliver methods the store front could instruct the user that USPS isn't currently available and they must choose another method.
Re:Crashing Web store applications? (Score:5, Informative)
Parent
Re: (Score:3, Informative)
Re:Crashing Web store applications? (Score:5, Insightful)
osC is insidious. You install it because it's open source and a zillion other sites are using it, so it can't be THAT bad, right? Then you have to install a few contributions to add the basic functions you need - and with each contribution, adding the next gets harder and harder.
But you get up and running. Then you discover all of the little annoying usability quirks - US state names don't use proper abbreviations, customers in Ireland can't place an order without a post code even if you set the minimum length to zero, the payment option selection looks like it has a default selected when it doesn't, users don't know that 'authorize.net' is actually the credit card payment option, and so on.
So you spend hours fixing those. Meanwhile, you've got the database populated with all of your products, and you're using it to track sales figures. Maybe you add some back-end order processing stuff for your own convenience.
Eventually this mass of haphazardly patched spaghetti code becomes an absolute nightmare to do ANYTHING with, but by this point you've spent so many hours working on it that it's easier to just keep slogging away at it than to just abandon the whole thing, install something new, and try to move your catalog and your sales records over to the new system.
And THAT is why there are so many osCommerce sites.
Parent
Re:Crashing Web store applications? (Score:5, Interesting)
Only a really terrible developer would hit a web services API and not code for it to fail. No one should expect a third party service to be up 100%. The apps should fail gracefully. Anyone finding their e-commerce software handling this situation poorly should find another package.
I have developed a fulfillment system using UPS so I have some experience with this. This was a few years ago, but at the time, UPS's system was designed on the assumption that a shopping cart system would contact UPS during the order placement process (or checkout). USPS appears to be very similar. This is incredibly moronic, because it means that if their server doesn't respond or any reason, you simply lose the sale. Problems AFTER the order has been accepted are far less serious - that just means fulfillment is interrupted - you don't lose orders. It would be even better if you could print the labels without having to reach their server, but they don't provide this (UPS's own software, however, can.)
The only reasonable way to do it (obviously?) is to have your own shopping cart software calculate the shipping rate. At the time I did this in 2001, UPS did not offer reasonable data to let you do this. Their rating system is extremely complicated, and the spreadsheets they gave me had clearly been maintained by hand and needed a lot of massage to get them into a reasonably parseable format. Not only that but it was very difficult to get these tables in the first place, requiring multiple escalations through tech reps and ultimately a "god damn it just give him the tables" from our sales rep.
I imagine things might have improved a little since then and I know there are shopping carts that can now do the rate calculations internally. But if any shipping company forces its customers to contact a server to calculate shipping costs, then they are just as retarded as the merchants who use them (for a web checkout).
Parent
Re: (Score:3, Interesting)
Most e-commerce sites, including the ones I've built, have a multi-step process to place an order. If the step for shipping (or tax, etc.) failed, then the system would re-load the page with any appropriate messages and option changes. So, for example, if the third-party payment processing service didn't respond before a timeout the user would be asked to call customer service to complete their order. The sales rep could see the order in its incomplete state and finish it over the phone.
Re: (Score:3, Informative)
Most e-commerce sites, including the ones I've built, have a multi-step process to place an order. If the step for shipping (or tax, etc.) failed, then the system would re-load the page with any appropriate messages and option changes. So, for example, if the third-party payment processing service didn't respond before a timeout the user would be asked to call customer service to complete their order. The sales rep could see the order in its incomplete state and finish it over the phone.
I would not consider
Very poor design (Score:2)
I don't mean the USPS, I mean the people using their service.
They never thought this might fail?
Re:Very poor design (Score:5, Funny)
This is why my company ships 2 of everything that is ordered. One via USPS and the other by UPS, just in case one gets lost. Always need redundancy.
Parent
Re:Very poor design (Score:5, Funny)
Does your company offer mail order brides?
Parent
What would USPS do? (Score:5, Funny)
Re: (Score:3, Interesting)
You laugh, but older contract offices still get their software updates, including price changes, via mail.
On 3.5" floppy.
The terminals still use green-text monitors.
UPS is flaky too (Score:4, Interesting)
Same here (Score:5, Informative)
I came in yesterday morning to find the USPS module non-functional. Worse, the only working option was DHL overnight - and in case you've missed the news, DHL is now about an order of magnitude worse than the post office for domestic delivery. Even for places they say they can do next-day delivery to, actual delivery can take more than a week.
Why? Because they hand it off to the post office rather than deliver it themselves. Why it takes the post office a week to deliver it when I can get it there in two days by sending it by priority mail myself is a mystery. In any case, DHL's out of the (US) domestic game entirely next month.
My site was up last time I checked, but if the USPS option goes down again, I think it's time for a 'free economy shipping' promotion. No messy rate tables to deal with!
Express Delivery (Score:5, Funny)
USPS
at address: 0x1234 Main St., Hometown USA, zc=0x10001
hosted vs downloaded/licensed e-commerce (Score:5, Interesting)
The company I work for provides a hosted e-commerce shopping cart solution, SEO-Cart, which supports the USPS Web Tools. Of course the first call coming in for the day was from a client using USPS and having incorrect shipping prices being calculated for their store.
I went ahead and called USPS and the lady who answered was quite rude and explained to me that they had a Worldwide outage which affected other applications than just their Webtools API, and also that they hire a 3rd party company to handle their Webtools API software. She couldn't provide any other information at all and I told her a company of that size should have some sort of fail over plan in place to prevent them from being down as long as they have been. I was really disappointed in the fact she didn't even ask me for my name, phone number, or company by time the conversation was over, but she was probably being bombarded with phone calls all day.
After figuring out that USPS was completely down, I looked through our fail over code and found the following equations seem to come close to the USPS pricing:
National shipping: [cart-weight]*1.6+3.00
International shipping: [cart-weight]*1.6+15.00
These also include pricing for insurance.
After tweaking the fail over pricing code to this, it seemed that everyone using USPS were happy with the results. We also had to decrease the connection timeout set for the request to the USPS Webtools API which was also slowing things down.
The Webtools API seems to be both up and down today, with some orders having shipping prices directly from USPS and others having the fallback pricing. Either way, hopefully their IT department learns from this and also provide us information as to what exactly went wrong.
On that note, this is a prime example that I use when speaking to prospects about the advantages of using a hosted shopping cart solution rather than a licensed/free download solution. Besides the obvious IT benefits that you get with a good hosted shopping cart solution, hosted shopping cart software is typically a centralized application that can provide quick updates to problems like these. Of course this is assuming that the prospect is serious about their online store and doesn't want to handle technical support themselves.
USPS rate API SUCKS (Score:5, Interesting)
Apart from the trouble reported in this /. article which I found occuring on one of the existing sites I wrote yesterday (simply because there were no USPS prices being returned, no error, but took about 30 seconds to work out what happened), USPS simply sucks ass.
Here's why:
Some time ago, they had an API to get rates, it was called RateV2.
Then they "updated", and now have RateV3.
RateV3 is the only specification published.
To get access to the Rate API servers, you must first test your implementation against thier testing servers successfully, when they see that they let you on the production server....
Thier testing servers only work with a limited version of RateV2.
So, in order to use the USPS API, you must:
Write to the now unpublished RateV2.
Test that RateV2 on the test servers.
Ask USPS to allow you to use Production (and get the keys etc) because you have successfully tested.
Write completely new code against RateV3.
Test that RateV3 on the production servers.
And if you try and show the USPS staff the logical problem in this process, they will reply "I can not put you on production servers, until you have done three successful tests on the test servers".
Re:USPS rate API SUCKS (Score:5, Funny)
If you want to get USPS's attention, FexEx them a package. For some reason, it really pisses them off.
Parent
Re: (Score:3, Insightful)
There is a better way. Code your own backup solution, revert to it when the SaS you are counting on fails. In this case, that would be a static shipping 'estimate' process that substitutes data when the USPS service fails. Writing code what depends on a third party is dangerous. If your business depends on that software you are fooling yourself if you think it won't fail. It will. This is where coding disaster recovery functions into your system is important.
Many of the posts here are about how stupid this
Re: (Score:3, Insightful)
In this case, you might want to only guarantee prices quoted from USPS and use your data as 'estimates' in the event that your own data is wrong and costs change at shipping time.
In yet other cases, the backup plan might simply be to use data from last week or yesterday if that is sufficient, but is data that is held in house on your servers.
Depending on the situation and requirements, any number of solutions are possible. By using a 'back up' solution that is not as good or perhaps as accurate, when the sy
Re:Healthcare? (Score:5, Insightful)
Parent
Re:Healthcare? (Score:5, Informative)
UPS and FedEx can't, because it's illegal [aei.org].
Parent
Re:Healthcare? (Score:5, Insightful)
The government can't even manage to keep a simple web service online, and people still believe that it would be wise to let them control health care.
Get real. On several occasions, I've had to manually intervene to fix idiotic billing f*ckups between my PRIVATE insurer and a PRIVATE hospital, who had entered into mutual contracts to be in the same "network". For some reason, they couldn't get their own computers to talk to each other and I had to fix their bugs by going deciphering cryptic paper printouts myself and wasting hours calling customer service. This kind of stupid private healthcare IT problem happens routinely to millions of people every year. Therefore, using your reasoning I conclude that due to a clear history of incompetence, it is unwise to let private parties handle health care, and such practice should be banned.
Parent
Re:Sounds like fun. (Score:5, Interesting)
USPS has IT people? Oiy veih I can imagine that job.
A few years ago, I used to be one of them...
Oy Vey, indeed...
Parent
Re:Sounds like fun. (Score:4, Funny)
Parent
Re:Couldn't have come at a worse time (Score:4, Funny)
The crap I get, I hope that people buying me things are having problems.
Parent