Why 37Signals Abandoned the Cloud (thenewstack.io) 92
Web software firm 37Signals has migrated off the cloud after spending $3.2 million on Amazon Web Services last year, said co-founder David Heinemeier Hansson, who is also the creator of Ruby on Rails. The Basecamp project management software-maker bought $600,000 of Dell servers and expects to save over $7 million in five years by running operations in-house. From a report: DHH likened clouds to "merchants of complexity" where they are incentivized to make things as complex as possible to keep customers hooked. He compared that to the original Internet, which was not built on complex cloud services geared for multi-tenancy, but rather on simpler tools such as Linux and PHP, which anyone could use without cost. This is not to say cloud has zero value for all use cases, [Kelsey] Hightower and DHH agreed.
Clouds make perfect sense in many cases, for start-ups that do not know how much infrastructure they will need, and also for enterprises with a lack of expertise and money to burn. For many companies in the middle, though a lot of profit margin can be recovered by reducing cloud costs and running things in-house instead, the two argued.
Clouds make perfect sense in many cases, for start-ups that do not know how much infrastructure they will need, and also for enterprises with a lack of expertise and money to burn. For many companies in the middle, though a lot of profit margin can be recovered by reducing cloud costs and running things in-house instead, the two argued.
Remind me (Score:3)
!RemindMe in 5 years. ;-)
Re: (Score:3)
5 years wont change that.
Re: (Score:2)
Agreed. It is not unheard of for orgs to 'outgrow' the cloud. Facebook and others eventually needed to build their own datacenters and even their own hardware.
Let's see where this company is in 5 years and how accurate their savings statement was. It could be grossly inaccurate up or down. PHB's and their quotes are just silly to me.
Re: Remind me (Score:2)
"At scale" is not a thing. That's like saying "big". Does every "big" entity work exactly the same way, with exactly the same considerations over what they do or use? Nope. This is hand-waving mumbo jumbo to keep from thinking about the problem.
Re: (Score:3)
Security is usually where things go wrong. While cloud accounts do get compromised, it's rarely on the scale of companies that run their own infrastructure.
I think it's as much the lack of excuses (can't use "AWS doesn't allow us to turn off X even if it's inconvenient") as the lack of hiring the necessary security people.
Re: Remind me (Score:1)
Re: (Score:2)
!RemindMe in 5 years. ;-)
Exactly. When they've -actually- saved $7M in 5 years, it'll be news.
Perhaps 37Signals' business is tolerant of downtime and slow access via the Internet. If not, they're either going to spend a lot of money with a lot of false starts getting that right, or else they're going to get it wrong and fail.
Speed and reliability require proximity and interconnection. Achieving that without leasing it from someone who already has it is quite the challenge. Way, way more than $600k worth of challenge.
Re: Remind me (Score:1)
Re: (Score:2)
According to the faq, or more precisely what's missing from the faq, they haven't yet experienced the joy of remotely detecting and diagnosing flaky hardware. Apparently they haven't even hired someone with the talent to do so.
FOr many companies, hybrid cloud is the way to go (Score:3)
But, for some reason, companies preffer "multicloud", with the added complexity and the curse of the "least common denominator"
You see, in Hybrid cloud, you run some workloads in a local cloud, and some workloads in "someone elses cloud", but with the possibility to move workloads between the two clouds. Your applications can be cloud native all the way (and is recomended thay are, to reap the full benefits). When workload is low, as much tasks as possible are run on the onsite cloud, when load is high, the less sensitive* tasks are moved to the public cloud, and only the most sensitive ones are kept on the local side.
In order to do this, you need both clouds to match. Google and amazon (and some others) can not do it. Only Asure, Openstack, VMware (through partners) and, to a lesser excent Xen type clouds can do that.
But may companies opt for multi-cloud, running on amazon + azure + GCP at the same time. that ussually results in a veritable mess...
Well, that's life.
* Sesitive can mean many things, and more than one at that. Sesitive to latency? Sensitive to keep confidential info in your datacenter? sensitive because of regulations? all of the above?
They didn't migrate 'off the cloud'. (Score:2, Informative)
The 'cloud' is the damn Internet. They relocated their servers from someone else's rack to their own. To everyone else, they're still 'in the cloud'.
"Cloud" is just stupid terminology made up by idiots who got excited by a flowchart/diagramming symbol because it was easier for them to say it without understanding what it meant.
Re: (Score:1)
Maybe we can distinguish between a LAMP cloud and a proprietary cloud, like AWS.
What's wrong with LAMP-style* hosting for typical normal-sized web apps? These proprietary environments seem to be breaking things that already work and are time-tested.
You can add OSS or proprietary deployment management engines or whatnot on top of lamp rather than depend on Amazon or MS if that's what you need. Magic deployment is the alleged selling point of proprietary clouds, but let's explore this claim.
* App languages ca
Re: (Score:2)
What's wrong with LAMP-style* hosting
Only the P(hp) part. If it is P(erl5), all is golden :)
Re: (Score:2)
Absolutely nothing is wrong with self-hosting, IF one is both able and willing to deal with the challenges that come with it in 2023/2024, which likely include security, scalability, geographical redundancy, regular patching and testing, and so forth.
But managing such things is far outside most companies' core competency, so it can (in some cases, and not others) be prudent to pay someone else like AWS to take care of those things.
Not really true anymore (Score:2)
The cloud has also become a stack of application services offered by said cloud company. They offer far more than just compute for rent.
That said, most large companies are far better off running their own data centers and using the cloud as an extension when it makes sense.
Re: (Score:3)
Of course they do. If cloud computing was commoditized in a standard way, you could move nodes from cloud to cloud and take a hybrid approach running some nodes of your own. The add-on services are what creates the vendor lock-in and makes it so that the cloud providers don't have to compete on cost - because you can't easily leave anyway. We are hitting a point where cloud computing costs as much as and sometimes more than just running your own servers, even if it is more cost efficient for the provider
Re: Not really true anymore (Score:2)
It's always been that way. I remember doing the calculation for my company back in 2012, and AWS was slightly more expensive than a similarly specced virtual host, assuming you are running it 24/7 and have a consistent bandwidth requirement.
Where it had the financial edge was when your usage peaks would otherwise require a much higher spec than your baseline would need or when you didn't need an always-on server (e.g. if you could switch off overnight).
Our requirements were relatively modest so maybe that d
Re: (Score:2)
Cloud comes in varying degrees. Starting from colocating your gear in someone else's racks to IaaS, PaaS, and finally SaaS. There are varying degrees in the *aaS offerings depending on what level of fault tolerance you are looking for.
The higher you go up in the offerings, the more you relinquish control and responsibility but gives you the most bang for your buck.
The biggest mistake companies can do when moving to the cloud is to lift and shift their severs and services 1:1. This is the most expensive o
Re: (Score:2)
"Cloud means you're saving data on / renting someone else's computers and storage space."
AWS doesn't become something different when Amazon is using it internally.
Re: (Score:2)
I'll never forgive the otherwise intelligent jwz for pushing that dumbass "there is no cloud" meme.
Re: (Score:2)
"Linux and PHP"? (Score:4, Informative)
Well, Unix dominated anyways. Does this guy think the dot-com boom was "the original internet"? Linux was pretty entrenched by that point.
In any case - PHP's dominance of the internet (well, of the web anyway) came much more recently. From what I recall, perl was king in the early days (and lo, how the mighty have fallen...).
Re: (Score:3)
Re: (Score:2)
The name of the game in web development today is Node/React, Angular javascript et al.
Using Node and Express lets you quickly prototype APIs, MongoDB schemas locally, etc, while a React frontend gives you access to SPA features and theme libraries like MUI/ChakraUI
You can do that much faster than setting up a MySQL database with Apache/Nginx and PHP.
Re: (Score:2)
The name of the game in web development today is Node/React, Angular javascript et al.
Which is why the biggest site on the internet is programmed in PHP.
My first web app was in Perl (Score:2)
I wrote my first web app in Perl and we used it at my workplace for a decade. It's a shame that Perl went through a drawn-out engine rewrite and is now just a lowly scripting tool again (and even in that space, nowhere near as popular as it was in its heyday). Such a shame.
the cloud is NOT about saving money (Score:1)
the cloud is for convenience, speed and high availability, not for cost.
if you are happy with running things onprem, have the staff and you don't care about a single point of failure, then sure stay onprem.
if you want to scale, quickly deploy hardware and don't want to maintain it, while planning for high availability, then the cloud is the only option unless you want to maintain multiple datacenters
Re: (Score:2)
Re: (Score:2)
Quite frankly, that is the only cost-efficient use of the cloud I see. Also if you occasionally need machines with really large memory for some things.
Convenience? (Score:1)
Not sure how convenient it is to learn an obfuscated, OSS tool with arbitrary metering attached. Maybe convenient for the the CTO when his salary is determined by expenses. What if you have latency requirements? Well, you better hope that 90% of your customers are within 50 miles of us-east-1. That's pretty convenient. Otherwise, cut out the middle man and go directly to the CDNs.
Re: (Score:3)
I like to use the Netflix movie NextGen as an example. The studio behind it had to do it in 3 months. Before services such as AWS, they would have needed to buy huge amounts of servers and infrastructure.
They simply scripted to spin up as many servers as needed to do all the rendering and then just shut them all down after they were done and returned to normal levels.
Rendering in the cloud? (Score:4, Interesting)
I've worked in TV and movie studios for the past couple of decades. I've seen a few attempts to push all the rendering to the cloud, but I've never seen it work out in practise. Tangent Animation, which made NextGen and did a bunch of the work to get the Deadline render manager working well on AWS, is a relevant recent example. They shut down in 2021 and terminated hundreds of employees without notice, termination pay, or severance pay.
I'm sure one of these days cloud rendering will work out. It's not a bad idea in principle, especially with more and more people in the industry working from home. But... the huge amounts of data (multiple versions of multiple layers of every frame in every shot) make for very expensive data transfer bills. Combine that with the way that production companies squeeze VFX and CGI companies, and it means that the companies who survive have to squeeze every bit of computation out their equipment that they can. Just a couple of months ago I was helping an animation company decommission equipment from 2009-2010, because it was only after 13-14 years of images had been squeezed out of that equipment that it finally became economically sensible to get rid of it.
There are few things cheaper than a render node that's already paid for sitting in a rack that's already paid for that stays shut off when you don't need it.
Re: (Score:2)
Granted, it was a sad situation with Tangent and one that really sucked as they were quite active with the Blender Foundation.
Re: Rendering in the cloud? (Score:2)
Technical success and business failure are not entirely disconnected in this case. Or rather just having some technical achievement doesn't resolve all other requirements they failed to meet.
Re: (Score:2)
Technical success and business failure are not entirely disconnected in this case. Or rather just having some technical achievement doesn't resolve all other requirements they failed to meet.
I believe they are disconnected. Animation studios fail and close all the time because they either go insolvent or investors stop giving money.
We aren't saying that they managed to deliver NextGen because they didn't pay their employees. The closures happened long after that movie was done.
Re: (Score:2)
Who says on-prem means single-point-of-failure? I have several audit-customers that are on-prem for anything critical (and plan to stay that way) and all have redundant DCs in sufficiently far apart places. Given the observable history of cloud outages, that makes them more resilient.
As "don't want to maintain it", even when using cloud services, there is a lot of maintenance you still need to do yourself. For example, backups not in that cloud are still mandatory, and CloudNordic (google it) and the comple
Re: (Score:3)
Fast, reliable, cheap.- Pick any two.
It takes one to know one (Score:2)
DHH likened clouds to "merchants of complexity
...says the man who created Ruby on Rails
That's rich.
Saving $7 million over 5 years = $1.4 million/year (Score:1)
Their spend is $3.2 million and they're going to save $1.4 million per year so their new annual cost will be $1.8 million. And $600k of that is hardware over 7 years so uh 3.2 * 7 is 22.4 million and their hardware cost is $600k so that means they're buying 21.8 million in hardware + an army of labor + facilities instead of having a team of like 4 cloud engineers @ $200k and the headache of managing $20 million in assets and liabilities is Amazon's problem.
I'm sure whoever is the new VP of Spending
Re: (Score:2)
Re: (Score:2)
The things that they may not be thinking about:
- rapid scalability - once you're out of hardware, you're buying more and waiting for shipping, arrival, assembly, rackmounting, configuration, software deployment, and finally cluster inclusion. Cloud providers can give this to you in seconds, automatically. And you can scale back down just as instantly and automatically as you shed load instead of paying for a lot of idle equipment turning electricity you pay for into heat you pay to remove, so you can cove
Re: (Score:2)
Running your own DC adds very little effort. Most of the things you need to do you need to do anyways, regardless whether on-prem (usually means private cloud) or public cloud.
Re: (Score:2)
cloud does have an lot of hidden fees / stuff that (Score:2)
cloud does have an lot of hidden fees / stuff that has an front cost with an more hidden back end cost.
say logging is not free and that low front end cost for logs does not cover that data cost or the backend storage cost for them.
Re: (Score:2)
There are a lot of charges which can easily bite you, unless you factor everything into the calculator. I've seen people not realize that data egress can add up, so someone doing backups from one provider to another can be expensive.
In any case, you are paying for that server, be it on your site, or at the Amazon data center.
Re: (Score:3)
Their spend is $3.2 million and they're going to save $1.4 million per year so their new annual cost will be $1.8 million. And $600k of that is hardware over 7 years so uh 3.2 * 7 is 22.4 million and their hardware cost is $600k so that means they're buying 21.8 million in hardware + an army of labor + facilities instead of having a team of like 4 cloud engineers @ $200k and the headache of managing $20 million in assets and liabilities is Amazon's problem.
I'm sure whoever is the new VP of Spending Money on an Army of IT techs is very happy with their new promotion. Seems like a terrible investment IMO
WTF are you talking about?
1) Do you think Amazon just runs all that compute for free? They have the army of cloud engineers and you're just paying for them in higher cost of service. They buy compute, and manage it, roll in all those costs PLUS 50% and pass it on to you. You're just paying for amazon's engineers (at a markup) instead of your own.
2) Saving 1.4M annually seems like an amazing investment to me.
Re: (Score:2)
But are they actually saving $1.4m annually after realizing costs of data center construction, data connectivity agreements, hardware buildout and maintenance over the anticipated lifetime of the equipment, increased labor required to build and maintain all of it, increased risk for disaster recovery planning due to many more single points of failure, etc.?
I doubt they are. This sounds a lot like the same corporate shell game of "oh hey we made this single large invoice go away, but the need for the servic
Re: (Score:2)
There can be benefits to scale. I'm not at all sure that Amazon passes those on, though. And there can also be advantages to being able to do some parts of the job differently.
I would EXPECT that any large company could save money by running it's own servers. And they could be expected to decide how secure they want their backups to be. (You can save money by skimping on backups, or by only backing things up at critical points in time, but it's at the cost of increased risk.)
That said, I can't guess whe
Re: (Score:3)
Indeed. It seems we have a certain population of cloud fanbois here that are incapable of accepting that the cloud may not actually be that good. And hence they start making up bogus arguments why obviously only the cloud is the One True Solution. Not rational, not competent, and utterly pathetic. Not unexpected though. Kind of a "safety in numbers" and "sunk cost fallacy" effect.
Re: (Score:2)
It's the contemporary equivalent of "no one got fired for purchasing IBM / Intel"
Problems Incoming (Score:1)
Looking forward to seeing the posts about their service being down and a host of other issues.
Cloud isn't just about savings, It's about uptime, access, speed, scalability, and more. The fact they're switching for the savings for themselves, rather than to serve the customer, is all you need to know. This is going to blow up in their face.
Re: (Score:1)
Re: (Score:2)
How can so many be wrong?
All you need to do is look at the current state of politics to figure out why that particular logic is a complete failure. It's very easy for "so many [to] be wrong" when they're working with incomplete / incorrect / plain bad information, combined with focusing on the wrong goals.
Re: (Score:2)
It's a legitimate worry. There are several. These are things you need to deal with if you're handling your own servers. I don't know if it's really still possible to have multiple independent connections to the internet, but that used to be standard practice. But perhaps those have all be "centralized for efficiency".
Re: (Score:3)
Bullshit. A well-done redundant on-prem infrastructure is not only a lot more reliable than typical cloud offerings, but also when something is down the sysadmins are immediately working for _you_ and not for some more important other customer.
Re: (Score:2)
In other words (Score:5, Informative)
Clouds make perfect sense in many cases, for start-ups that do not know how much infrastructure they will need, and also for enterprises with a lack of expertise and money to burn.
Cloud solutions are exactly like scab workers: they're twice as costly as regular employees but they're a lot more flexible to hire and fire as needed than regular employees. And when they go, they take the experience they gained working for you with them.
In other words, unless your business is highly unstable or seasonal, in the long run, you're better off investing in hiring and training real employees and pay them enough to retain their expertise, and you're also better off doing your own IT for the same reason.
Re: (Score:2)
That REALLY depends on the size and nature of the business. For small companies, the cloud allows them to avoid putting resources into things that aren't their primary area of expertise. For companies that depend on their data not being shared with others, the cloud is pure poison. They need to keep that stuff off not just the web, but off the internet. For large companies, they'll save money and retain expertise by running their own stuff. But there are a large chunk of businesses that the cloud is a
Re: (Score:2)
Indeed. A lot pf people moved into the cloud because of the hype. They now are finding out that hype os not good advice. Almost nobody actually needs the added flexibility of the cloud and you can very well run your on-prem stuff and, if needed, still add sme cloud stuff for a while.
But the real fact of the matter is that most of the skills and people you need with an on-prem installation, you still need when going into the cloud. And the cloud is not even really more reliable and may vanish (CloudNordic) o
Re: (Score:1)
Welcome to the real world (Score:3)
Glad to have you.
The cloud has definite uses. I love AWS for prototyping, for instance, and it's very easy to dynamically scale up and down. But they're right, the intention is to rope you into services you might not have needed on-prem, and to lock you into using their products. This isn't evil, it's just part of doing business.
All this really means is to use the right tool for the job. The cloud is indispensable for some applications, a money sink for others, and it's important to know the difference.
Re: (Score:2)
"the intention is to rope you into services you might not have needed on-prem, and to lock you into using their products."
Whereas a hardware vendor will only sell you the absolute minimum you absolutely need, right?
Re: (Score:2)
"the intention is to rope you into services you might not have needed on-prem, and to lock you into using their products."
Whereas a hardware vendor will only sell you the absolute minimum you absolutely need, right?
Not at all, as you well know. Hardware vendors will sell you more than you need if you let them. On-prem has advantages in certain areas. For instance, having your dev environment on preowned equipment, providing you have the expertise to make that work. It's the hardware vendor's job to sell you as much as you can be convinced to buy. It's your job to buy only what you actually need.
Re: (Score:2)
A counterpart to me at a different college explained it this way to his boss (the provost....)
"It gives a lot of comfort knowing we can walk down the hall to the systems folk or the developers and smack 'em on the head and say 'WTF are you doing?!?!'"
Cloud is computing as a service (Score:2, Interesting)
Re: (Score:2)
You can get all that on-prem, and look, some people even know that and have it.
The Koolaid glasses are tall and deep (Score:2)
I talk to a lot of business owners and IT people, and it's definitely the exception rather than the rule that they want to keep servers in-house.
I hear an endless stream of 'moving to the cloud' dialog that carries a conviction that is almost religious.
While there are certainly many scenarios when the cloud is the best option, I would be hesitant to go that direction if one of your reasons is that 'you can't be bothered to maintain a server'.
If you can't be bothered to maintain that, how will you feel about
Clouds also work for companies that nead scaling (Score:2)
Re: (Score:2)
Sure. What you do is you run 100% on prem and add 200% cloud services for that one month. If you are prepared for that, it is easy and does not cause a lot of additional effort.
Re: (Score:2)
It falls apart with data. Having 2/3 of your capacity on the wrong end of a WAN link to your database makes that kind of bursting difficult.
Re: (Score:2)
If you have a relevant volume of data that you need to access locally, the cloud is not even an option. People are keeping things on prem to have fast access to large-volume data.
Re: (Score:2)
Cloud services are like a marriage (Score:4)
Getting in is fairly straightforward and quite painless, but if you want back out, be prepared to pay a fortune and quite likely a good deal of your stuff will go missing.
Too Complicated? No (Score:2)
While certs have a limited amount of actual usefulness outside of stuffing your resume, they do a service in that you have
Re: Too Complicated? No (Score:1)
> ...too many other tools to mention
That's the problem, right there.
37Signals becoming less specialized (Score:2)
For a technology company like 37Signals I suppose it is possible to reach the same level of infrastructure specialization in-house as a dedicated cloud provider, but I doubt it - especially since they had once already gotten out of the business. More likely, some of the true costs and risks of operating its own infrastructure will just be hidden in support areas like finance, legal, facilities, help desk, etc. There is also the company-wide opportunity costs of getting distracted with this exercise. Better
It has always seemed odd to me... (Score:2)
Despite the fact that real estate, hardware, electricity, and labor can only get so cheap, that somehow adding a middle man in the mix is a cheaper / better option? Most of the New Tech Ideas since the 2008 financial crisis have been adding new layers of middle men to things and riding it all the way to IPO. Take something that used to be cheap and worked great, fuck it over so that it's fragile, and in the case of software, make it so that it costs more per year than the previous perpetual+maintenance cost
Lots of good reasons (Score:4, Insightful)
There are lots of good reasons to move to the cloud, but saving money isn't necessarily one of them.
We run into this over and over and over and over at work; teams are pushed to migrate to the cloud "to cut costs" but then come to find out that in some (many) cases it costs more, and there are other hidden costs as well.
The clouds are greener on the other side (Score:2)
Back in reality...
It just took me 15 seconds to pull up a user list of 50 people on the Office 365 admin portal on a 500x500 fiber line. Why doesn't MS give a shit? Because they already have our money. Welcome to the cloud. We still run on-prem AD with authoritative over the Azure mirror and it's staying that way. It's another example just like this one of how it's all great until you see what their infrastructure
Who cloud is and isn't for (Score:2)
The Cloud isn't for people who already know how to run and manage their own servers and infrastructure. If you do, it's always cheaper to co-locate your own servers and do things yourself.
The Cloud is for people who don't have that know-how. You're paying for your provider to handle managing the servers, doing OS updates, managing the database server software, managing the container cluster needed to auto-scale applications based on load and other criteria, managing the network infrastructure, firewalls, ga
The "cloud cliff"... (Score:2)
In past recessions, businesses could make it by cutting things like support agreements, using unsupported hardware, or keeping with older versions in order to make it through lean times.
These days, with stuff in the cloud, if a business can't cough up those cloud costs, after three months, boom, lights go out. In my experience, cloud providers are not going to allow someone to pull from storage until the back usage fees are paid, so the business is no longer able to function in any capacity. At least with
The cloud.... (Score:2)
I have worked with several Enterprise level clients over the past few years that have gone from on premise to cloud. My observation is that it's a better value proposition for small companies than big ones.
First, the good news:
1) With cloud based systems, the vendor generally takes care of backups, disaster recovery, DB & OS patches, etc.
2) Cloud vendors offer redundancy. Data is synched between data centers so if your main DC goes down you will be back up with little to no interruption.
3) With cloud ve
Lies (Score:2)
They abandoned the cloud because they have always had NIH syndrome. They just love to make things from scratch. They probably wittle their own desks and chairs from slabs of wood.
The cost savings thing has always been a joke. The TCO of IT spend depends entirely on the particular needs of that business. Sometimes it makes more sense to use CapEx for your infra spend, sometimes OpEx. Sometimes you need specialization in house, and sometimes you can farm it out. And "The Cloud" is not a fixed price. Literally
Re: (Score:2)
I have a lot of sympathy for the DIY crowd since that's how my own mind works, but the simple fact is that hardly anyone can build, power, and cool a data center as well and as cheaply as the big cloud providers. Hardly anyone can deliver data across the world as fast, either, regardless of how much they spend.
Sufficiently large enterprises that have special requirements probably should keep certain things in-house, it's true. But for the rest of us web & database monkeys, ser
Cloud costs are unpredictable (Score:2)
The smart move: See what 37signals is doing ... (Score:2)
I mean this from a technical perspective, of course. If your idea of success is billable hours and bamboozling clients then by all means, follow in their footsteps.
When Netflix announced its plan years ago.. (Score:2)
Migration in both directions (Score:2)
There are plenty of companies where using the cloud (someone else's computers) isn't really a viable option. For example, think defence, or critical infrastructure (telecoms).
For everyone else, many existing businesses which grew up before the availability of cloud based solutions tend to dabble, move a few things off site, to achieve cost savings. They are likely to get burnt by problems stemming from lack of understanding of cloud infrastructure (migration between cloud vendors, that sort of thing), and s