The Cult of DevOps 114
packetrat writes "I was at OmniTI's Surge conference today, which turned out to be, among other things, a meeting of the cult of DevOps. Ars Technica covered the keynote and some of the presentations, but some of the best stuff is in the comments. Google CIO Ben Fried told the tale of a really poorly engineered trading application at Morgan Stanley that he was associated with, and how the way IT was structured there contributed to that engineering and to its spectacular failure, costing the bank untold millions in stock trade processing fees from its institutional customers. He said what he learned from cleaning up the mess has informed how Google runs its IT operations, and a culture that promotes generalist skills. A lot of how he describes Google's approach sounds like the DevOps kool-aid a lot of the other speakers were serving, but it also sounds like common sense — are most IT organizations really that poorly run that developers are totally unaware their software is sending messages that are generating network storms, or network engineers are clueless enough about QoS to route leased lines into their data center through their public-facing Internet?"
Cult of DevOps? (Score:4, Insightful)
The traditional way of doing things didn't work for 30 years. Why is it that when people are trying to make (and apparently making) a difference to how companies work, they're regularly denigrated by a large subset of the very people whose working lives they're trying to improve.
Haters gonna hate, I guess.
Re: (Score:2)
I read the article, I think the title is basically talking about:
'he thinks Google gets this right. “We go to great lengths to hire people with engineering skills, put engineers in operational roles and give them power and accountability."'
The big issue is lots of people are being replaced (Score:1)
By scripts.
Add to that that devOps is much, much harder than system engineering. A DevOps engineer needs to know enough about multiple programming languages *and* algorithmics *and* data and telecom infrastructure to tweak the programs that are really controlling the network. There aren't many people like this at the moment (so if you want to move as a programmer to a $200k/year job, understanding the packet datapath and it's controlling plane is where it's at, provided you're good at optimizing algorithms.
Re: (Score:2)
I don't want to be an asshole.
But if you don't adapt, you might loose your job. Especially in IT.
Is that something people are surprised about ?
Re: (Score:1)
Lots of people are going to lose their jobs, because only the better ones will be retained. Managing systems manually is a skill that will go the way of car building skills. That's a lot of people.
Re: (Score:1)
And IT is just about the only sector that is supposed to be adding jobs to the economy. It will not go down smoothly once this starts happening.
Re: (Score:2)
It could also be that we end up being a lot more effective. That is after all the whole point of IT. I've never made much sence to me why people did the 'manual labor' in IT.
But if I look at education in my home country: the Netherlands in Europe, I can tell you there aren't even enough people studing programming to fulfill the available positions (well, I don't know the real numbers, but that is what they keep telling us in the local IT-press and looking at the people that get hired I'm not all that surpri
Re: (Score:2)
Sorry, English is not my first language
Re: (Score:2)
The people who can take one sentence and turn it into this [wikipedia.org] might be a cult.
Re: (Score:3)
Huh?
DevOps isn't about mere communication or awareness -- it's about operations being automated via tools maintai
Re: (Score:2)
That's pretty much how it is where I work, it's been that way for as long as anyone remembers (at least 15 years). But no one ever called it DevOps.
In any case, however you define it, it's a concept and as such it can be expressed in a sentence. Most of us are more than capable of extrapolating the implications. Well, at least those of us that understand all seven sigmas.
My view i
Re: (Score:2)
15 years is quite some time -- by any chance were y'all an early adopter of CFEngine? That's the only toolkit I can think of under an OSS license for doing OS-level configuration programatically that was available at that time, unless y'all wer
Re: (Score:2)
by any chance were y'all an early adopter of CFEngine?
All in-house tools back then (though I wasn't there). Well, still actually. There are hallway conversations about chef and puppet. Some very well thought out scripts and rsync still work so there isn't a very strong motivation to experiment. A few people want to be able to put Puppet on their resume so it ( or something else ) will probably show up somewhere sooner or later.
I have to agree that labels are useful, but I also think this devops thing
Re: (Score:2)
Personally, I can't recommend Puppet -- it makes simple things simple, but when you get to the point where you want to do hard things, you end up needing to make use of integration points that aren't so well-thought-through... whereas Chef has extra primitives ("data bags", search, &c) that are well-integrated and more power in the language (real Ruby, as opposed to a limi
Re: (Score:2)
Re: (Score:2)
"eventually, you just can't afford to waste developers' time on teaching them admin skills"
OK, maybe you can't afford it. But then you will be forced to afford the alternative -and it's not going to be any cheaper.
Re:Cult of DevOps? (Score:5, Insightful)
I'm guessing the main haters are sysadmins, who see threats to their importance and way of working.
It's no longer desirable to have a gradually grown system. Everything should be built on virtual machines from scripts. You no longer need a wise old guy with a beard to change anything - who cares if your modification might trash the system? You can spin up a test instance, and dump it if it doesn't work right. If you have two apps that don't play nice with each-other, just put them on difference machines. The philosophy of a DevOps guy is very different to that of a sysadmin - you don't do brain surgery on a horse while it's in the middle of a race. Just shoot the damn thing, and send in a healthy clone. (Or would you prefer a car analogy?)
There was a world of difference between "works on the dev box", and "works in the production environment, without screwing up anything else". That gap is getting bridged by DevOps practices. OK, you still need people with sysadmin skills, but the way they work (and the things they have to learn) is becoming a lot more like development than sysadmin work, and sysadmins are not bred to like change.
Re: (Score:1)
This guy gets it. How many manhours - manmonths even? - have I wasted at previous jobs trying to revive a system or piecemeal fix a misconfigured or broken system? How many other companies have done the same? It's just not worth it - we have the technology to spin up a new VM instance to replace it... and while that seems obvious, the implications on how your build process ought to be created and the benefits you can get from it are pretty significant.
Re:Cult of DevOps? (Score:5, Interesting)
And how do you know the new VM won't have the same problem? If you never know what went wrong and fix the actual problem you will just end up restarting VMs constantly.
Re: (Score:2)
Re: (Score:3)
Sorry but no. What you are suggesting is a more modern "reboot the server and see if that fixes it" and it solves nothing. Somewhere the bug is lurking and waiting to rear it's ugly head again. It could happen in a week or it could happen tomorrow but it's very likely it will happen.
Re: (Score:1)
Sorry, but no. What the parent is saying is something like
"reboot it once and let's see what happens. If the shit hits the fun again, clone the damn thing, reboot it again to keep the service alive and try to figure out what's happening on the clone."
There is no point, for the patient, to have his disease figured out after his death.
Re: (Score:2)
Better idea: bring in backup server/VM or whatever then figure out what went wrong on the original. If it happened once than it will happen again given the same config. You might have some bad hardware, or you could have a bug that only happens under select conditions.
Case in point: the sasl software most people use with Postfix has a memory leak when the wrong password is entered (yes they know, no they don't actually care) Some would say a memory leak isn't a problem because it would take thousands of
Re: (Score:2)
I think many admins are ok with the "reboot fix" technique, because they don't work in high-stakes environments. If Bob in Accounting's windows desktop goes down twice this week due to the same issue, who cares (just reboot it again). The same goes with most non-mission-critical services (and those without strict SLA). The same can sometimes also apply in vastly-redundant heterogeneous environments, where having one application instance go down is not a big deal since you have 1000s mo
Re: (Score:2)
You have the time between when a service starts misbehaving until it's detected and then reported to someone who can start a new VM, That time gap is something the customer will notice Fixing it right the first time prevents more outages and the customer will care about that. A VM/Backup server/ whatever may help with shortening the outage but it will not reduce the time spent debugging.
Re: (Score:2)
Us old-timers call that "shotgun debugging"
Re: (Score:1)
Re: (Score:2)
Sure -- but that's a thing to be replicated and fixed as a development task, not a critical-production-maintenance one. Not like you can't keep a copy of the bad VM around for inspection.
More to the point, if you're Doing It Right, the "spin up a new VM" process involves applying configuration-as-code, not replicating off some handcrafted mys
Re: (Score:2)
On a properly deployed system you should expect config problems for the first couple of days after you deploy something and someone does something you didn't expect even if the same config worked in testing and after that the bugs get into the "one off" variety. In either case you want to find out what went wrong.
It's not the "load a VM to get the service working again" that I object to. I object to the idea that you can just reload a VM and never debug.
Re: (Score:1)
If you don't have configuration management then you're barely even doing Ops, let alone DevOps.
Re: (Score:2)
I'm guessing the main haters are sysadmins, who see threats to their importance and way of working. ... Everything should be built on virtual machines from scripts.
The sysadmins like virtual machines; they make life easier. What they don't like is developers telling them how to do their job. Internal software doesn't come with instructions, it comes with a coworker making demands about server resources, DNS/firewall/network configuration, licensing and backup requirements, etc... Not all those developer requests will be reasonable or correct; devs make mistakes too.
Re: (Score:2)
Thing is, that's not how it works in a mature devops shop.
Development writes the code that set up the DN
Re: (Score:2)
Development writes the code that set up the DNS, the firewalls, even the kickstart files (or whatever your local equivalent is) that control OS installation. If development's code doesn't work, developers' dev VMs and QA's environments break first, because they're configured just the same way.
Development does that? So developers know all about DNS, firewalls and OS installation? Errrr, no. They think they do and they think it's easy, but they simply don't. Those that do get burned. Badly.
Backup? Development writes the code that automates it.
Backup as well? How fabulous. In reality it's sys admins who should be encroaching on developer turf, not the other way around, and the reason why this Devops cargo cult crap has come into existence is because developers now need to know about it, they don't want to and as such they want to pretend it's somethi
Re: (Score:2)
And that's why you have a crossdiciplinary team -- DevOps -- taking care of that stuff rather than foisting it on the application developers.
Good DevOps folks tend to also be system-level developers -- ideally people who know everything from the kernel level up -- as well as having a good chunk of system adminis
Re: (Score:2)
Development writes the code that set up the DNS, the firewalls, even the kickstart files (or whatever your local equivalent is) that control OS installation.
I'm unfamiliar with the term "DevOps", but in my kind of workplace, Operations do that kind of OS configuration stuff (including writing automated scripts to manage it), and then we have Application developers who don't know and don't care anything about the OS, but want their precious application to write to C:\Windows\WhyDontIhaveRightsHere and have an always-on two-way RDP connection on port 31337 to a data centre in Korea.
OS and Applications Development will fight eternally, in my experience. It's nothi
Re: (Score:2)
I've always been lucky enough to work places where the app team's senior architects had actually heard of the concept of "security" and were able to beat t
Re: (Score:1)
Hmmm, perhaps someone with some form of cross-disciplinary Developer/Operations knowledge and experience could help?
Re: (Score:2)
I started life as a traditional sysadmin. Now I'm very much a DevOps guy. I love it. DevOps is a massive improvement on the old throw-it-over-the-wall, compartmentalised way of doing things.
Re: (Score:3)
I think you have missed the entire point. You still need people who know what they are doing to make the scripts and setup in the first place.
What DevOps fixes are places where any problem gets passed off on someone else. Software is too slow? Software devs blame the database people who blame the systems people who blame the networking people (if they can) and as a result pretty much any problem is fixed with hardware upgrades and when that becomes impossible you end up with a badly running system but ei
Re: (Score:2)
I'm guessing the main haters are sysadmins...
And that is exactly the chasm of the "us" and "them" attitude that the DevOps folks (IMHO) are trying to bridge. Thank you for your fine example, sir.
WTF are you talking about? Devops is nothing new (Score:4, Interesting)
Historically a sysadmin has been able to and does write, fix software, and administering large scale systems has always meant templating and then deploying from updated templates... which means having a build system which can build to a template usually automatically (it's boring) which apparently is now called continuous integration. You also need an infrastructure designed to easily deploy changes. VM or physical is almost totally irrelevant, VMs make life a little easier.
Lets see... Infrastructures.org for example was definitely there mid nineties (and still is, cool.). That's almost a whole generation ago, how old are you?
I have no idea what makes people think this devops stuff is new. Is it just the fact that there are now thousands of newbies trying to make names rediscovering good practices? Is it just that we now have more large scale systems, or is it that we now tend to have highly distributed vs centralised systems?
And it's not development. It's engineering. The difference is maths.
Go on, get off my lawn!
Re: (Score:2)
In my last job I pointed out to our developer when the development work dried up, as it does, that our clients paid for t
Re: (Score:2)
I'm guessing in the 90s, it was pretty rare to actually use those practices. A new shop would have one server called "server". Then it would get a few, named after the Seven Dwarfs. Eventually, you would get to the point of needing to get a new sysadmin who could handle large-scale deployments (or have the old sysadmin change the way they worked), but that would be around the post-IPO stage (if ever), not the "three guys, one of whom is figuring out how to script EC2" stage.
Re: (Score:2)
"I'm guessing the main haters are sysadmins"
And then you are wrong.
There certainly is *some* uneasiness on *some* sysadmins, but not because what you seem to think but because of very valid concerns.
Devops, as other have stated, while a new word is not a new concept -other highlighted infrastructures.org for instance, and it is basically having the sysadmin mindset to use the developers' tools. And here comes the uneasiness because using the developers' tool puts developers in a tactical advantage... for
Re: (Score:2)
Sorry, that's just a bit of bullshit. In today's IT organizations, a sysadmin can't be a curdmungeonly small-minded person. He's got to accept and embrace newer technologies for the specific purposes of reducing the amount of work he has to do.
If the sysadmins are resisting, it's because it means it's more work, or they're simply not up to the task of newer things. The distinction may be hard to understand by someone who doesn't do sysadmin work. Often, changing to a new approach means they won't be able to
Re: (Score:2)
Spoken like a true developer ;-)
Developers need to get used to the idea of Operations, as much as the Ops folk need to get in bed with the dev side of things.
Devs and Ops have traditionally led very different lives, and it's that schism that's really made DevOps-style movements so hard (and important).
As a random example - Ops people are typically on call, and get woken up (automatically by a monitoring system
Re: (Score:1)
I'm not sure I can take anybody who calls an attempt to make IT and Development more aware of each other a cult, seriously. .
Actually, the people who were espousing devops were the ones who called it the "cult of devops". I'm not hating, just wondering if it's just relabeling.
It is always IT's fault (Score:2)
are most IT organizations really that poorly run that developers are totally unaware their software is sending messages that are generating network storms
How is it the IT organizations fault the developers are unaware of the effect of their software? Is it the IT departments responsibility to debug software?
Re: (Score:2)
Is it the IT departments responsibility to debug software?
Where I work? Yes.
Re: (Score:2)
Well, solely blaming 'IT organizations' for developer unawareness is unfair, but in answer to you second question, yes, a good IT department can and will debug software to the extent it does not take down other services in the process. Sometimes a behavior cannot be readily replicated and knowing enough to capture viable debugging data before having to revert it to a working state can be critical. You don't want to leave it messed up until a 'developer' can get to it because you need it back to working AS
Re:It is always IT's fault (Score:4, Interesting)
I think this is the whole point of the devops stuff. It's not IT's fault, it's not the developer's fault, nor is it QA's fault. The larger your organization, the larger communications gap between these 3 areas. The devops folks are there to coordinate and communicate between the three. They may review/write a little code, they may consult on infrastructure, they may make sure things get tested appropriately and meet the customers expectations. For small shops/teams this fairly straight forward. Big shops have to have someone who's job is to know everything about the project. There is a similar position in commercial building. It's the "site manager" they make sure that the contractor is building what the architect asked for. From the number of bathroom stalls, to the depth of the foundation. Which at the same time working with the architect to fix problems that arise from misplaced city sewers lines, an unavailability italian terrazzo tiles in rural Illinois. All the while making sure that the customer is getting what they paid for and understanding and approving changes. Sorry I didn't have a good car analogy.....
Very Well Put! (Score:1)
Wish I had mod points handy.
The separate department structure for computer systems encourages everyone to take the approach of 'Not My Problem' even when they could fix it. From a broader perspective, the barriers among Dev/QA/Ops are artificial: it is all computer people doing computer-ish stuff. External constraints are vastly simpler than the construction analogy -- no opcode shipments stuck in Customs, no delays from rain or freezing temperatures. If the computer industry can not overcome these lesse
Re: (Score:2)
The problem is not that organizations are large or any of that. Small shops have the same problem. The problem is management. It's a manager's job to make sure the lines of communications up/down and horizontal are open and that his team gets the resources (whether physical or not) it needs to do a better job. SOOOO many managers fail at that. Some think they are good programmers and get in the way of architecting the software, others think they are good talkers and start acting as a filter. That's why ther
Re: (Score:2)
Maybe my workplace is just unusual, but the problems I've seen include:
1. A chargeback system that discourages anybody from helping anybody else with anything unless that help is formalized and expensive and planned in advance.
2. Since only the most routine functions of internal service organizations actually get formalized and planned in advance, the ability to do anything else gets whittled out of those organizations in the name of efficiency. The windows guys know how to create/edit groups, the oracle
Re: (Score:2)
Re: (Score:2)
Communication between QA and dev is a given. If it isn't, you have a major problem. Communication between QA and tech support should also be a constant. The best leaders of dev teams, IMHO, work from within tech support.
Re: (Score:2)
How is it the IT organizations fault the developers are unaware of the effect of their software?
Because they didn't take the time to send a quick email saying "hey, your code breaks in production"?
As far as I can see, DevOps is just a fancy name for "people who are working together on the same project should probably talk to each other", and I'm amazed at the number of people who are surprised by or even hostile towards this suggestion.
Re: (Score:2)
The developers are using third party API's which can do all sorts of odd things, like send out multicasts or broadcasts for servers on the local network (eg. 127.0.0.1 -> xx.yy.zz.255, xx.yy.255.255, xx.255.255.255), while sys-admins are responsible for the network topology including firewalls, bridges, routers, and general IP routing.
Sysadmins fail to set up the IP filter tables correctly, and local broadcasts intended for an small network can go circulating around the entire corporate network.
Any in-ho
Re: (Score:1)
Recently (Score:3)
A while ago I thought most OSS application and framework projects including such Gnome and KDE are in trouble, due to the large use of the fumble-around development approach. Also known as the first code then think approach. All the great model-based, model-driven and agile development methods seam to be far away from the way many OSS projects are developed.
However, lately I came out of my ivory tower and stood eye in eye with experts from the industry who largely believe that they are professionals and really do great stuff. They also use the fumbling approach. The main difference is the call it agile. Even though it is far away from such an approach. I always thought that one of the problems of our new economy company back in the 2000 originated from being too deep in code and too light on design, planning and documentation. But it looks like, that tinkering is a more widespread way of software development. So I guess that leaves me with bad management (which was not my responsibility).
Most of these tinkering approaches originate in the absence of developer discipline and the "Add this quick"-management method. but I am telling nothing new. We all know for decades now what is wrong in software development. A lot of people wrote books on patterns (design and otherwise), but in the end if no one follows these patterns the problems remain.
The "Add this quick"-management originates at large by a misconception of IT and its importance for businesses and other organizations.
Re: (Score:2, Offtopic)
From the keyboard of Charlie Stross:
http://www.antipope.org/charlie/blog-static/2009/06/how_i_got_here_in_the_end_part_7.html [antipope.org]
Re: (Score:2)
These problems will always exists.
You can not predict everything and you can't generate all possible testloads.
There is always friction between:
- adding features
- lowering operational expenses
- trying to predict what part will fail first under load, increasing scalability
- deploying quick fixes because you did not anticipate something
- deploying real fixes for quick fixes
I could be wrong of course :-)
Re: (Score:2)
The last one appears to almost never happen. One just end up with a house of cards in the form of quick fixes until the whole system is scrapped and a new one erected (likely because some manager got a nice dinner out of a silver tongued sales rep).
Re: (Score:3)
It's like anything else. The better your initial plan or design the less likely you are to get surprised and the more likely that the surprises will be small. There's an old saying about plans in the military "The best plans only last until the first bullet flies" and it's true... but that doesn't mean I didn't spend hours and hour planning operations. Sure nothing ever goes exactly according to plan, but a good plan considers as many variants and contingencies as possible in order to anticipate reaction
Re: (Score:2)
Now that I think about, getting back to what the article is about.
I think the point of the article is: know the environment where your code gets deployed and have some idea of how it will probably get used and actually monitor that so you can better adjust.
An other thing I got from the article is that this does not really fit with the idea of cloud computing.
Most people would have no idea how the environment looks where their code will run if it 'runs in the cloud'.
Re: (Score:2)
You can do all the stuff in the cloud as well. However, I do not know why this cloud thing is such a hype. But that's a different story. Good cloud environment provide you with the right information on provided performance properties and the avail virtual machine/platform. The problem is that people think in code. And code is not good to think in. Especially when you have to handle multiple sets of aspects which even are not all part of the implementation, but of the requirements to the application.
I know t
Re: (Score:2)
To quote Eisenhower, who possibly faced the biggest challenges of this sort one man ever had to: "Plans are worthless, bu
Re: (Score:2)
"However, lately I came out of my ivory tower and stood eye in eye with experts from the industry who largely believe that they are professionals and really do great stuff. They also use the fumbling approach. The main difference is the call it agile."
You mean, cowboy coding is now called agile? I thought Agile still has documentation/process just somewhat lighter than RUP/waterfall.
Re:Recently (Score:4, Interesting)
Also known as the first code then think approach
No, it's think *as* you code. You do think some before starting to code, a vague rough picture of the pieces, but you don't invest a lot of time in that 'pure' design phase because the more detailed you plan without proving it out, the more you *should* throw away as you start implementation and realize how ill-conceived your design was or that maybe your design was adequate, but a better way makes itself apparent when actually implementing. Generally when I see a development team incapable of operating at all in this manner and fail to achieve any measure of 'complete', they only 'complete' under other strategies of development on a technicality and produce low quality product. Some management people think that methodology can be used to make piss-poor (cheap) developers serviceable and avoid having to reward/retain good talent.
A lot of people wrote books on patterns (design and otherwise), but in the end if no one follows these patterns the problems remain.
I think there are two aspects to this. One is that most development an organization is predestined to operate in a specifc way depending on who comprises it, regardless of what name they pick to describe it. I know organizations that did 'waterfall' and 'changed' to 'Agile', but really just renamed things they did, acted the same, and called it 'Agile'. However, I don't know if this is a bad thing. I think getting too hung up on a specific 'pattern' others preach in conferences is bad, and organically feeling your way for a process that works for your team is better. Awareness is good, but getting locked in is bad.
However, I have found in the sea of Agile and Waterfall and all sorts of buzzwords a methodology that really resonates with me:
http://programming-motherfucker.com/ [programmin...fucker.com]
Re: (Score:3)
"Flowcharts? What is it with you engineers. It's software, it's not as if we're building a bridge or anything. We just keep on typing stuff in until it happens and that gets the job done. If it doesn't work we can clean it up later."
Re: (Score:2)
"Flowcharts? What is it with you engineers. It's software, it's not as if we're building a bridge or anything. We just keep on typing stuff in until it happens and that gets the job done. If it doesn't work we can clean it up later."
And that's why the cats keep getting stuck in the tubes.
Re: (Score:2)
Yup - we use "agile" but we still start with a requirements spec and all the requirements are written before the first line of code. In fact, even if we have multiple "sprints" we still write all the requirements before we start the first one.
The only thing agile about it is that we use the right buzzwords, and we do a chunk of requirements at a time (as if that wasn't what everybody always did), and we demo the software to customers and maybe refine the requirements from time to time (as if that wasn't wh
Re: (Score:2)
That's exactly the point of methodology. It is to reduce the process of software development to a bunch of rote steps which low-skill people can perform consistently. Assembly-line software.
Re: (Score:2)
As a sysadmin dealing with these so-called 'agile' developers, I'd mod you through the fucking sky, if I could.
You describe it as it is. They write what I would call "quick one-off shit scripts/programs" and call it production quality code. Sure, it gets a 'finished product' done quickly, but it sure as hell won't be fun for the people who get hired on months later to fix it.
People need to realize that custom applications take a long damn time to develop properly. Maintaining custom software is expensive l
You are kidding, right? (Score:2)
The curse of 'enterprise' (Score:4, Insightful)
I've been around enough to know that this 'new' 'DevOps' philosophy is just the way it always has been done at many successful companies making extensive use of technology.
I have come to associate the phrase 'enterprise IT' with those who *don't* work that way, and make their lives needlessly complicated. Of course, every last party to the mess will generally recognize it and know what could hypothetically be done about it, but only bitch in private about it and rarely ever push for meaningful change. The reason is simple, so long as it is a complicated mess, it requires a great deal of human care and feeding, meaning job security. Management can't force things to change without huge risk as everyone has sufficiently entrenched themselves.
Re: (Score:3)
Size of an organization definitely also makes a real practical difference.
When you're part of (as I am) a technical staff small enough to all fit in the same room, it's very easy to consult across the admin-developer boundary, along the lines of a quick phone call or stop by somebody's desk. Admins and devs regularly interact, both professionally and socially. That also creates a culture where the admins and devs are very much on the same team, are working towards the same goals, and collectively creating l
rhetorical question (Score:1)
That last was a rhetorical question, right? You could have stopped at "Are most IT organizations really that poorly run". Yes. Upwards of 95% of CIOs are completely unqualified to hold their position. They are grossly overpaid talking heads with NO real understanding of the technical underpinnings.
Seriously. The single largest problem that most IT organizations have is that the people running them are completely unqualified to execute the duties required. It is not that IT workers are overpaid. Good IT work
More methodologies! (Score:1)
And more companies that take a methodology which has quite sensible premises and transform it to a paper-pushing-based freak-child.
it's not about having a clue (Score:2)
oblig. Fred Brookes quote (Score:1)
This cancerous attitude happens in any project, across any set of roles, once things get big enough.
Not a triangle (Score:2)
They see things as a triangle: developers, Q&A and IT, and in the middle DevOps.
That's not at all how it works from my experience.
While developers and Q&A are very close, developers are not particularly close to IT at all, while Q&A is.
Testing is what requires large infrastructure set-ups. Development doesn't really: the Q&A guy directly gives you access to a working configuration that demonstrates the problem. As a software developer I never have to interact with IT.
Re: (Score:2)
And that's the whole point of DevOps. To bridge the gap and bring the teams closer together. This is one of those "if you don't know who the chump is, you're the chump" situations. Because you haven't experienced it means your company(s) are missing DevOps and the whole point behind it.
Get rid of dev/qa/ops silos.
Not just ops (Score:2)
Devs need to drink more of the DevQA cool-aid too.
Dripping with Contempt (Score:1)
In an attempt to answer the barely articulated question buried amidst the run-on editorializing, I will answer "yes". Many otherwise extremely talented developers carry little understanding of how their code performs once exposed to chaotic and nondeterministic production environments. Likewise many ops engineers hold little appreciation for the motivations of and the pressures on those developers.
Much of this isolation stems from culture, but at least as much is from experience. Being paged at 3am because
Agile - no "deep cross-departmental integration" (Score:1)
While there is a lot of agile practises around Google, including "stand-ups", iterations, scrum, planning poker, and whatever, there is hardly any "cross-departmental integration". Communication is as poor as at any other huge behemoth. Just like we've read about multiple internal teams fighting each other at Microsoft and Nokia, so do many teams at Google create similar products, both externally and interna
architects of technology vs architects of product (Score:1)
the software industry in silicon valley by and large does not have a role 'architect of product'. we have architects of technology. the product comes together as various teams: tech pubs, qa, tech support, professional services, it, ops, manufacturing, marketing -- all contribute their efforts to take the technology delivered by the typical development team and transform it into a product. DevOps is one attempt at mitigating or filling this gap to varying degrees depending on the folks involved. Now what is
The "it's not my problem" problem (Score:3)
It can be summed up like that: If the problem will arise in an area he isn't responsible for, and if implementing it that way regardless is cheaper, faster or less problematic for his department, he will implement it that way.
Running the traffic through the internet instead of the LAN? Why not? It's not going to be the development's problem (since that's operation's problem), and it's faster to do it that way, so it's done that way. A user interface that makes users beg to break the programmer's fingers and make him a consultant? Not the programmer's problem, it's the user's problem, and if he can implement it faster with a shabby interface, it will be done that way.
The problem isn't that the people wouldn't know that these problems will surface. Often they know that quite well. The problem is that this is not a criterion, while the time they spend on solving the problem is. Hence, whatever lets him get it faster off his table will be the way it is done.
Sure I'll comment... (Score:1)
...and I'll even try to remember to check back in case people respond.
Assumption: the trading application being critiqued is the same one that was there when I was an IT consultant at Morgan Stanley. I left in August 2000.
I know the application well. It was developed by a department headed up by Vinny (whose name is withheld because...I'm senile and don't remember it). I worked in the department that wrote the messaging infrastructure that was used by every application on the sell side of the firm.
If the
don't care (Score:2)
cults are stupid.
if you want to name yourself something stupid, i don't care. Stupid people do stupid shit because they are stupid.
Re: (Score:2)
This sounds more like a blunder on the IT-Sec's department than anyone else's.
Security is important (hey, I can't speak against my trade, can I?), but the moment it keeps people from being able to do their work, they will start looking for workarounds around the security system. And this is about the WORST thing that you could possibly have as an IT-Sec. It's akin to an inside job, only without the intent to actually cause trouble.
But intent doesn't really matter when the data is lost...