Want to read Slashdot from your mobile device? Point it at m.slashdot.org and keep reading!

 



Forgot your password?
typodupeerror
×
Programming

Programmer Moneyball: Challenging the Myth of Individual Programmer Productivity (cmu.edu) 123

Slashdot reader jbmartin6 summarizes a new article from Carnegie Mellon's Software Engineering Institute: An academic study challenges the notion that "some programmers are much, much better than others (the times-10, or x10, programmer), and that the skills, abilities, and talents of these programmers exert an outsized influence on that organization's success or failure."

Instead, the author shows productivity variation is often a result of poor-performing outliers and some wide variation in individual's productivity from day to day. Once these factors are eliminated, the gap between top performers and normal performers isn't that great, and there is a very small supply of consistent top performers anyway. This result has a lot of implications for how software teams and projects are managed.

The article concludes that "while some programmers are better or faster than others, the scale and usefulness of this difference has been greatly exaggerated....

"Rather than try to label programmers with simplistic terms such as 'best' and 'worst,' the most motivating and humane way to improve average performance is to find ways to improve everyone's performance."
This discussion has been archived. No new comments can be posted.

Programmer Moneyball: Challenging the Myth of Individual Programmer Productivity

Comments Filter:
  • meetings (Score:4, Insightful)

    by stuff-n-things ( 89988 ) on Saturday February 01, 2020 @07:44PM (#59680202) Homepage

    It may be that some programmers hit 10x for a while, but the more productive you are, the more recognition you get, the more meetings you get pulled in to (including those unofficial ones of "hey can you look at this" types), and before you know it, your productivity is down (or your work hours are up). I guess some grad student who hasn't worked outside of academia needed a research topic to publish.

    • by raymorris ( 2726007 ) on Saturday February 01, 2020 @08:10PM (#59680240) Journal

      The "study" was conducted by people selling a programming class that claims to teach anyone how to be a great programmer.

      Their conclusion is that that when looking at a selected portion of the PEOPLE DOING THEIR COURSE, when using the methods the course requires few of the their students really stand out on the measurements they chose to use. That is, most of the students in their class completed the assignments as assigned. Which they then uses to support the claim that was their sales pitch in the first place.

      • by Spazmania ( 174582 ) on Saturday February 01, 2020 @09:31PM (#59680376) Homepage

        It's not that some programmers are 10x, it's that some programmers can solve problems that other programmers can't, no matter how much time they take. Just like some painters are that much better than others and some physicists are that much better than others, some programmers are, in fact, that much better.

        • by raymorris ( 2726007 ) on Sunday February 02, 2020 @12:04AM (#59680674) Journal

          There are no sports I'm any good at. I can't sing, I can't play any instrument. I don't even know the names of any of the popular video games. I don't know Marvel characters from - what's the other company? I haven't learned any of that because I spend my time on one thing - studying my craft. Steve Wozniak had no interest in running a company in any way, "I just want to be a good engineer", he said, and he was dedicated to becoming the very best engineer he could.

          Some people study general ed for two years, study programming related stuff for two years, then they get a degree and a job. They use their job to buy their favorite games, a guitar, whatever they enjoy. That's a pretty cool way to do it. They enjoy life.

          I don't. I buy books on defect-free programming. I spend my free time reading studies like the one we're discussing (but hopefully not bullshit ones). Earlier today I was working on my assignment from a postgrad encryption class at one of the best universities in the country. I read and listen to talks about leading development teams.

          I haven't learned to play the guitar, I haven't learned to windsurf or a lot of other fun stuff. I don't know any the hot artists on the radio right now because I don't listen to radio, I listen to podcasts that may help me master my art. That's what I've been doing for 20 years. I don't necessarily know MORE than someone else my age, but most everything I know is something I studied to become better at what I do.

          As it happens, 20 years of intentional study and practice in a particular field makes a person better at it than 2 years does.

          Again, not that I'm a better PERSON than $joe_coder. He probably does 100 things much better than I can do them. And 20 years of effort toward getting better and better at the intersection of programming and security has made me get a little better each year.

          Now it's time for me to get back to what I was doing - writing aome code to crack this encryption.

          • And for most people, it'll be that way--just a job.

            Sometimes I wish it were that easy. Then I could do any old thing, and call it a day. But I, like you, I suspect, care far too deeply about what we do.

            There's a place and necessity for passion, but it is by no means the easy path compared to the textbook itinerary of life.

          • They enjoy life.

            I don't.

            I get where you're coming from, but there's such as thing as taking something too far. Like you, I'm not interested in consuming mindless entertainment or hanging out with "friends", I want to focus on doing my own thing. But at the heart of my own thing is being creative. It helps when you do some very different things once in a while. Are you being creative, or are you just learning everything that other people have figured out?

            ESR's "How to be a hacker" can be a bit of a cliche, but there are some goo

        • by thogard ( 43403 ) on Sunday February 02, 2020 @01:23AM (#59680816) Homepage

          There is an out of print book called New Plague by Dr W. L. Livingston where he describes problems as "track 1" or "track 2". A track one problem can be broken down into sub parts and understood by one person. Track two problems are too complex to have one person break down or are broken down into problems that are too complex for one person to work on. What is a track two problem for most people might be a track one problem for a good software architect. That software architect can easily be 10x more productive.

          • I like to apply my built-in supercomputer to the track 2 problems. I see (visually, in my mind) where we need to be. Then I walk over and inspect the solution, so I can reproduce it.

            This is, functionally, working backwards like the first type of problem, but the trick here is to not look for a solution. Rather, you want to empty your mind and let it take the collective shape of all the solutions and refine it down by necessity. Physical objects are a good proxy for concepts in visualization, because, if you

            • And in that visualization, imagine which variation gives the least coupling (shared data, shared assumptions, ...) between the various pieces!

            • I should add that I use the same technique in reverse when debugging or reading some code.

              I don't judge, I don't think, "oh, they're doing this because...", I simply read and follow all the possible paths and accumulate the shape in my mind. Sometimes this is difficult, so I'll draw it out on the wall (dry erase paint is wonderful, thank you thank you thank you boss!), until it's manageable to hold in my head.

              Once it's up and running in my brain--if I haven't already answered my question, which I usually ha

        • by aliquis ( 678370 )

          Just like some painters

          I just want to throw in there that supposedly how much you practise matter for painting.. And more obvious I assume for playing an instrument.

          There may be this idea that "oh you are so talented" or "I'm not good at painting" (we can likely throw in dancing here too), but the thing is that one of the persons may have done it for 5+ or 10 or whatever years whereas the other one haven't really tried and then no shit the person who have practised are better.

          Of course having a creative mind is still valuable ass

          • Talent without practice is waste. Practice without talent is futility.

            A high school friend shared a computer science course with me. Smart guy but no programming talent. He did all the homework and more. He could handle the basic basics but couldn't apply even the algorithms he knew. I know this because I'd repeatedly help him step through a problem and with a little prompting he could get all the way through. Then I'd give him a nearly identical problem and he'd be completely stuck again.

            He went to college

      • The Personal Software Process (PSP) measures (1) "the time spent in each process phase," (2) Lines of Code (LOC), and (3) quality (defect density, etc.). The process phase starts AFTER requirements, and ends at "postmortem" which is BEFORE the customer receives the product and defects become the most costly to fix.

        IF you make a distinction between software engineer and programmer, AND you give programmers really good requirements, AND you carefully review every line of code, AND you test it thoroughly, AND

        • Over my 20+ year career, most of it mentoring less experienced programmers if not outright teaching beginners, I've noticed one metric:

          When one programmer takes 150 lines of code to do a task that another programmer does in 3 lines, the person who knows how to do it in three lines is pretty much always the better programmer.

          So one of their measurements is actually measuring badness, not productivity.

          • When one programmer takes 150 lines of code to do a task that another programmer does in 3 lines, the person who knows how to do it in three lines is pretty much always the better programmer.

            However, some programmers can maintain a 150 line Cobol program for 70 years, whereas most programmers can't maintain a 3 line Perl program for 70 minutes.

            Almost anybody, with sufficient financial motivation, can come up with meaningless studies/statistics.

            Disclaimer: While a student, I was able to undertake resea

          • 3 vs 150 seems a rather extreme example. I've seen the "3 line solutions" which are incredibly densely written code, with no structure, no comments and no one else can penetrate to try and fix the edge cases not accounted for. The end investment in that code over it's useful lifetime can end up being far greater. I'm not convinced that make the programmer better than the 150 line over verbose, over structured code not utilising language constructs, or library calls well. Though I guess I've had more luck tr
            • I would say that in the end doing the job in less lines is not the important factor as you point out. But a confident programmer does just write the code knowing it will work, where as some programmers seeming to be trying to solve problems (that don't exist.) Or to put it another way, to one programmer they are writing a function and it takes a little while, another programmer is just experimenting and keeps doing so until the get success.
            • while() is well understood, is is foreach.
              There is no need to use for(true;false;true) on every loop.

              requests.get() is already well documented. There is no need to implement it yourself, very incorrectly.

              I've also seen that the number of bugs is roughly proportional to the number of lines. The more code you write, the more bugs you just wrote.

          • Businesses might learn to appreciate that experience and skill that properly condense complex problems into foundational solutions add long term value .

          • It's hard to reliably use LOC as a proxy for elegance. But generally, yes, the programmer who can use standard tools and idioms with the least amount of fluff and hand waving is going to write the best code for long-term maintenence.

            Sometimes, you need someone who can time the instructions down to the rotation of your drum memory. But a really good coder would save the tour-de-force for when it's an absolute core, critical business necessity, because you're trading away a lot of simplicity to make something

        • Also the best programmers write good architecture, so if you give a good programmer and an average programmer a task and then time it, there is not going to be a 10x difference in fact the good programmer maybe slower, but as time passes the good programmer will put in systems to make their job easier, whereas the bad programmer will put in hack after hack making adding extra functionality a nightmare. In that case 10x is probably an underestimate.

    • Re:meetings (Score:5, Insightful)

      by fermion ( 181285 ) on Saturday February 01, 2020 @08:22PM (#59680266) Homepage Journal
      I would add that productivity suffers in a culture where perceived output is prioritized over quality. If you focus on output, then eventually you get bogged down with code that cannot be refined, refactored, or expanded. You end up not being able to move forward at all because all the time is spent fixing code.

      Back in the day where programmers were so in demand that people who barely graduated high school were considered qualified, much of my time was spent fixing simple problems that were hidden in bad code by people who had no concept of the engineering process. They were not able to solve the problem so the hid their kludge deep in the code, and the bug stayed there for years, and took weeks to fix. These were the highest and most respected coders, because they were fast, but ultimately lead to the end of good films.

      We are now in that situation again, where the drive to minimize cost is putting kids who have gone through a two week boot camp in control of out code. Even though all we are doing are using sophisticated frameworks to hack together apps from existing structure, there is still some design decisions that require some thought, and the fastest programmers are not the best at doing this.

      • One of the junior programmers I work with did one of those boot camp courses. Surprisingly, he's actually a pretty good programmer. Certainly as good as or better than his peers with CS degrees. I suspect that's because he has a generally high intelligence level, not because of the course.

        Fun fact: even the most junior programmer on our team has the title "senior programmer".

        There is definitely a conflict between code quality and apparent speed of task completion (for values of "complete" that include "half

  • Comment removed (Score:5, Interesting)

    by account_deleted ( 4530225 ) on Saturday February 01, 2020 @07:45PM (#59680204)
    Comment removed based on user account deletion
    • Now what has what you said to.do with what TFS said?

      • by Entrope ( 68843 ) on Saturday February 01, 2020 @08:02PM (#59680230) Homepage

        It says that the study is bunkum.

        The study is based on a course that demands every programmer follow essentially the exact same process, which is designed to reduce variation in productivity, on a collection of relatively small programs in a well-understood application area.

        That is a terrible basis from which to generalize, as the /. commenters who have professional experience have attested.

        • Comment removed based on user account deletion
          • by Entrope ( 68843 )

            To elaborate a bit on the types of programs in question -- they are essentially toy problems to demonstrate the application of the Personal Software Process (PSP). The results are useful for using the PSP, but they are designed to be simple enough to complete in a short period of time because writing them is not the point of the class. The programs are useful for teaching someone the PSP, but entirely inadequate for this kind of study.

          • Complexity is (usually) a sign of poorly thought out solution.

            For example, I recently rejected a PR from one of the young programmers on my team. It had nested loops five deep in one method. I designed the system it's part of, and I really had no idea what it was doing. After politely pointing that out to my colleague, he reworked it. Now it has half the number of lines, no nested loops, and it's at least somewhat obvious what it's doing.

        • That is a terrible basis from which to generalize, as the /. commenters who have professional experience have attested.

          Of course, much of /. professional experience consist of thinking they are a 10X and everyone else a 1X.

          • I would dispute that this is a uniquely Slashdot trait.

            Then again, my broader team of ~60 developers is about ~85% folks from India. 3-4 are what I would consider somewhat close friends, and this small, not-sure-how-representative sample often joke about who was able to get away with doing the least. We’ve had several conversations where they explain that they don’t have a passion for the field, but are literally pushed into the profession by their parents (it’s either programming or medi

    • by Shaitan ( 22585 )

      Agreed. And it isn't limited to development either. The same thing occurs everywhere in technology right down to the call centers.

    • by GuB-42 ( 2483988 )

      I think I observed it, but I later realized I didn't.

      Sometimes, I am that "10x programmer". I manage to solve in minutes some problem some other guy took days trying. It is the kind of moment I remember, the other guy looks at me like some kind of god, great for my ego... But no one, me included, remember the many days I spent being not that productive. I just did my job at 1x, nothing to complain about, but no highlight either, that's just work as usual. And if someone asks why I wasn't doing my "10x" stuf

      • These days if you have a classic CS education it's not hard to be a 10x programmer. If you put your phone down, close Reddit/Slashdot/etc and focus on your work for eight hours, then you're already a 5x programmer. It's that easy.
    • by lgw ( 121541 )

      Regardless, the difference in productivity between the 10x and 1x programmer is small after corporate culture sets in, and makes the 10x guy responsible for all the paperwork and meetings for the team.

      The much more important difference in our industry is the difference between the 1x programmer and the -1x programmer. That's the difference in productivity that really matters. We've all met him, the guy on the team who seems to only author bugs, and the more hours we works, the more hours are lost to rewor

      • Comment removed based on user account deletion
        • by lgw ( 121541 )

          That being said, I'd tend to thing the better programmers are the ones, that if given sufficient time will, by their very nature seek out ways to improve stuff. Sure maybe you can't do major changes in prod, but maybe in a dev fork you can reduce code complexity, or figure out an abstraction layer that is follow-able, while reducing boilerplate code by 50%.

          I spent most of my career doing that, as did the better people around me. Those of us who valued net-negative lines of code knew each other well. Sadly, I never found a corporation that valued any of that.

          One thing that really affects productivity is requirements. Unfortunately, sometimes you simply aren't given them, or the requirements you have are incomplete or poorly thought out.

          Sure, but then half the job of a senior dev is to clarify requirements, and get the team moving despite unlcear and ever-shifting requirements. That, at least, is an area where productivity is recognized, as its something your average manager can see and understand.

      • An even more important gap, and the one most often mismanaged, is the programmer who knows how to solve the problem no one else can see.

        I dread being this person. It never fails, I always seem to get troutsmacked across the face with the weird-ass corner condition nobody ever thought anyone would try.

        Then, there's just experience. Seeing a read-modify-write cycle on a shared resource, for example, will quickly catch the eye of an experienced dev looking for potential issues.

        • by lgw ( 121541 )

          Indeed. I reached the point at Amazon where none of the managers in my group liked me at all, because I kept pointing out flaws, but none of them would let a design go forward without getting my design review. Funny old business, software development. Never seems to mature.

    • by jythie ( 914043 )
      What I find fascinating is that someone claims to work in the industry AND still believes this isn't a myth. But if one is dismissive of such research, that is just the type of mentality that is going latch on to narrative supporting myths and see confirmation for them everywhere.
      • Comment removed based on user account deletion
      • by Cederic ( 9623 )

        What I find fascinating is that someone claims to work in the industry AND still believes this isn't a myth.

        Are you really suggesting that a spotty 16 year old with a "Learn Python in Three Weeks" book and Linus Torvalds have the same programming productivity?

        Just that for 10x productivity to be a myth, you'd have to be.

        I mean, shit, when you outsource your development you replace your generally average development team with six times as many people in India and net output falls.

  • Oh really? (Score:5, Informative)

    by thesupraman ( 179040 ) on Saturday February 01, 2020 @07:48PM (#59680208)

    I can only assume that the authors of this study have not actually worked with or managed any real programmers.

    Both the quality and productivity measures in terms of 'problems solved' varies hugely across programmers, and this is very clear from working with them (and having been one).
    Of course it is not simple - programmers tend to have areas fo specialization and 'talent' also, like most trades.

    Reading the overview, it seems what they are REALLY saying is 'programmers have varied skills and also varied day to day performance, they both have a distribution, therefore there are many numbers! its complicated! so... just pretend it doesnt matter, finding the good ones would be hard! so why bother'

    Which is... basically a cop out at best.. no one said finding great programmers is easy - just that they exist (and anyone sensible also realises that it is application dependent - a great database programmer is probably no good at ui, a realtime kernel developer wont be good at data mining, etc, etc..)

    And then, I note that they used STUDENT PERFORMANCE for this? and we are supposed to be looking for the best developers out there? Hello?
    Just perhaps being a great developer takes experience! who would have thought it.

    Not however that they DID find that the distribution is skewed... even for student developers.. and it is biased to lower performance. It is very easy to imagine that this bias will grow with experience, and hence we WILL see their supposedly mythical 10x programmers, as it only takes doubling of the bias from their student numbers..

    So, why are they publishing this rubbish one has to wonder. they appear to have basically proven the opposite of what they wish to conclude, but concluded what they wished anyway.. sigh.

    • Programmers aren't monolithic. Some are innately able to do more work and have really high focus over long durations with high accuracy and few re-writes. Rockstar coders often burn out, but some do well their career-long.

      Like other humans, people vary highly and can be motivated or not. They may have the uninterrupted mental energy to very sophisticated projects successful. Others are better at code review, or specialized scopes like FPGAs, DSPs, GPU arrays, low-level coding, driver & interface, UI, or

    • Re:Oh really? (Score:5, Insightful)

      by Todd Knarr ( 15451 ) on Saturday February 01, 2020 @07:59PM (#59680224) Homepage

      Much of that's because the studies measure in terms of code output, not problems solved. That makes the studies all but irrelevant since in the real world we don't care about how much code it took but how fast we can solve the problem. To quote my father, "I don't want the industrious guy who'll spend all shift, every shift, mucking out the spillage from the basement. I want the lazy sod who'll figure out how to stop the spillage so he doesn't have to do all that work.". The top programmers are the ones who can figure out (or who already know) how to get the job done faster with less work, even if that makes them "less productive" because they turn out a quarter as much code per day (but solve twice as many problems as anybody else).

      • Much of that's because the studies measure in terms of code output, not problems solved. That makes the studies all but irrelevant since in the real world we don't care about how much code it took but how fast we can solve the problem.

        Not just that. Management is often looking for a particular kind of solution.

        Years ago, I was working a project for a controller for a hydraulic system. One morning, our customer told us that the mechanical safety valve in the system wasn't as fast as they want. They asked if we could add an additional layer of safety in our software.

        Of course, the problem wasn't as simple as just comparing to a limit. It requires monitoring the first and second derivatives.

        In my solution, I used the fact that in a discrete

      • In my experience, the worst programmers write far more lines of code to solve the same problem as a great programmer. This probably factors into the 10x rule.

    • by sjames ( 1099 )

      And then, I note that they used STUDENT PERFORMANCE for this?

      Worse, they selected the students from a population that was already drinking their brand of cool aid:

      To better understand the causes of software development productivity, we performed our own study. We used data collected from our own work teaching the Personal Software Process (PSP). PSP teaches software developers how to measure their own performance and implement a personal Demming-styled plan-do-check-act improvement cycle. A foundational teaching of the PSP is that improvement starts with consistency. During the PSP class, we compiled and showed class data to the students every day; thus the instructors observed firsthand how individual performances varied from one program to another.

      For the study, we used data from the PSP class programming assignments for which students record effort, size, and defects for a series of programs. The students used their preferred programming languages and environments. To simulate a development cycle for their data collection, we had the students read and understand the assigned problem, implement a solution, and then test the solution against a set of predetermined test cases.

      I see no justification for the assumption that that pool will be representative of the larger pool. It's not even clear that the best of the programmers are IN school. They may be more likely to find a fast track and so be under-represented in the population of students and a few may yet find a way to skip school altogether.

      To your point, since they're still in school being heaped with Java, they may not ha

      • No surprise from the Software Engineering Institute. SEI thinks programming is like bricklaying and if it's not, they'll damned well make it so.

        • It might be like bricklaying, as tested. Not only were the "programmers" students, they were forced to use a narrowly conscripted methodology. That they were in a class to learn, so they hadn't mastered.

          IME all highly productive programmers use a carefully crafted basket of tools that work well for their unique capabilities and challenges. They may even have below-average productivity when forced to use a specific set of arbitrary process rules designed for large teams of interchangeable workers.

    • I am in awe of real programmers. Which is not really susprising since I'm always in awe of top engineers. Programmers are NOT computer scientists. But that's not an insult. It means they can massively out think a scientist when it comes to managing complexity and finding efficient and right ways to implement things. For eleite engineers you save massive amounts of time letting them write the kernels of your crritical operations.

      You then higher the C students to write the boiler plate code that wraps it

      • It took me a couple tries to parse that before I realized that you meant students who got C average grades, and not students of the C programming language. I was like, "wait, but the elite engineers are already using C." LOL

  • Some might prefer math more, while others prefer data structure design, and some could go a month just optimizig code and be super happy. It's silly to try to put a one-dimensional number on something as complex as a person.
    If you measure all animals by their ability to climb trees, you will miss out on the elephant who could have levelled the entire thing.

    Use everyone where he's best at, and stop following silly business esoterics.

  • Academic studies, as samples, shows that some Data Scientists are much better than others (the times-10, or x10, scientist). Also shows that average people won't understand the quality of studies, what make them to believe in any kind of article, since they can't cross-check any study. Most of the times they won't even bother to check anything, perhaps the title of the article that they broadcast to show that they know something.
  • âoeFor the study, we used data from the PSP class programming assignments for which students recordâ

    Yeah. You used your students for the study. Oh brother.

  • Unionize!

  • The languages and interfaces matter. A haskeller with a good org-mode organization system, emacs workflow (and possessing an agenda to eliminate the job of someone who pissed them off) is a scary thing.
  • by reanjr ( 588767 ) on Saturday February 01, 2020 @09:50PM (#59680400) Homepage

    The idea that the best are ONLY 10x productive is ridiculous. The worst actually deliver negative productivity.

    • Heh yeah, I once decided to outsource a project. Near the end, the supplier decided to rotate out their two senior developers, and let a junior developer fix the loose ends. Except he introduced a bug for every problem he solved. In the end, I had to fix it myself.

  • I was a programmer for a number of years. If left alone I could bang code out like nobody's business. However, distract me with pointless meetings or other nonsense and I would get derailed for the rest of the day. Oh, and those, "last minute changes" are always a PITA.
    • You obviously were not x10. Real x10 can handle distractions and last minute changes in stride.
      Can you imagine if ER doctors behaved like programmers? "I could save him if I were left alone". "That undocumented penicillin allergy was a real PITA. I'm to keyed up to see another patient today." "WTF, that nurse completely knocked me out of my groove asking me to clarify my handwritten prescription."

  • Not everyone is a Knuth, Dijkstra, Ritchie, Booch, Zaharia, or LuCun.

    And, yes, the field of software is still new and open enough that inventions are frequently called for in everyday software projects. Arguably, even more so now that we are advancing AI faster than ever.

  • ...the most motivating and humane way to improve average performance...

    So I have to ask... what about inhumane ways to improve average performance? :)

  • by Anonymous Coward

    What a complete joke. Nice try academic, but statistics is one of those fields that you make any piece of data say anything you want, which is precisely what you did here and you know it.

    That's why I became an engineer. My stuff has to actually work.

  • As with any engineering related field, some software development is basically turning the crank on standard procedures, while other work requires real creativity and / or expertise.

    There are problems for which average and rock-star programmers will have similar productivity. There are other tasks where the average programmers will never succeed, or possibly worse, create an unmaintainable / inefficient monstrosity that is best thrown out.

    Convert this program from python 2.4 to python 3.6

    vs

    We need a very ef

  • It is difficult to nearly impossible to accurately measure the population of a significant number of programmers over any period of time. I would guess that if you were able to then productivity would most likely resemble the bell curve. A small number of outliers on either end with the vast majority clustered somewhere in the middle.

    A 10x programmer would be like a person with an IQ > 1000. Not impossible but extremely rare.
  • Well, yeah.

    It is not that a small number of programmers are much better than average, it is that human catorigation likes breaking off small grouping and hyper focusing on them. Superstars in any field are not stars because they are much more productive or skilled, they are stars because everyone knows who they are and people know who other people tend to know. In programming, some people get a reputation, and have a reputation because others 'know' they have it.
  • "Of the 3,800 students in our classes (from 2000 to 2006), this study included only the 494 who used the C programming language and who also completed all 10 programming exercises."

    So they started by discarding the bottom 3,300 students (the bottom 87%) and selected out only programmers who used C and who could complete all 10 assignments.

    This right there that shows the "myth" isn't a myth. 87% of their students didn't even make it to the starting line for their study.

    Second, they're studying students, not

    • So they started by discarding the bottom 3,300 students (the bottom 87%) and selected out only programmers who used C and who could complete all 10 assignments.

      Yes, but that's not a flaw. The other 87% are most likely not going to end up as professional C programmers. If you want to know how programmers compare to one another, you don't want non-programmers in the mix.

      The big flaw is this one:

      Of the 3,800 students in our classes [...]

      To begin with, they're only looking at people who went to Carnegie-Mellon. That should homogenise results quite a bit. And they're all students, so hardly any of them will have any work experience. Is that really supposed to tell us anything about how they'll perform af

  • productivity variation is often a result of poor-performing outliers

    So when you remove the outliers, the variation in performance decreases. Who would have thought? This is a completely unexpected result: when you discard people with bad performance from measurement, the difference between the worst and best gets smaller.

  • Protected managed environments, automated clouds, all of them are designed to make programming as easy as an assembly line. There are many smart people who are working on making sure that this will happen, I believe that eventually at will, and then most programmers will be treated like assembly line workers. And yes, this includes you who are about to reply and say he is special.

  • What makes a good programmer is not just programming some well-defined task. That's easy, everyone can do that.

    What's valuable is understanding the problem to solve, having a vision of what the solution should look like, building up an architecture for it with a consistent design throughout.

    While good understanding of the problem domain certainly helps, expertise in technogy is the deciding factor in how to translate it into a software system.

  • Poorly designed study fails to find the thing it was looking for because the authors didn't know what they were looking for in the first place.

    The study was measuring programmer effort and bug rate on a bunch of tiny test problems that were too small to show any differences between bad and good programmers. 10x programmer productivity is real. But it's not "writing the same code 10x faster". The root of higher productivity is in better code design which prevents you from getting tangled in your own technica

  • by munch117 ( 214551 ) on Sunday February 02, 2020 @06:54AM (#59681210)

    The original IBM study, then one that the 10x number comes from, compared the best with the worst. The best IBM employees, mind you. It's important to note that these studies always take place within some kind of pre-selected group, and what you are doing is measuring the properties of that group, nothing more, nothing less.

    Over time, that has been mangled into a meme about the best being 10x better than average. Which is nonsense. Sure, there might be one or two John Carmack's out there in the world who can do exceptional things, but among the rest of us, in the group of professional programmers, the best are more like 3x the median (half-way to 10x).

    • by coats ( 1068 )
      Actually, the original study said " The best are 10x better than the average; the average is 10x better than the worst."
  • See https://en.wikipedia.org/wiki/... [wikipedia.org]

    Reproducibility of (even published, peer-reviewed) social science results is extremely poor.

  • and some wide variation in individual's productivity from day to day

    My productivity varies day to day mostly based on the work that needs to be done. It's not my fault if the sales pipeline dried up, or if nothing broke today (in fact, think about that last one ... nothing broke today because I programmed well).

  • As projects get more complex and require teamwork the benefit of any one member being really good decline relative to the number of small but highly complex tasks the project might require. If you're trying to squeeze performance at the bit level then one programmer could be more than 10 times better, but in the modern day and age that mostly doesn't happen like it did in the past. You need an emerging technology for an exceptional person to really show up what they can do and even then design and maybe ma
  • I assume any coder with decent knowledge of the language can throw out code and where that's good enough that's good enough.

    However in the cases where coders have come up with clever tricks which improve efficiency you can't really assume everyone will make them. Clearly people are failing in writing secure code too and of course there's a speed difference in how fast people write code too.

    So what is this about? About on par at getting code written or about on par as far as how well the final product runs?

  • I am not the fastest coder in my experience, but I am good at planning, and I have always automated writing the boring stuff like data access. The result is my generating thousands of lines of code after a week of writing code to read my database and generate stored procedures and C# code. More recently, I use Microsoft's tools to generate app skeletons in minutes. Sometimes developing smarter can create a huge difference in productivity.
  • According to the author's LinkedIn profile, he went to college for 13 years!
    https://www.linkedin.com/in/wi... [linkedin.com]

    Clearly, he is not a member of the 10x club. This might explain why he doesn't believe some programmers could be much faster than he is.

  • The study does not show a 10x performance difference between the top 10% and bottom 10% of developers. It shows a 5x difference. More broadly, it shows a linear distribution of productivity. This distribution's linear shape is my biggest concern in a first pass reading of the paper. Human abilities usually follow a Gaussian distribution. Drawing a straight trend-line through that data doesn't feel right.

    TFS is available legally from IEEE "The End to the Myth of Individual Programmer Productivity", Will

Lisp Users: Due to the holiday next Monday, there will be no garbage collection.

Working...