Become a fan of Slashdot on Facebook

 


Forgot your password?
Close
typodupeerror
×
Technology

Salon.com on Open Source Medical Software 95

Posted by Roblimo from the this-is-too-serious-for-BSOD-puns dept.
mke writes "Life or death software; new medical programs show the strengths of open-source coding -- and its weaknesses." One of the biggest weaknesses mentioned in the article is that there's no software company to sue if something in the code kills or injures a patient. Another problem, at least in the U.S., is that FDA approval for medical software costs millions. The question is, can these problems be overcome, or will proprietary software continue to be the norm in critical medical applications?
This discussion has been archived. No new comments can be posted.

Salon on Open Source Medical Software More

Salon on Open Source Medical Software

Comments Filter:
  • I've spent a lot of time thinking about this. Last September, my wife had a focused radiation treatment to her brain. Anyway, the question is this: would open source development benefit software that controls radiation machines? I don't think it would.

    Yes! It would. One of the most critical parts of the procedure that your wife had was the selection of the target(s), the selection of the collimator, and the dose. You want to really zap the target, but spare nearby important structures. These decisions are often made by "educated guess" - paramters are entered, and the display shows the dose delivered and the dropoff of the radiation as it relates to surrounding structures. If this was a Gamma Knife machine, several spherical targets were chosen (since the device is roughly spherical) in order to approximate the desired target. This is a nontrivial problem that is not yet automated and optimized. If an open source project to choose the optimal parameters was created (using genetic algorithms, simulated annealing, or whatever), the treatment plan can still be modeled and directly compared to the best "educated guess" solution to see which one delivers more radiation to your tumor and less radiation to your optic nerves - before pushing the button.

    Open source works better when there is a large developer base.

    But there potentially is a large developer base. Several very important medical advancements were made by nonmedical people who had an affected loved one. Often they didn't perfect their inventions in time to save their loved one, but many others benefitted. Everyone will have a loved one get sick and/or die. Medical charities are common. Many people donate money to help advance medical research or to buy medical treatment for those who cannot afford it. If someone is going to donate their coding skills for open source projects anyway, why not donate their coding skills for such a noble cause.

  • The systems themselves use various RTOS's (which I'm not sure policy let's me mention). However, ouside of the devices themselves, we use everything from Solaris to NT to BSD to Linux. We have used off the shelf testing software, but ended up with our own custom system based on Perl.
  • Free Software has nothing to do with the Bazaar development model, that was just ESR's observation of some open source project structures.

    Here is what Free Software is about: when you buy something, it becomes yours.

    When you buy a car, it is yours. You can keep it as-is, you can paint it a new color, you can soup it up, you can give it to someone else, you can crash it into a police station in your quest to kill Sarah Connor. You buy the car, the car is yours. It no longer belongs to Ford, it belongs to you.

    Proprietary software doesn't work that way. You buy the software, but you can't change it. You can't give it to someone else. You can't modify the software. You can't use it to hack into a police station's computers in your quest to kill Sarah Connor. You buy the software, but it doesn't belong to you.

    Folks on this discussion who don't think that a chaotic development model would fit life-or-death software are right. But chaotic development models are something peripheral to free software, not integral to it. (ESR has emphasized this development model; but ESR is, in this capacity, a salesman, albeit a very insightful one.)

    The point is, once you buy a piece of software, you should own it (IMHO). Once a hospital buys some equipment, they should own it, and its software.
  • The liability question, at least as put above, is naive. There are some things you can do, notwithstanding even express contractual disclaimers in negotiated and executed agreements, that under certain circumstances cannot be disclaimed -- and conduct leading to personal injury can lead to such liability.

    (Think about it -- if liability for medical malpractice could be disclaimed, would there be a doctor or hospital in the world who wouldn't make you sign away your rights as a condition of treatment?)

    Accordingly, publishing OSS medical software probably is a risk -- although most publishers (in their individual capacity are likely to be relatively judgment proof compared to the size of most such claims.

    But the interesting observation here is the suggestion that open software must be orphaned from regulatory approval for failure of a company to pay for such approval. In my view, that objection is highly overstated and takes perhaps a naive view of the economics of the situation.

    Indeed, the company never really pays for the software's approval -- at least at the end of the day. Nor do they pay for the outrageous liability insurance. Customers do. Proprietary medical software that is highly regulated or requires elaborate insurance is expensive, in part, because of these expenses.

    If truly good OSS medical stuff were out there, approval might arise in time by the marketplace that intends to use it, either through grants, communal conduct by the marketplace, or "new economy" ideas such as websites soliciting voluntary contributions to support worthy quasi-commercial work.

    Those notions should work, that is, unless you believe the "free rider" problem precludes such benefits to society, in which case the arguments for strong IP were right after all. . .
  • ...took CS courses from a guy at the University of Ohio-Miami (can't remember his name) who allegedly wrote the control code for the U.S. military's nuclear missiles.

    He brought this up in class as an example of why programmers should be aware of the purpose of their code and the consequences if it doesn't work.

    His thoughts: "OK, there are five billion people on this planet. If I fuck this up, they're ash."

    J.
  • Gotcha. I've not had to deal with that one. Seems like a pretty dumb ass way of doing things from FDA's point of view (yes.. I know). The assumtion should be the other way around -- Everything is experimental unless explicitly labeled as OK for clinical use and signed off by someone that can give the user a "get out of jail" card.

    Someone can bring a freakin' bicycle into an operating room and use it because it wasn't explicly labeled as experimental and not for clinical use?

    garyr
  • But what if a car manufacturer has made a calculation error in the brakes of the car and you get killed because the brakes don't work? OK, you own the car but the manufacturer is still responisible of its mistakes.

    And what if a hospital buys some software (open source)? Do the hospital have to check the code line by line before using it?

  • There are some markets where open source, end user applications just won't go. Medical software, Nuclear power plants, Military apps, etc. It's not like, say, the Gimp, where you can get several thousands, or even tens or hundreds of thousands of people to use it and sniff out the bugs. The "noone to sue" argument is irrelevant. The scale by which open source offers any benefit doesn't exist.

    Also, the consequences of failing software in these instances are potentially high. This is not software where if it fails, the worst that happens is a core dump takes up some space on your hard drive. Even a forced reboot that leaves dirty filesystems does not compare to the potential for thermal nuclear meltdown, or radiation overdose.

    This is software that is reviewed line for line by independent auditors. Software where each individual change to the code is justified by at least a few paragraphs of design review, which include required documentation changes, and all manner of approvals and signoffs. I've been that auditor, and programmer, and documentor, and I even have a few KSLOC running in several of our nuclear power plants.

    I don't see these people downloading "ReactorMonitor-0.3.1" and performing the task of documentation necessary to qualify the software for mission critial, safety of life applications. Not going to happen this lifetime. What you're paying for when you purchase such a program is the 6 foot stack of documentation that justifies the accuracy of every line of code in the program. If I were to perform that task of documentation (and I have), I would be reluctant to give it away (i.e. license by GPL or BSD or ...), considering what the buyers are willing to pay for it, and considering that the target market for such software is so small.

    However, an application based on an open source OS or open source libraries may be employed in such an environment, pursuant to the requesite code reviews and documentation, of course.

    On a more promising note, one of the guys I used to do nuke work for told me, and I quote, "You'll never see Microsoft Windows of any kind in my control room." Linux was just barely on the map back then, so I never got his impression of it, but if any non-embedded OS came into his control room, it was guaranteed to be unix-based. =)
  • Great post. I do similar work and was thinking I'd have to post this same info myself. Thanks for saving me the time.

    I don't follow you on "Open Source's explicit refusal to permit change control". What refusal? Say (for the sake of the argument) that you wanted you burn a modified linux kernel into the ROM of your device. You've changed the kernel and you've redistruted it so you will also release the source of your mods. Where does change control become an issue?

    If I later come along and make further changes to the software ...fine. My changes are not under your control, it's true. So? If I were crazy enough to crack the case on your device and put my new "improved" firmware in it I have shifted the entire validation burden onto my shoulders. If someone dies I am to blame, not you.

    FDA requires *you* to have adequate change control. You would have to be sure that the
    device that you are shipping does not have my unvalidated mods in it. you would need to take reasonable steps to make sure that your customers can't apply mods without understanding the consequences (encase ROM and seat in plastic like cable tv boxes or at least a big nasty seal on the box). If it were a plain old piece of software that you were selling, something as simple as an MD5 checksum would do the job (this is not the software we sold you, we are not responsible for it).

    I don't see the issue you are raising here.

    gayr

  • Arandir wrote:

    Whether or not medical software is open or closed doesn't matter to much. What does matter is that it works 100%.

    But opening the code makes it a hell of a lot easier to determine if the code works at all. I'm not saying we should all sit down and start writing a GPL replacement for all the hospital firmware out there--frankly, I have better things to do and you probably do too. But think of it this way: we all know that open code works better, and that peer review is the world's best debugging process. Shouldn't our most mission critical applications have that security?

    Now, there are a few ideas in that last statement that need to be expanded a bit. I think we can all realistically say that there won't be a flock of programmers rushing over to write hospital software, so let's think about how open source ideas can be implemented here.

    1. No open source software can pay for FDA Approval - but a standards body could. This is exactly the sort of thing that should be funded by the doctors themselves. See number 2.
    2. That leaves the AMA liable - if the doctors as a whole take the time and spend the resources to insure that open code works as it should, then that leaves the doctors, all of them, liable. That's a whole lot better for the developers, and it puts the blame (IMO) back where it belongs.
    3. AMA should clamor for proprietary code to be opened - this clears up the issue of "who will write it?" The GPL is certainly not applicable here, but maybe some other license would be (God, I hate to contribute to the proliferation of licenses, but I think we really need a new one here). Maybe this is where the government should get involved (ouch, that hurts, I'm a Libertarian).

    To me it looks like it is in the best interests of the medical community to regulate itself, following the above guidelines. It's the classic case of the private sector taking up regulation for both the benefit of itself and the benefit of the customer (with the side benefit of taking up some of the responsibility that the government once assumed).

    OK, just my opinions of course. And I just rolled out of bed...

    -k

  • Like another poster here, I've written software for a medical device company. We used a large open source component in the device. I'm not at liberty to comment on specifics.

    The whole liability question is easy to answer: the device manufacturer bears responsibility for the whole device. For example, if they store data in a file system and then use that data to control patient treatment, and the file system gets corrupted, the manufacturer of the device is liable, whether they bought the file system from Microsoft, bought the file system from Red Hat, downloaded it from the network, or wrote it themselves. So the device manufacturer has to choose which means of procuring a file system will give them a file system which will harm the fewest patients, because they will have to pay for that harm.

    Again, as another poster mentioned, the FDA does criminally prosecute people who are negligent. They don't just regulate the industry -- they can walk into any company, any time, ask to see any records, and suspend or shut down the operations of any company at any time.

    The FDA has extensive documents on software requirements for medical devices:

    http://www.fda.gov/cdrh/comp/swareval.html

    My experience is that medical software is somewhat higher quality than non-medical commercial software, but not as good as the best open source software. Peer review works.
  • ...That little chunk of code that controls traction in your new car can kill you with the same efficiency as the code in your heart pacer.

    IMO, the best thing to do is to require that ALL software code used in ALL life critical applications is disclosed and open for audit. Companies can find other ways to protect IP - hardware design, just having the best quality etc.
  • When an unexpected blizzard stops business, damages property, and takes lives, you don't see a whole rash of lawsuits against meteorologists for not warning of the danger

    Russian meteorology Center was sued by the Moscow government for fialing to predict some strong winds. I will bet something like that should have happened in US - gee, people sue for much smaller things...
  • Why would dates be included in a fission reactor

    short int month_since_checkup = 12*(current_two_digit_year - last_checkup_year)+ month_since_april;
    if(time_since_checkup > 12 && today_date == April_1) {
    shut_down_that_critical_valve();
    // and who cares that other valve is working OK
    }

    And it all hardwired into some small chip...
  • Okay, I wrote a pretty huge DICOM3.0 implementation for the university and I was thinking what license to go. As I thought over and over, I realized that medical software simply costs too much and an open source model could keep software solutions with a very limited budget.

    In my case, this might mean that many hospitals could be provided with low-cost and highly-available medical imaging workstations. Linux boxes with internet connection would be the way to go.

    So I say GPL it. It will work very good at least for the non-ultra-critical kind of medical software. And who says that free software can't do it. I think my lib surpasses most of the commercial DICOM implementations, and it's gonna be truly free.
  • Why would dates be included in a fission reactor

    short int month_since_checkup = 12*(current_two_digit_year - last_checkup_year)+ month_since_april;
    if(month_since_checkup > 12 && today_date == April_1) {
    shut_down_that_critical_valve();
    // and who cares that other valve is working OK
    }

    And it all hardwired into some small chip...
  • I'm getting pretty sick of hearing that if your code isn't Open Source, then it's full of bugs. I can point you to a pile of applications that have been developed by individuals and companies that are efficient, reliable, and robust. For a group that keeps screaming that people should have an open mind, some have pretty narrow views about commercial software development.

    On top of the FDA certifications costs, there's this little thing called domain understanding that need to be taken into consideration. You just can't put source code on the net, and ask everyone to take a look. Applications such as anesthesiology monitors require years of medical training and experience to write, and you can't expect an expert in OSes to be able to take a look at it and make it significantly better. He may catch semantic errors, but not logic errors, which is the most likely place an error will occur.

    Like everything else, Open Source is a great concept, and it's great for application that are used by a large number of people, since it's large user base have an understanding of the application, and they can contribute to making it better. But it's better to let experts deal with specialized applications.
  • My company runs web frontend servers on Windows NT and backend servers that acquire various pieces of data on Solaris. They're all smart enough to agree that Linux would be a more reliable way to run the frontends than NT, (Solaris is prohibitively expensive for us) but Windows guarantees Y2K compliance, and if it turns out they are wrong, we CAN sue them. We couldn't sue Linus Torvalds. And that's the reason why we use inferior proprietary software - we need accountability.
  • Actually, you are (somewhat) correct- I should have been more clear.

    You will not see two guys in a garage introducing a drug, like you would a few guys in a garage coming up with a program that everyone will use. It takes a massive corporation to fund the testing and handle the liability.

    You also will not see much "open source" drug development in the sense that the research will be open to all. Corporate drug research is very tightly controlled and heavily patented- it's the only way you can afford to go through the testing. You can get the results of the human studies of a given drug and will see occasionally papers on the actions of a drug, but no company will ever release the base research documents for perusal- it would be suicide.

    Yes, university researchers will still publish. They do much of the basic research using public dollars. The applied stuff will never be open source.

    Eric

  • Once again, just because software is free doesn't mean it was chaotically developed. There can still be a single company, or single programmer, behind it (just as with cars, there is a single company, and a team of safety inspectors). If a car company sells you a faulty car, it (the car company) can be sued; if a software company sells a hospital faulty software (free or otherwise) it can be used.

    You should always be able to sue someone if they screw you over by selling you a faulty product. This is irrelevant to the issue of what degree you should own that product once you buy it. (If I buy a car and die because I messed around with the brakes myself, I have no one to sue; similarly, if a hospital buys and modifies free software, and someone dies as a result, that's the hospital's fault.) Software companies are responsible for the quality of their products as they are at the time of sale.
  • Change control and open source go together just fine.

    Suppose I needed a TCP/IP stack in an EEG monitoring machine. I could:

    (1) write my own
    (2) buy a commercial off-the-shelf stack
    (3) buy an open source stack (e.g. from Caldera)
    (4) download an open source stack

    In all four cases, who bears responsibility for validating the software? I do!

    In all four cases, what happens if the original author (me, Wind River Systems, Joe Open-Source Author) changes the source code? Answer: I am responsible for the version of code that I choose to include in the device. Just because Joe O.S. Author, or Jane New Contributor, puts up a new version of the TCP/IP stack doesn't mean I'm going to blindly stick it in my ROMs.

    Open-source peer review is not a substitute for the giant stack of paper, code review, and testing which the FDA requires. But if I start with an open-source product then I am going to have fewer bugs to deal with and access to the source code, which is going to reduce my validation effort.

    BTW the change logs and deltas for open-source projects are usually far more accurate than the equivalent artifacts maintained by closed-source third-party software vendors.

  • If anyone's interested in starting an open-source, consortium-based clinical information system, send me some mail. If there's enough interest, I'll start up a mailing list.


    -Bill
    -----
    Bill.McGonigle@hitchcock.org / FAX: (419) 710-9745
    Dartmouth-Hitchcock Medical Center Clinical Computing
  • Well, I believe that the software itself would be liable. We, in the Open-Source community create software that is akin to a living being. It exists of its own accord, and is not controlled by anyone. As we allow these self-controlling programs to essentially create themselves, we have to become accustomed to the fact that any flaws are the fault of the program itself, and no person. I realize that this sounds a bit weird, but as far as I can tell, we are moving toward a system where software wills itself into existence, using the work of Open-Source developers to manifest itself. The software must be made, and we are there to do its work. It's a strange concept, but it makes sense if you really think about it.
    --------------------------------------------- ----------
    If you need to point-and-click to administer a machine,
  • . . . but I fear thet the FDA's historic dedication to corporate profits over health will torpedo the project.
    Which brings me back to the basic question on the FDA: why do we have a government agency ruling by fiat on matters that should be resolved within the _open_ scientific community?
  • In a way, this reminds me of the debate over sattellite phones. Since the FBI doesn't know how to listen in, they want us to not use the phones. The attitude seems to be: In this case, since our litigation happy society won't know who to sue, let's use some buggy, proprietary software. So what if a few people die, we'll be able to sue Microsoft for big bucks.

    This is stupid reasoning, people need to get it into their minds that sometimes when bad things happen there just isn't anybody to sue. My brother has a congenital heart defect, who is he supposed to sue, God? If he did, where will he connect the money from.

    It would be tragic if some had to people die because open source is too radical for the medical establishment.
  • I find it hard to imagine that this is an argument against Open Source. I'm confident most commercial software packages for the medical industry already have disclaimers against this sort of thing so what would be the difference?


    Mind you, I'm not an expert on this type of thing. Anyone have any details on when the last time a software company was sued because of failure of their product?


    BTW: There's a harsh thunderstorm going on and my router's still up! Maybe MS should buy a Netopia ;)


    AC

  • That maybe true, but the fact still remains, there is no one to sue. =] I'm almost thinking, in a situation-critical, I'd want to see something like a piece of software, open-sourced, sure, but owned by a company who supports it. Therefor that company still makes money off support, has the community to find flaws, has the businesses to get it free, and can get kicked in the butt when something goes wrong. =]

  • Whether this is a good thing or a bad thing, the computer industry has a long tradition of not being accountable for software (even commercial) killing people.

    In a computer ethics book (don't have the title handy) I read, there were some articles about a software controling the doses of radiation applied to cancer patients in therapy. Apparently, the software was buggy enough that it would often give the wrong dosage. Sometimes, the patient would get no radiation and in a few cases, patients were zapped with something like 100 times the amount of radiation they were supposed to get. There were a few deaths, but I don't believe the software manufacturers were ever held responsible.

    I don't see why open source systems should held accountable when commercial products aren't.

  • I love Free Software. In my eye, it's the only way to go. However, I want the person writing the software controlling my heart to be paid to do so. And I want it tested for twenty years before they put it in me, or hook me up to it, in every possible condition, so that I don't die.

    In this case, very complex software like Linux and Windows CE and various other things *cannot*be*used*. As good as the software may be, it isn't infallible - and if anything has to be bug-free, it's medical firmware (well, and military firmware, stuff in plans and the like). Releasing the software as Free Software, freely available, probably wouldn't do much for you - after all, how many people will have this sort of equipment?

    That being said, there's no inherent problem with releasing the firmware code of a medical appliance. You could probably get *some* improvements. But I don't want my heart being driven by software that is potentially not completely and utterly tested to the fullest extent. I just don't think that someone not being paid would be willing to do that.

  • Seems to me like freedom from a chain of responsibility is just what we need. Besides, it's not so radical a concept to have nobody to blame. Think "act of god." When an unexpected blizzard stops business, damages property, and takes lives, you don't see a whole rash of lawsuits against meteorologists for not warning of the danger, or the state and local authorities for not providing enough emergency support to prevent catastrophes. Nobody sued the builders of I-880 for building on sand that could liquify in an earthquake.

    I know that's not exactly analogous to open-source, which is much more human-implemented. But it's useful to think of open-source software as a sort of natural resource, like sunlight. For Joe User, it might as well be the same thing, cause there's no ownership attached and it's infinitely renewable. And, much like the sun, you have to be careful when you use it. Test things out. Extensively. Convince your (suable) corporation/hospital that the product is sound. Then use it. And if it fails? Yeah, they can sue you for not testing it more thoroughly.

    Here's a still different analogy that may work. Let's say all the tobacco manufacturers are being sued for giving people cancer caused by their cigarettes (I'm ignoring the whole adding-addictivs-and-carcinogenic-chemicals-with-f ull-knowledge argument for the moment). The tobacco companies can't turn around and sue the tobacco. They knew there were risks associated with using it. But they marketed cigarettes as a safe product. So the burden of error is on their shoulders.

  • This is a JOKE! Almost every license for proprietary software removes your right to sue for damages. I ran the Microsoft Windows Refund Day newsletter, and I've read many a EULA now (though I've still never used their software). The rights you sign away by using proprietary software are mind-boggling.

    Just remember: open source licensing takes away fewer of your rights as the software user. This is a demonstrable fact.
    --
    I noticed
  • When it comes down to it, EULA's really don't do anything. Neither does any agreement when you sign your life away. If a 747 uses NT (well, they're asking for it) and for no apparent reason the plane "blue-screens" (ie, crashes into the ocean) you can bet your bippy Microsoft's gonna have to shell out some mad hush money.
  • How can you get the necessary source code audit without open source?? And if you aren't auditing the source code, you're going by guess and by golly.
    -russ
  • One of the bugaboos which keeps raising its head whenever we advocate the use of Open Source Software is the question of who to blame when something fails. In a very real sense, most of us can laugh this off because the potential for dire consequences is so low: if some weird confluence of events causes my Apache httpd process to go belly up... so what? There are ten of them running, and if they all go down, I can just restart the server. The odds of something really bad happening - say, losing all of the documents in the docroot - due to a glitch in the software are remote enough to not be worth thinking about.

    But how would we react if it actually happened? Personally, I'd stand around looking dumb for a few minutes, then catch a plane for Argentina before my boss caught me. And in that situation, no one winds up dead. (Unless said boss catches me.)

    I think, however, that the "what's it gonna take" question is pretty easy to answer, at least in the medical field the article discussed. What it's going to take is a dedicated group of programmers and engineers, willing to incorporate and put out a product which runs on Linux, but for which the company is responsible. In the same way that I would hold Penguin Systems liable if they sold me a server which was defective (not that this has ever happened), the manufacturer of the system would have to shoulder the responsibility for faulty design of its own systems.

    So, what do you think? Could this be made to work?
  • Having no one to sue is not a problem with open source. It's a matter of having someone accepting liability. Accepting liability is a responsibility that someone can assume in exchange for a fee, same as any other responsibility, like the responsibility to provide tech support.
  • This liability thing should be a non-issue.

    If I tell you that I'm not guaranteeing a particular function for anything (or even the entire program), then I have explicitly removed myself from any liability.

    If some dummy were then to use that program and somehow be harmed by it, that would be their problem.

    This will lead to companies that *could* guarantee their product. It would also lead to certification companies that could take on the liability as part of their business model.

    Or the consumer could just suck it up and use the program anyway after deciding that they didn't want to pay for a certified version.

    As for the Open Source vs Closed Source question, the same thing would apply. The author could explicitly say he is liable for his particular version, or after the software is produced it could be certified, where the certifying agency would take on the liability.

    One thing they were afraid of was that users would be slipping in the newest and latest bug fix which could crash the machine. Er? And why could this not happen with a closed source model? Obviously, if the vendor was responsible for slipping in the new release, they'd be responsible (or the certifier). Unless they perhaps set rules, such as that the machine could only be upgraded when a patient wasn't on it. Then whoever did the upgrade would be liable.

    I just don't see where Open Source would be a problem in any of these cases. Even if the Uniform Computer Information Transactions Act doesn't pass, it should be implicitly obvious that there are no guarantees about the software, except those stated by contract.

    Ger

  • Ho hum... (Score:2)

    by jd ( 1658 )
    If a hospital uses software in blind faith, without some kind of testing - at least to see if it'll even work on their machines, and to familiarise staff with it - then I don't think it is the software house who needs to fear being sued.

    On the other hand, if the staff are familiar with the package, have checked to see if it'll run on their computers, and are comfortable using it, then you might expect them to compare the results they get with those they expect.

    If the hospital is confident the software behaves as it's supposed to, and the staff are confident in it's use, then the chances of there being any accidents to sue over become sufficiently remote to make it stupid to make that a deciding factor.

  • But why do you think that OS software can't be as thoroughly tested as commercial software? If I'm running a hospital OR, I would sure as hell test and software I'll be using, commercial or OS, before I hook it up.

    Something else to consider...what would you trust more, software developped by volunteers where no-one is making money or software developped by a commercial firm who might feel pressure to push a product onto the market quickly. Or worse yet,would you trust systems from a commercial company that may produce mediocre software, but have a fantastic marketing team?

    In some ways, like product honesty and openness, OS is probably *better* for medical software.

  • Hey, software is just as much a part of devices today as any other. Why should there be different standards?

    If I produce a valve for the Narcomed6000, don't test it adequately and it fails (killing the patient), I'm responsible. If my company produces jet engines that fail in flight and cause a crash, I'm responsible. In both cases I'll get sued. How is writing code that fails to sound an alarm or screws up a HUD any different?

    There's a good reason why it costs millions to get a device FDA approved, ditto getting a plane in the air. I used to work for a drug company- it's in the 10-100s of millions there for a single drug. Why? Because people will die if you don't.

    Open Source is no magic panacea. Bugs just don't go away- you need extreme levels of testing for products like these. If you can't pay for that level of testing as well as afford the insurance on such a product, then don't play. There's a good reason why there are no Open Source drug companies, and there never will be.

  • ..write code for nuclear powerplant after a meeting... ;)))
  • Whether or not medical software is open or closed doesn't matter to much. What does matter is that it works 100%. The anesthesiologist doesn't care if it's Morally Superior(tm) Free Software, or Evil Capitalistic Proprietary Software, just as long as the decisions it makes won't kill his patient.

    I work in the Medical Hardware/Software industry. When the FDA discovers a bug in the software, you MUST fix it. You don't have the luxury of hoping some other hacker might figure out a solution. You don't have the luxury of pondering what to do. You WILL fix it, and you WILL fix it by a certain date, or your software will no longer be used.

    The FDA doesn't care if your license announces the lack of warranty. Neither does the anesthesiologist. Neither does the patient. It will be demonstrated to work right the first time it is used in a clinical setting, or it won't be used at all.

    Because of these "rules", some Open Source project models won't work at all with medical software, and the typical "release it as GPL and post it on the net" is one of them. However, something similar to Apache's "gather a group of experts and limit membership" model would work. The FSF be damned when it comes to a patient's life! I don't want medical software to be "free"! I want the software under the developer's shackles and chains! I only want experts working on it. I don't want something that's Joe Schmoe's weekend hobby. If the developer needs to use a proprietary libary, I want him to have a license that lets him do it. Ditto for air traffic control software.

    I'm sure that there's a Free Software development model that will work for such projects. And I am sure that there will be several such project. However, no proprietary closed source software that saves people's lives will ever be Evil in my book, ever.
  • I believe that things like medical software do not include a disclamer of liability. So yes in that case you *COULD* sue if you wanted to.

    Just for the record many avation related things would have the same problem. You can't install anything in an aircraft without it being aproved by the FAA. For more or less the same reasons. This could cause some problems for Open Source General aviation Software. Which is annoying as I had been thinking about writing some.

  • Open Source is already being used in at least one major International Pharm. where FDA audits are a regular and accepted thing. The argument for using it revolves around accepted risk, the same kind of accepted risk you have when using software from a defunct company, or distributed by the gov., or, ultimately, bought from a company whose lawyers have made an airtight escape clause from any and all responsibility. The company in question performed it's own QA on the software, made documentation for it's SOPs and then used it. In other words, it accepted liability for it (though it does help that Samba is used by many other major companies). There was less risk in using Open Source then using in-house because the level of testing and feedback from the Open Source community is so much higher than in-house. And anyway, the precedent is already set in most companies that use any kind of UNIX and use any of the Free Software that comes with it (sendmail, vi, etc). It all comes down to accepted risk (and if the FDA is involved, documentation stating you are aware of the risk, have analyzed it, and accepted it for these many good reasons).

    One last note, the FDA does seem to distingish between device/embedded code and other types of code. The critical functionality is probably the reason. Although even a little wordprocessor problem, say one that moves a decimal point, is just as lethal the FDA is less worried about that. An arythmic heart machine vs. misrepresented dosage... I'd hate to have to make those decisions.
  • No thy don't. If boeing chooses to use NT in their planes and goes wrong, boeing is to blame not MS. After all MS explicitly says in their license that they don't accept responsibility.

    Actually I'm not sure what MS licences say in particular about such things. But if you take SUN's license for Java, for example, it explicitly says that their software is not intended for controlling airplanes, nuclear plants etc.

    Greetings, Jilles
  • Um... ok...

    Software wills itself into existance because there is no single corporate entity which claims authorship / ownership? Ummm... I don't think so... I think that the OpenSource programmers who write the code will it into existance...

    However, let's say you have a program which, through methodologies of evolutionary computing, is actually self-controlling in a reasonable sense of the phrase... (Though, as humans wrote
    the EC code, not self-creating...)
    How does a computer program accept responsibility?
    What punitive measures would a computer program be subjected to in the case of failure?
    How does a human or business entity or goverment extract compensatory damages from a piece of software?

    If a patient dies due to the failure of a machine, the hostpital faces some kind of punishment, and if there is nobody to pass it on to, they are not interested in using that machine.

    The authors of the program could be sued, but they probably don't have enough money, or any binding contractual liability as to the performance of the program, to be worth suing...

    However, OpenSource medical software could be put into use through a RedHat-like software company which would verify the software, accept liability if it failed, provide tech support, etc. They'd, obviously, charge for this... but so does RedHat...

  • I work as a programmer for a Medical Imaging company (InSight Therapeutics), and I feel exactly as you are.

    Before a medical device is FDA approved, it has to go through a lot of testing, the fact that millions of users COULD HAVE tested it is not enough. There has to be defined testing schemes/plans, and extensive documentations of the testing results, bug reports, bug fixes, etc...

    Each new component has to go through a design review before implementation, and then an extensive code review. If either of those reviews is not satisfactory, you'll have to redo your work. Those reviews are not easy, and a lot of time a programmer has go back and reimplement some part of the code.

    All the work has to be documented for regulatory porpuses, incl. all the formal/informal meetings, and we even have to save our notebooks. How will you collect all this information from the harddrives of 10s of the people that helped that OpenSourced project? How will you enforce them to save all the papers they sketched on?

    Moreover, the workers should have a lof of knowledge in computer science, mathematics and physics. My boss interviews 100s of job applicants a year, but we only accept one new member in few months. In OpenSourced project you cannot check the background of every new programmer/alg. designer and confirm that he/she knows what he/she is doing.

    We spend a lof of time talking to M.Ds, trying to understand their needs, going to operations, and trying to invent new MMI components that will work best in ORs. I'm afraid that OpenSource members won't be able to reach those Doctors. I'm not saying that it will be impossible, or that people that participate in OpenSourced projets are not as intelligent as people who work for software companies (hell, most of them do work at a software company), only that I'm not sure whether the doctors will work with them.
    I KNOW that if I had to set the meetings with the M.Ds myself, I would not have met anyone since I don't have the time for all the administrative work needed to perform those meetings.

    Before working for InSight Therapeutics I worked as a system programmer/administrator for Unix machines, and used a lot of open sourced software and helped some too. I just do not think that medical software is the right niche for OpenSource.
    I am a big OpenSource fan, my home computer has only Linux (Debian) installed, and most of the programs I use (all but two) are OpenSourced.

    Half a year ago we had to choose the platform for our next product. I enthusiastically offered Linux, and after a long discussion it was decided not to use it since the FDA might not approve the product.

    BTW, is there an FDA approved product that works on Linux (I'll be able to use it as an argument for the next time we have this discussion).

    Anyway, there are a lot of opensourced projects that need more help. If you have some time on your hands help those instead of statring new ones.

    Lets hope that none of us will have to actully see the devices I work on in OR...

    Enjoy (healthy) life,

    Liran.
  • Something else to consider...what would you trust more, software developped by volunteers where no-one is making money or software developped by a commercial firm who might feel pressure to push a product onto the market quickly. Or worse yet,would you trust systems from a commercial company that may produce mediocre software, but have a fantastic marketing team?

    How can a medical company rush to marketing, when it has to go through about a year of FDA related procedures?!?!?

    The company cannot afford to release another version while in the FDA approval process, since the process has to begin from the beginning!

    Another problem is that only a paid Q.A team, that its manager spends WEEKS/MONTHS designing testing plans can perform all the needed testing.

    I work as a programmer for a medical imaging company, and we spend A LOT of time ensuring that our software/hardware runs without flaws. I do not think that OpenSourced project will be able to enforce all the needed procedures.

    Another point is that FDA approval takes time. We have special people that all they do is interact with the FDA. I do not think that OpenSourced project's members will have the time/knowledge to interact with FDA.


    Liran.
  • I work as a programmer for a medical imaging company.

    My experiance shows that the hospitals test each new product at least two weeks before perofming operations on animals and then humans anyway.

    The staff at the hospital does not have the time to get to know the packages. If the product was not well tested before and/or it is not FDA approved, they will simply won't tauch it!

    There are some safty concerns that I'm not sure all OpenSource programmers are aware of.

    Liran.
  • For example, how many of you know what DICOM is?

    There are a lot of other examples, but I don't want to waste your time...

    Liran.
  • When was the last time you spent some time in a modern hospital?

    Nowadays all the information is transferred over the network. Our products cannot work unless connected to the hospital network.

    You should do some reading in the subject before writing about it:
    Read about PACS [pacsnews.com] and DICOM [psu.edu]. I'm sure that there are better links to either of those, but I'm not at the office right now and took those from google.com.


    Liran.
  • Thanks for the links.

    When was the last time you spent some time in a modern hospital?

    Last night. And the night before that, and the night before that, ad infinitum .... :-(

    Nowadays all the information is transferred over the network. Our products cannot work unless connected to the hospital network.

    Of course, the scanners are networked. But how much does this help people other than radiologists? Some hospitals are installing radiology "terminals" to make images readily accessible to all providers, but these are the exception rather than the rule. The same applies to electronic medical records. The healthcare industry is way behind in IT - partly because of reluctance of doctors to learn new ways of interacting with the medical record.

  • Somehow I don't think medical software is going to get a good push from the open source community. Making a big medical application/database is not exactly "fun" work akin to open source developemnt. Most successful open source projects are for productivity/desktop apps that the developers have fun with and will actually end up using in the end. I don't see that happening with a piece of software that tells a nurse when to give a patient their medicine.

    "The voices in my head say crazy things"
  • Awesome comment. A very nice way of putting it.

    Ger.
  • I believe that the tort system, as it applies to
    product liability today, has (like medical mal-practice) lost any corrective function it had. Any unfavorable result is assumed to be reasonably preventable, and the participants therefore liable. Yes, it's nice to have someone to kick in the butt, but what's the point?
  • Oh, please.

    Software for critical life-care situations isn't some web application that can crash and you just hit the reset switch.

    There are design methodologies for high reliability software. And it isn't just throw it out into a bazaar and let a bunch of hackers pound away at it.

    There are controls, oversight, audits. Everything is documented, and virtually everything is traceable.

    Hint- people like it that way. You don't, but we'll give you a few years to catch a clue. You won't be let near any critical life care development anyway, with your attitude.

  • Medical software developers get sued constantly. If a patient dies because the software controlling the MRI is faulty, that manufacturer will probably wind up out of business. I work in the medical hardware/software industry. We haven't been sued yet, but some of our competitors have. And we are always disheartened when it happens. Our latest product was released to manufacturing after one and a half YEARS of software testing. We fear that there might still be an undiscovered bug that could lead to a patient dying.
  • It is important to supply open information regarding:
    1/ Specification methods used
    2/ How the software was validated and verified

    Aircraft software is far more complex than medical software and many more lives depend on just a few computers. Sure it's incredibly expensive... But let's apply rigorous CS methods to life-depending software before releasing the code and all data as open source.
  • The software testing department in the medical equipment company where I work can halt the shipment of systems. As one of those testers, I can pull the plug (pardon the pun) at any time. I might have to explain myself, but they can't go over my head without a full hearing.
  • I think that Free Software can, normally be tested in most situations to as well as you can get it - much better than commercial software. But, I don't think that, even as good as Free Software testing is, that it's good enough to run, say, the firmware of a heart-and-lung machine. This sort of software has to be tested in *every* possible case, and it costs a *lot* of money to do extensive Q&A on them. I'd be surprised if the firmware in those sorts of machines is younger than 10 years.

    Like it or not, the regular sorts of development, be it Free Software development or proprietary development, just isn't good enough for most medical software. It's not data or money you're toying with there - it's people's lives. *I* wouldn't trust my life to Linux - would you? (I should specify - I wouldn't trust my life to pretty much any OS. Add a layer of abstraction, you add more things that can possibly go wrong.)

  • I've spent a lot of time thinking about this. Last September, my wife had a focused radiation treatment to her brain. The software ran on HP/UX machines (HP makes a very reliable Unix, although it isn't as pleasant to work with as GNU), and while I was chatting with the doctors, I noticed that the software version number was 4.0b3. BETA SOFTWARE? Well, it turned out that the beta test was just for an unimportant feature, but even so, it made me pretty nervous about it.

    Later on, I found out how lucky we were to have the software running on Unix; there are stereotactic radiosurgery systems running under NT, which gives a whole new meaning to "blue screen of death".

    Anyway, the question is this: would open source development benefit software that controls radiation machines? I don't think it would. Open source relies on the ability to make mistakes and correct them; medical software relies on not making them at all on live patients. That's why medical software goes through highly formal testing before it's released, and why open source software goes through alpha- and beta versions. However, that's not the whole story.

    Especially in America, people are prepared to pay big bucks for anything related to health care. And the systems in use are reliable enough. Since open source development competes on the "cheap and reliable" ticket, I don't think it's going to make anything any better. However, it is definitely useful in two areas:

    1. Infrastructure. Hospitals seem to want to use Microsoft Exchange and NT for server software, and the frequent failures make it harder for people to do their jobs. Open source could work well here.
    2. In poor countries, open source software could compete on cost, but probably not for software that controls expensive equipment.

    Bottom Line: Open source works better when there is a large developer base. I don't think that's the case for medical software, and I don't think it would work. I wish Stefan Harms all the best -- and he's chosen a project with a better than average chance of success -- but I don't think the medical profession would benefit from open source software.

  • GPL'd software comes with no warranty. However, this does not prevent a company selling GPL'd software and including a warranty. This is just one of the ways that one can make money selling Free Software. If you are able to provide some kind of accoutability for your customer, and that is what they are looking for, you can do that.

    (assuming you are talking about GPL software, otherwise refer to your appropriate license)

    I would mention how this compares to proprietary software but it has already been discussed.
  • But, I don't think that, even as good as Free Software testing is, that it's good enough to run, say, the firmware of a heart-and-lung machine.

    A ventilator is actually much simpler than emacs, mozilla, linux, gnome, etc. Air goes in, air comes out. Air did not all return=>sound the leak alarm. Pressure too high=>set off high pressure alarm. And so on. Even so, there are checks and balances in place becauase it is unwise to trust everything to a single machine. That is, even with all of the testing, they not uncommonly malfunction or fail (sometimes software, more frequently mechanical). Thus, the ventilator does its thing while oxygen saturation and CO2 monitors monitor the results of ventilation and set off alarms when things don't look right.

    Now there is much more complex medical software - software that attempts to read EKG's, for example. It is generally not trusted. The EKG will be read by a physician anyway.

    Most medical software applications. Visual navigation software for the operating room is complex, and will be a major application of VR. It shortcomings (registration errors, shifting of structures) are very well known by the people using it.

    It's not data or money you're toying with there - it's people's lives. *I* wouldn't trust my life to Linux - would you?

    Well, you may find youself trusting your life to NT, IRIX, or even DOS :-) - not just in the medical realm.

  • As someone who working in the Medical Industry, I can tell you the 'closed source' stuff is more buggy than 'open source' stuff.

    Lots of FUD in the comments going on here...

    With closed source, I'm stuck relying on the single source for fixes. Whoever that might be.

    With open source, if need be, I can hire someone to fix the problem, or get the fix for free, because everyone watches out for the problems together.

    I'd trust my life to something GPLed with far more confidence to something with a EULA like Microsoft's. Ever notice that anything with Java isn't safe for use in Nuclear Plants, Medical equipment, or other 'dangerous' environs?

  • The code in closed source medical software is but through a GREAT deal of scrutiny. In a lot of cases, the FDA reviews the code line of line. Even if they don't, they do demand detailed design and spec documents, thorough testing records, proofs of correctness, etc. And the European equivalents of the FDA are even stricter.

    If this software were the type that you could take and run on any equivalent medical device, then it would be a much better candidate for open sourcing. However, the software is always an integral part of the hardware. One will not work without the other. Asking a medical equipment company to publish its source code would be equivalent to asking them to publish detailed blueprints for their hardware.

    You talked about a "standards body". We already have one in the US. They're the FDA. If you wish to privatize the FDA, I'm with you 100%.
  • For example, how many of you know what DICOM is?

    A Radiology file format. I've been looking for some open source code to manipulate these files - do you know of any?

  • "My experience is that medical software is somewhat higher quality than non-medical commercial software, but not as good as the best open source software."

    That's an interesting statement, considering that there are not any open source medical applications, yet.

    It also assumes that medical software will have the same pool of millions of developers and users to fix and correct the software. This is not the case. It's entirely possible that a given company may already be employing a full third of the available qualified developers for that application!

    Okay, here's the code for the new $5,000,000 BioTronics9000. Are you going to download and use it? Can you even understand what the code is doing? Are you even interested?
  • ... then I will read the code. Yes. I am interested, I can understand it, and I presume you can, too.

    You are misreading my statement. Look at the noun phrases : "medical software", "non-medical commercial software", and "best open-source software". Yes, I am comparing across domains. But medical software has configuration parameters, data storage, user interfaces, and (these days) TCP/IP networking, there are plenty of overlapping sub-domains to compare.

    You, and a lot of other people here, are also confusing "open source" -- software whose source code is available to everybody -- with "volunteer software" -- software written by unpaid people.

    I am looking forward to the next red herring, where someone compares "medical company tests its software" to "open source volunteers review the software. That's a false comparison. The real comparison is "medical company tests its software" versus "medical company tests its software PLUS independent volunteers review the software". In other words: A + B > A, if B is positive.
  • I think people have to stop thinking of 100% reliability as a feasible goal. After all, malfunctioning cars kill people every year, its a price of using cars in our society, and though there are lawsuits no one really expects cars to be 100% reliable.

    IANAL, but a friend of mine who's had some law school told me the operative phrase was "honest best effort". You can be sued for negligence in an accident if there is something you should have done to prevent the accident, but didn't. This has to be the test of liability, not the undeniable fact that people were damaged.

    Large software is just too complex to test all cases, no matter how many years of test cases you generate. There are often assumptions during such testing anyway, such as the hardware CPU being without faults. If the chip glitches and the software doesn't handle it, who is at fault now?

    I see medical software being a big problem for BOTH proprietary and open source models. I think open source definitely provides some advantages. The number of trivial suits is likely somewhat decreased because there's no real money to go after, OSS is not rich. From the customer's point of view, at least the entire codebase is available for inspection. This should avoid letting rushed-out-the-door spaghetti code loose on a patient! It also means better accountability because if the software vendor did something really stupid that they should have caught, you can prove it. Finally, it would make it far more difficult to settle software liability suits with a monetary reward and non-disclosure agreement - the fault is there for anyone who wants to look, and will be fixed. This last point, that bugs WILL be fixed when found, is what really matters, IMO.

    Just one last point though. Medical software bazaars can't be anything like Linux. The emphasis has to be purely on bug avoidance and fixing, not on feature rollouts. No itch scratching here, sorry. It would also require far more management and review in terms of which patches are allowed, so could possibly be rather frustrating to work on. Once released, even if a feature is broken it might be better to disable it than attempt to fix it, unless it is truly critical. This isn't how Microsoft does software either though, that's for sure. :)

    Jim


  • All the work has to be documented for regulatory porpuses, incl. all the formal/informal meetings, and we even have to save our notebooks. How will you collect all this information from the harddrives of 10s of the people that helped that OpenSourced project? How will you enforce them to save all the papers they sketched on?

    Archived email and newsgroups (alt.frameless_stereotaxy.modules.coordinate_trans form, etc) for discussions. A CVS or other database to trace development of the source.

    The sad fact is that the medical industry is slower than other industries to adopt new technologies or pardigms. Computerized medical records? A few places have them. Most do not. This is like trying to run a bank or stock brokerage with file cabinets. Yet, this is how it's done in most hospitals. Your medical information or xrays cannot be summoned over the network - they are stuffed in dimly lit shelves in the basement, often out of order and difficult to retrieve. Or they are checked out to another hospital or doctor's office. Do you think that your doctor will know about your life threatening allergies or other medical conditions when you roll into the trauma center unconcious? He has your drivers license and SSN from your wallet, after all. Nope. Amazon.com or E-Trade knows more about you, because they have your records in a database. OK, enough flaming about that.

    I would like to see all medical related software and file formats become open source, simply because the power of peer review will make it safer. This should apply even if the source was developed by a private company and is proprietary. If John D. Hacker's wife is on the ventilator, he might just sit down and look over its code. Maybe he will find a bug and submit a fix to the device company.

  • Thanks for the answer. I pretty much figured that this was the way it [sh|w]ould work out, but wasn't aware that there were already companies out there using open source software.
  • You will have problems to sue someone when there are very critical areas.
    Think of a company writing software for nuclear power plants. When there is a "relatively" small failure, lets say 1.000.000 have to leave their homes for a while for security reasons, there are just some companies which could pay and stay in business.
    So - thats what insurance-companies are for already nowadays.
    This could be a great opportunity for a new kind of insurances, perfectly fitting to open source.
    A insurance-company takes some piece of open source software, reviews it (and the target hardware) and distributes it together with an insurance.
    Thats the same system as today, you have only eliminated the man in the middle, the software company (who needs them anyway? :>).
  • I used to work as a software tester for Win32 applications. I was wondering what OS/platform your company uses for your medical software? What kind of test tools do you use? Did you use off-the-shelf test tools or custom tools? My team had access to cool tools like Purify and BoundsChecker, but nobody (but me) cared about the results. Part of the whole "Let's just ship this sucker" attitude.
  • Risks Digest [ncl.ac.uk] has been chronicling risks to the public through computing for quite some time. It would be interesting to see how many cases involved law suits.
    --
    I noticed
  • Check out The Case of the Killer Robot [amazon.com].

    (Note. I'm biased. The author is my advisor.)
  • It's a matter of having someone accepting liability.

    Bingo! After all, this isn't a case of hospitals buying off-the-shelf PCs, installing slackware, and then perusing Freshmeat for MRI control software. Generally you get a package of machine plus software, and the software development/testing costs are part of the cost of buying the machine. Open source isn't really an issue, since you aren't supplying source for someone else to debug, and you don't have to worry about someone else releasing a CD with your software for $1.99 -- they still need the machine. Now you might snarf some algorithms off the net, test the hell out of them, and then incorporate them into the MRI software, but the Free Software Bazaar isn't going to start receiving offers to write controllers for radiation emitters any time soon.
  • Previously, discussions of software liabilty were a non-issue. Closed-source developers (such as Microsoft) disclaimed liability, or, if they were feeling especially generous, limited liability to original product cost.

    I can't confirm this, but I suspect the only vendors who depart from this policy are value added resellers and those who sell support-contracts.

    The GPL just is operating from the philosophy-- "Look, you can't really sue anybody in the software world, so why should you be able to sue us?"

    But in the medical devices arena, vendors are responsible for what they distribute, and can be liable for damages in the thousands, or millions of dollars.

    An open source model may be appropriate, but unless some company can take the source-base, internally validate it, and provide signed, certified releases, no insurance company will want to touch it.

    Frankly, the avoidance of risk has been one of the major reasons medical costs have been steadily climbing. Many medical devices, however complex, are disposable, because sterilization can never be 100% effective.

Slashdot Top Deals

"More software projects have gone awry for lack of calendar time than for all other causes combined." -- Fred Brooks, Jr., _The Mythical Man Month_

Close