Open Source Software For Experimental Physics? 250
jmizrahi writes "I've recently started working in experimental physics. Quite a few programs are used in the lab for assorted purposes — Labview, Igor, Inventor, Eagle, to name just a few. They are all proprietary. This seems to be standard practice, which surprised me. Does anybody know of any open source software intended for scientific research? Does anybody work in a lab that makes an effort to use open source software?"
Fermilab... (Score:3, Funny)
https://fermilinux.fnal.gov/
Re:Fermilab... (Score:5, Informative)
Actually Fermi's Linux is their build of Scientific Linux, which is a distro they have in collaboration with CERN and others.
It's originally based off Redhat/Fedora IIRC. Or department uses it.
Re: (Score:2)
It's basically a customization of Red Hat Enterprise Linux.
Re: (Score:2)
I would never download a distro after clicking through on a splash page that lists five discontinued distros (or just name changes?) from the same people.
What gives?
Re: (Score:3, Informative)
Re: (Score:2)
Re: (Score:2)
Your Reqs Are Too Specific, Try R or Octave (Score:5, Informative)
Does anybody know of any open source software intended for scientific research? Does anybody work in a lab that makes an effort to use open source software?
We discussed [slashdot.org] R [r-project.org] which describes itself as:
R is a language and environment for statistical computing and graphics. It is a GNU project which is similar to the S language and environment which was developed at Bell Laboratories (formerly AT&T, now Lucent Technologies) by John Chambers and colleagues. R can be considered as a different implementation of S. There are some important differences, but much code written for S runs unaltered under R.
...) and graphical techniques, and is highly extensible. The S language is often the vehicle of choice for research in statistical methodology, and R provides an Open Source route to participation in that activity.
R provides a wide variety of statistical (linear and nonlinear modeling, classical statistical tests, time-series analysis, classification, clustering,
One of R's strengths is the ease with which well-designed publication-quality plots can be produced, including mathematical symbols and formulae where needed. Great care has been taken over the defaults for the minor design choices in graphics, but the user retains full control.
While it's not geared specifically towards experimental physics, that's probably going to be your most fruitful endeavor.
Then there's the Matlab knock-off [slashdot.org] of Octave [gnu.org] which describes itself as:
GNU Octave is a high-level language, primarily intended for numerical computations. It provides a convenient command line interface for solving linear and nonlinear problems numerically, and for performing other numerical experiments using a language that is mostly compatible with Matlab. It may also be used as a batch-oriented language.
Octave has extensive tools for solving common numerical linear algebra problems, finding the roots of nonlinear equations, integrating ordinary functions, manipulating polynomials, and integrating ordinary differential and differential-algebraic equations. It is easily extensible and customizable via user-defined functions written in Octave's own language, or using dynamically loaded modules written in C++, C, Fortran, or other languages.
I'm surprised you're surprised that you only find proprietary software in the highly specialized realm of "experimental physics." I mean, you have to be like a PhD in physics with a good deal of programming knowledge to make something accurate & useful (and there's probably gotta be like 50 failed projects before you get a good successful one).
... I have no idea what Labview, Igor, Inventor, or Eagle do. Ask yourself why these programs are standards and then maybe add to Octave's wish list or contribute to it even! Unfortunately, this isn't easy--I myself started to implement proper handling of sparse matrices in Octave but gave up as I was trying to form low level requirements ... You probably already know though that this is going to have to be done in C or another very low level, very quick language.
You're probably wondering why there's not a project of Firefox or OO.o quality for experimental physics but I'll tell you why: it's too specialized and your user base is ridiculously small. You're not going to find a company that is going to benefit greatly (or at all maybe) by releasing their product into the wild for a community to grow. There's probably not a community for it to grow in.
You should tell us what specifically you are looking for something to do
If you're looking for something specific or outline some high level
Re: (Score:2, Interesting)
Agreed. I've know people who work in experimental physics labs. Some of them code. And the use of open source software is quite a bit more than one might think. Check out this guy's blog [phacker.org]. He regularly hacks together interesting scripts in Python, for instance.
Re:Your Reqs Are Too Specific, Try R or Octave (Score:4, Insightful)
I'm surprised you're surprised that you only find proprietary software in the highly specialized realm of "experimental physics." I mean, you have to be like a PhD in physics with a good deal of programming knowledge to make something accurate & useful (and there's probably gotta be like 50 failed projects before you get a good successful one).
To me, that makes it all the more important that ones work is distributed as widely as possible so that others may build upon it.
For Great Success, Look at It Realistically (Score:2)
To me, that makes it all the more important that ones work is distributed as widely as possible so that others may build upon it.
Hey man, my heart is behind you 100% in this respect. Unfortunately, ideals/maxims/truth are not what create change/progress in today's world. Money/greed/market have replaced them. He was a great leader but Gandhi would not be a CEO today. I don't think it's a bad thing. Sure is unfortunate in some respects. But modern day people operating on the former usually get laughed at/written off. Look at Stallman!
So I unfortunately have to point out that if you approach your boss (or one of these compa
Re: (Score:2)
So I unfortunately have to point out that if you approach your boss (or one of these companies) with a business proposal yielding a net gain of karma or helping society without a tax write off ... you might be written off from that point on.
Nope. As long as we get a good paper out of it, giving it away is no problem. In fact, the more people who use the results of our work, the better off we are. Science only works if we can build on each others results. And this is why there is a lot of very high quali
Re: (Score:2)
But he would still be a great leader. The real question is, do great leaders make great CEOs? If not, then whats wrong with business?
"But modern day people operating on the former usually get laughed at/written off."
You mean principles? Did you know that to maintain accreditation MBA programs have had to introduce a fixed percentage of ethics lecturing for *every* class? Looks like scandal and profound dismal failure in that culture are try
He means I think experimental control (Score:5, Interesting)
The programs he names:
labview, igor, matlab, are not simply data analysis tools but also have hardware control modules. You can run your data acquisition from Igor, and matlab and of course Lab view.
things like R or octave or scipy dont' have control and acquisition modules that I know of.
That said, it seems to me that since scipy is in python that any python DAQ system would fit the bill of a scientific software system.
So the real question is, what DAQ's are available for python.
and the answer is going to be, that daq softwares tend to get written to drive specific hardware systems. e.g. labview makes a whole suite of DAQ boards. there's oscilloscopes and so forth.
As a scientist you don't want to get bogged down building a custom daq. SO the real bottom line is what commercial DAQs are open in design.
The only system I know that might possibly be in the open is the OASIS daq's developed for flow cytometry. these are mass produced and were developed at the National Labs. But I don't know how it is lic.
of course, one can always use an oscilloscope or DAQ made to be controlled by GPIB or simmilar common language. In that case it's pretty easy to write your own as long as you can find a suitable open source GPIB output driver.
Re:He means I think experimental control (Score:4, Informative)
Re: (Score:3, Funny)
For those of us (like me) who were confused when the DAQ acronym started being peppered in the parent post, it means "Data AcQuisition".
Oh man, that makes way more sense than what I was thinking, "Dark Age of Quamelot".
A punny name and you're halfway there - (Score:2)
Clearly this calls for a new open-source project to be called DAQuiri.
Tragedy of progress (Score:2)
The real tragedy about this is that there is already a vast library of Physics computational tools, but few new students are using it because it's all written in Fortran. Computing has moved forward (python vs. Fortran), but the libraries don't move forward with it.
I once heard a story about a CS professor who every year assigned the translation of a Fortran library function into C to his students. I cry thinking about it, since C is rapidly being left behind as well.
Re: (Score:2)
Fortran isn't dead yet, and it executes faster than Python.
f2py (Score:5, Interesting)
Python has mad fortran all the mor valuable. the combination of fortran and python is a match made in heaven.
fortran is a wonderfully good algorithmic language but it is terrible at everything else. python is great at everything else, but slow as a dog.
one of fortrans strengths when combined with python is oddly enough it's simplicity and lack of sophistication. unlike c++ It compiles so fast that you can feasibly have python write fortran on the fly optimized for the algorithm and compile it, all during run time.
Fortran is good for non-computer scientist in part because it is quite hard for a typo to be a logic error. (e.g. = instead of ==) as a result it's comparatively easy to debug compared to C.
but it can't do all the fun memory management tricks, introspection, or objects that advanced languages do.
But with f2py you can push all of that into python and then just have it call short simple fortran routines for speed.
Re:f2py (Score:4, Informative)
Agree. However, Fortran still evolves and Fortran 2003 (and 2008) is suppose to add some OO capabilities beyond encapsulation in modules.
I will point out that I've seen more Perl than Python in scientific computing. But most of my scientist coworkers tend to be older..
Re:He means I think experimental control (Score:4, Informative)
If you want an environment like MatLab or LabView, there is Scilab:
There are many such toolboxes: DATA ACQUISITION AND REAL-TIME [scilab.org]
Re: (Score:2)
Re: (Score:2)
cool. thanks. I wish someone would make some python wrappers for these!
Re:Your Reqs Are Too Specific, Try R or Octave (Score:5, Informative)
It also depends on what is meant by "experimental physics". CERN has released one hell of a lot of physics software over time. That which is maintained and comes in a tarball is largely listed on Freshmeat, but there's also plenty of unmaintained but useful code, along with code only released via CVS. The same is true of many US experimental physics labs and NASA.
Your question simply isn't specific enough for me to identify whether you want OpenAtom, MultiBody Dynamics, Geant3, Geant4, Electron Gamma Shower, MIT Photonic Bands, a statistical language like R, visualizing software like DX4, the Interactive Spectral Interpretation System or GGobi, plotting software like Octave, general acquisition/control software such as COMEDI, data acquisition software over VMS such as MIDAS, or just a driver like the Cogent CIF Driver for Fieldbus. Maybe those aren't your cup of tea and you're more into CFD, in which case OpenFoam and NASA's FUN3D might be more what you want, along perhaps with the CFD General Notation System. Or maybe something in geophysics will help, like Seismic Unix.
Or maybe you've jury-rigged something Professor Heinz Wolff-style out of spare parts from an HVAC and need the BACNet package. Of course, data manipulation/storage then becomes useful and you'd need something like PETSc, the Vector Signal Imaging Processing Library, the Stanford Exploration Project Library or HDF5. Then, perhaps, the system you're controlling uses a modern high-speed, low-latency network, in which case perhaps DAPL would be of use. You could even use SensorWeb for general-purpose sensor monitoring.
Yes, this post is saturated with names. Deliberately so. (It could hardly be accidental.) This is not to overwhelm you, as much as it is to emphasize that Open Source science is very well established - so much so that a highly general question cannot be meaningfully answered. Maybe what you want is in the list above, maybe it isn't, but Slashdot would run out of disk space before I ran out of packages out there. Since nobody wants that, it is far more practical to give you the general idea and some pointers on where to look.
Re: (Score:2)
You're probably wondering why there's not a project of Firefox or OO.o quality for experimental physics but I'll tell you why: it's too specialized and your user base is ridiculously small. You're not going to find a company that is going to benefit greatly (or at all maybe) by releasing their product into the wild for a community to grow.
That raises another question: Why isn't there, then, a Linux or Apache of experimental physics software? Not ever open source project was born in the belly of a corporate beast which later got the Free Software Religion.
I would also say that this is exactly the case where having source code available should be a requirement. Even if it stays a commercial program, and even if you have to pay to see the source, this is exactly the kind of specialized niche which is only likely to get more specialized, where c
PDL (from Perl), GDL (from IDL), NumPy/SciPy (Score:2)
Perl Data Language [perl.org] is my personal favorite for data acquisition and manipulation - it is an extension to Perl that gives you the usual data analysis and plotting goodies, plus a nice mix of access to C code (via XS), access to the huge CPAN trove of modules (via, er, CPAN), GUI-isms (via PerlTK), and serial port / USB port access for instrument control and data acquisition.
It's based on Perl, which can either be a huge plus or a huge minus depending on whether you like Perl.
If you want bondage-and-disciplin
Not specific enough (Score:2)
There is one tool, root [root.cern.ch], that we us
Re: (Score:2)
Inventor is just 3D CAD software from autodesk, theres better software such as NX or CATIA which do run on linux, but for a hefty price.
Now there is Opencascade which is an opensource 3D Modeling kernel, but quite honestly as someone who uses NX on a daily basis, it is not up to par. I wouldn't want to waste my time with it at work, I waste enough time with NX which is written by a team of highly paid engineers.
Labview is a lot more than just a data acquisition suite. Although that is a primary purpose of
Re: (Score:2)
Of course I should add, Labview is relatively simple, for anything really complex the controller software is going to written from scratch anyways.
The question becomes, why would you opensource something that quite possibly, only you are going to use?
Re: (Score:2)
I have a love/hate relationship with LabView. I always considered it to be the Visual Basic of the scientific world.
A lot of times, it's too bloated, too resource hungry, and too expensive... But we buy it anyway.
On the plus side, it does instrumentation really well, and GI has a catalog full of sensors to choose from.
Re: (Score:2)
Generic Mapping Tools will do some really spectacular plots, and is GPL'ed software, Cygwin or Windows Services for Unix let it run in windows YMMV. It's commandline but gui add-ons are available
Re: (Score:2)
Igor -- Technical graphing for Mac & Win -- So why not GNU Plot?
I suggest grace [weizmann.ac.il] instead. It's easier to use than gnuplot, has a GUI and a CLI, can read data over a pipe and the output looks nicer.
Re: (Score:2)
The great thing about buying commercial hardware and software like this is once you have it, you have it.
And then it becomes part of LAST YEAR'S grant budget, and therefore becomes absolutely free :)
Re: (Score:2)
Re: (Score:2, Insightful)
Well, the average scientific experiment really is buggy, feature-incomplete and crash-prone. And this doesn't even include the software for controlling them. All that *really* matters is getting solid data.
Re: (Score:2)
"Feature-incomplete" and "full of bloat"? Is that not like dividing by zero?
Re: (Score:2)
It's more likely he's just trolling than astroturfing. As for posting as AC, I do that all the time, it's usually because I'm too lazy to log in.
Firefox, I can't agree with you. Not in the slightest. It crashes all the time on every platform I've run it on (Windows, Linux x86, Linux PPC, FreeBSD) and has some real annoyances like still using far too much memory (which causes horrible problems if you want to do something like LTSP).
Right now I've come to prefer things like Arora (simplicity at its best) and
Re: (Score:2)
Scilab is the GPL alternative to Matlab (Score:4, Informative)
Re: (Score:2)
Python, SciPy, VPython, etc... (Score:2, Informative)
There are tons of opensource software tools used in scientific research. One widely used language for building and extending these tools is Python.
Try a google search for +Python +physics to get started, but also try +Python Labview, +Python Igor, and so on. You will find that lots of people have built Python tools to extend the proprietary ones, or do preprocessing or postprovcessing of data.
Re: (Score:3, Informative)
Re: (Score:2)
A good Python library for making 2D plots is matplotlib [sourceforge.net].
Stay away from Labview (Score:5, Informative)
Labview has the advantage of being easy to learn for non-computer-savvy people that want to control and read special hardware (electrical monitoring equipment, servo motors, etc.). Unfortunately, it rather terrible for complex programs that do not naturally fit in LabVIEW's paradigm of deterministic data flow paths from user input to screen or file output. For example, storing things in persistent variables that are not visible to the end user are a horrible kludge. Reusing VIs (program modules) in other projects require you to endlessly draw wires on your screen rather than simply copy/paste something in an editor. Data processing that involves anything else than applying a standard library function (such as searching arrays for special conditions) that would have been 20 lines of straightforward C code will take you half an hour in LabVIEW, even if you use the "C formula node" that has no debugging facilities whatsoever. You will find yourself spending most of your time moving lines on your screen because there is no free space left on the flow diagram for that extra feature that you need to implement. Stay away from Labview if you can. Even Labview representatives will tell you that more experienced programmers tend to not like Labview so much.
I haven't used Igor myself, but I have watched over other people's shoulders. You can do quite a bit with it, but the syntax of the language is quite inconsistent; the central paradigm is that variables are either scalar or "wave", which is an array that implicitly represents a row of (x,y) values with fixed steps in x. Wave variables always have global scope, which makes functions that deal with wave variables unpleasant, especially if there is recursion.
Myself, I have been a happy Gnuplot (FOSS) user since 1996. Gnuplot is quite limited in terms of data analysis and dealing with higher-dimensional datasets, and also has a very inconsistent syntax due to historical reasons, but the latter doesn't bother since I have used it as it evolved. For serious (CPU-intensive) data analysis, I have always written C/C++ programs (with gcc, of course), using Gnuplot only for plotting.
I have played a bit with R (FOSS) for data analysis and visualisation, which I liked, but which I have never used apart from making a few drawings. Octave can be useful if you need to collaborate with Matlab people.
For hardware control, I also prefer C/C++. Unfortunately you will likely have to do that under Windows unless you want to write your own lowlevel drivers for Linux (I tried once and gave up when I had to read a 200 page document describing the PCI bridge controller to figure out how to do a DMA transfer).
(I'm out of the academic world now, working at a high-tech company where I have to work with Microsoft Office most of the time and god I do hate that...)
Re:Stay away from Labview (Score:5, Informative)
Your knowledge of Labview and Igor seems limited and out of date. Labview is far more sophisticated now that the primitive description you use. (for example, you can bundle all the wires into a single "object" wire) so there's no "endless" drawing of connections like you describe. It becomes more like drawing UML, except when you are done making the vi, it actually runs.
Igor is a very nice fast system. the command line syntax is a little archane but not terrible hard. But it has complete gui controls so you can use it before you memorize all the commands. It's not worse than GNU plot or matlab or perl data language or scipy or R in terms of having syntax oddities. it seems like all grpahics packages try to compromise between powerful short commands and completeness. (sort of the perl ethos out of neccessity to make it interactive rather than a structured program)
but it's not as good a general programming language as say Scipy. But it is more user freindly.
Labview can be frustrating because sometimes it is faster to write code. But here is one big advantage labview has over everything I have used for data acquisition. It's the only code I'm willing to modify during an experiment. it's not possible to make a syntax error, it is very hard to make big errors when simply moving or split, and you don't have to worry about the thread g wires in an existing program, and most importantly for DAQ since you dont' explicitly program the thread syncronization you cant screw it up with changes. So it's not too scarey to change a working program in the lab to do something a little extra right in the middle of some expensive instrument time.
Labview is a extremely crappy language for data analysis. It's designed for live instruments and real time work not off line anaylis.
right now I'm a big fan of scipy but it too has it's frustrations. it's wildley incomplete, undocumented, and they just changed all the 3d graphics to a much better but way too complicated system and the old programs can't be maintained with the new releases.
Re: (Score:2)
Your knowledge of Labview and Igor seems limited and out of date. Labview is far more sophisticated now that the primitive description you use.
I don't know. I use LabVIEW pretty routinely and the GP's experiences reflect my own. There's no question that LabVIEW can make setting up a new data acquisition system very fast. The tie-in with DAQ cards and many instrument control systems is fantastic. But the actual programming paradigm is terrible (at least for anyone acquainted with the power and flexibility of traditional text-code programming). Even with the various enhancements they've put (like bundling wires and all that), it takes a long time to
Re: (Score:2)
Commenting in LabVIEW only requires you to double click on the location you want the comment and then start typing... how exactly is that harder for you?
For one, you have to plan that space for the comment.
And, if you write too much comment, you have to keep scrolling around in 2D to see all of the "code".
Imagine if you had to write codes in normal languages and your "insert" was toggled on (and you couldn't toggle it back), so if you could imagine yourself adding anything additional (comment, additional code, etc.) later on, you have to plan for that space ahead of time.
That's exactly how harder it is to comment in LabVIEW.
Re: (Score:2)
This may be viewed as a disadvantage.
Re: (Score:3, Informative)
for example, you can bundle all the wires into a single "object" wire
Oh, yes you can bundle them, but that just means you have fewer visible wires; the underlying scheme hasn't changed at all.
For example, suppose you write a sub-VI and later on, you want to modify it to take more inputs. In real programming languages like Python, that would be as simple as dealing with keyword argument lists. You don't have to mess with the function definition; you just pass the information to the function and write something within the function to do something with that input (i.e. you don'
Re: (Score:3, Informative)
Try doing that in LabVIEW. When the smallest property of those bundles change (i.e. more number of wires in the bundle, or a different data type for one wire), you have to change a number of things to make the things connecting to that bundle work again.
Sounds like someone has not learned you can index bundles by name, not just by position in the struct. You are treating them like an array but you can also treat them like hash's.
But agreed it's not as simple in some cases as writing code.
On the other hand if you were writing the syncronization of multiple threads by hand your head can explode but in labview you don't even think about it most of the time.
it's a terrible generla purpose programming language. it's a great data aquiristion language and RAD g
Mod up parent (Score:2)
Thanks, nicely said.
I fight all the time with Labview weenies. It does have a place but for complex jobs interfacing with other programs or systems it becomes cumbersome. One old timer senior scientist where I work refers to Labview as programming for kindergartners.
Re: (Score:2)
[humor]
Probably for as long as LabView exists. If the PL/1 versus Pascal war is any indication, you have a long wait ahead of you.
[/humor]
Re: (Score:2)
Labview has the advantage of being easy to learn for non-computer-savvy people that want to control and read special hardware (electrical monitoring equipment, servo motors, etc.). Unfortunately, it rather terrible for complex programs that do not naturally fit in LabVIEW's paradigm of deterministic data flow paths from user input to screen or file output. For example, storing things in persistent variables that are not visible to the end user are a horrible kludge. Reusing VIs (program modules) in other pr
Re: (Score:2)
Re: (Score:2)
Coming from a more traditional programming background (I was a computational physicist before I was an experimental physicist), I had many of the same problems you have with Labview. There are very few people in academia (and no classes) which I feel properly use labview, but I was lucky to have someone recently back from industry to show me examples of what Labview should do. Everyone is caught up in hooking up hardware and learning the basic program design that most academics learn horrible, horrible pr
Qtiplot & root (Score:5, Informative)
For simple tasks, even with big data sets, I've had good results with qtiplot http://soft.proindependent.com/qtiplot.html [proindependent.com]. For really complex stuff, there's root from CERN http://root.cern.ch/ [root.cern.ch]
Geist3D (Score:2, Interesting)
I know of this one [geist3d.org], if only because it's written in Python and I ran into it looking for some unrelated 3D tools. It does do physics simulations, AFAIK.
GDL is the open source replacement for IDL (Score:2)
If you've ever wanted something FORTRAN-ish, but with matrices, see GDL [sourceforge.net].
They're trying to make an open source interpreter that can take IDL scripts directly. Unfortunately, I don't know if it can open up IDL save files, due to various threats from lawyers.
There's also PDL [perl.org], which deals with large data cubes in Perl.
Re: (Score:2)
Not to attack the great work that the GDL folks have been doing, but friends don't let friends learn IDL. It's too evil and bletcherous, and stunts the mind away from producing beautiful, generalizable code toward a twisty maze of special cases, all slightly different.
EPICS (Score:5, Informative)
It is an open source control systems frequently used for particle accelerator control and observatory telescope control. We use it slightly differently, but for what we need to do, it works very well. It is maintained primary by Advanced Photon Source at Argonne National Laboratory. You can read more at the following URL:
http://www.aps.anl.gov/epics/ [anl.gov]
In case you are wondering, no, they don't use EPICS for LHC. They use a commercial SCADA program called PVSS (for the most part anyway).
EPICS and RTEMS (Score:5, Informative)
Since you said experimental physics... :)
The Experimental Physics and Industrial Control System (EPICS [anl.gov]) is a set of Open Source software tools, libraries and applications developed collaboratively and used worldwide to create distributed soft real-time control systems for scientific instruments such as a particle accelerators, telescopes and other large scientific experiments.
EPICS is often used with the free real-time operating system RTEMS [rtems.org] to build custom control systems.
Users of EPICS+RTEMS include Stanford Linear Accelerator Center (SLAC), Argonne National Labs, Brookhaven National Labs, and Canadian Light Source.
EPICS (Score:2)
RHIC and LHC (Score:3, Informative)
Labview (Score:4, Insightful)
At the end of the day I could always justify the expense of the software when the equipment it is controlling is orders of magnitude more expensive and some of the automation I was able to roll out saved weeks if not months of time.
The is also a great community of labview users who freely share (source) code they have developed - many under specific open source licenses.
At the end of the day I wouldn't get to hung up about open source when you are dealing with equipment and budgets where the software is 1% of the cost - just use the best tool to meet the requirement or risk project overrun and funding issues.
Re:Labview (Score:4, Informative)
I've been a LabVIEW developer for 20 years now (since v2 came out in '89) and a C-coder for about almost as long and I can say this with about 99% certainty: LabVIEW is not for everything, but what it is good at, there is no good replacement (open source or otherwise). LabVIEW is second to none for data acquisition, control, (some) analysis, (some) simulation and (some) SCADA. On the flip side, unless you've a lot of experience with LabVIEW and/or a lot of time to kill, don't try anything with recursion, distributed computing or high-end visualization. I guess I'm not really sure what the problem is here: For less than $2k, you can pick up a copy of LabVIEW and save your boss hundreds of hours of your time. For about $5k, you can get the whole dev package and compile things to .exe's for deployment all over the company.
Just my $0.02...
Re:Labview (Score:4, Insightful)
At the end of the day I could always justify the expense of the software when the equipment it is controlling is orders of magnitude more expensive and some of the automation I was able to roll out saved weeks if not months of time.
Depends very much on the application. I once had a LabVIEW system that was getting camera input. I needed to do some simple image processing (and I mean really simple), but the only way I could do that in LabVIEW was to buy a gigantic add-on package that was very expensive. So in the end I coded it myself, but what should have been a 5-minute coding exercise (just modify one of the LabVIEW VIs to export some data) ended up being quite a bit more complicated because LabVIEW is closed-source (so I couldn't just go into a VI and make the simple change I needed).
This is just one example where LabVIEW's closed-source nature slowed me down. On the other hand, the community is quite good and there are many useful VIs being freely offered on the forums.
At the end of the day I wouldn't get to hung up about open source when you are dealing with equipment and budgets where the software is 1% of the cost - just use the best tool to meet the requirement or risk project overrun and funding issues.
Open-source isn't just about money. The ability to modify the code can frequently be very helpful, especially when you're trying to do something non-traditional (which seems like all the time in experimental physics!). In science there is also the issue of "knowing what's going on" inside the software if you're ever going to trust the data that comes out of it. And we shouldn't ignore the educational benefit (and all-around fun) of being able to get into code and modify it. Science is about learning (in particular for students; but tinkering is also crucial for established scientists!).
Re: (Score:2, Informative)
There's nothing stopping you from using a DLL from an open-source program for the image processing.
Shane.
Re: (Score:3, Informative)
when you're trying to do something non-traditional (which should be all the time in experimental physics!).
There fixed it for you.
My experience with LabView (used it once for a little undergrad acoustics experiment) was that it didn't do things quite the way I expected from its presentation. The whole block diagram data flow model thing made me think that everything was going to run in parallel. In reality, it did not, so real-time processing just wasn't quite the same thing as building a circuit that did what it looked like LabView was going to do. Of course, this isn't a very strong complaint, and somebod
Re: (Score:2)
Labview is horrible for memory management and very inefficient. So if you want something that pushes your hardware to the max (e.g. controls register allocation or defines custom communication protocol to minimize waste bits being sent over) then Labview is just not usable.
The other problem with Labview is that they every so often release new versions which happen to be incompatible with previous ones. So if you developed stuff for, say, Labview 5, then by now you have probably recoded your stuff several ti
NCL (Score:2)
Try NCL - NCAR-Graphics Command Language.
http://www.ncl.ucar.edu/ [ucar.edu]
astronomy (Score:2, Informative)
radware in nuclear physics (Score:2)
apt-cache search (Score:2)
from ubuntu 8.04, im sure debain has more than this
apt-cache search physics | grep -v game
cernlib - CERNLIB data analysis suite - general use metapackage
cernlib-base - CERNLIB data analysis suite - common files
cernlib-base-dev - CERNLIB data analysis suite - dependencies checking script
cernlib-core - CERNLIB data analysis suite - main libraries and programs
cernlib-core-dev - CERNLIB data analysis suite - core development files
cernlib-extras - CERNLIB data analysis suite - extra programs
cernlib-montecarlo -
Re: (Score:3)
The vast majority of this list are packages in the old, fortran-based CERNLIB [web.cern.ch]. Still around bug in legacy code mode. The modern replacement is Root [root.cern.ch].
Aimed squarely at high energy experimentalists, and used for data analysis and simulation. In that field, DAQ code is nearly universally highly custom since the electronics it is interfacing with is also custom. Usually built with FOSS tools, though, such as gcc.
Numpy, Scipy, pylab, VTK etc. (Score:2)
Transitioned from matlab to numpy/scipy/pylab/PIL/VTK and I'm much happier, much cleaner and easier to use. I handle interaction w/ pygame. Presently (like at this moment) working on two major network controlled interferometer data collection and analysis systems in python.
Open Source Physics learning software (Score:2)
Although not targeted to research and more towards learning, I though it would be good to give props to the people at http://www.andestutor.org/
Especially since my brother is one of the main contributors on the project ;)
It depends if you want to code (Score:2)
For some applications, like Labview and igor, there are alternatives if you want to code. These applications, and those like it, are tightly intergrated packages that allowed automated workflow in the lab with minimal understading of the processes actually being used. For control this is not neccearily a big deal. For data aq and analysis, I think the researcher should know what is actually happening to convert raw data(for instance a voltage) to the pretty picture.
It is
Replacement for Eagle (Score:2, Informative)
It's a tad less polished and maybe a bit more buggy than Eagle, but FreePCB ( here [freepcb.com] ) is FOSS Windows software. It works sufficiently well for me to be very productive. I've used it to make all of the circuitry for the Stanford Solar Car Project ( here [stanford.edu] ).
It's much easier to use than Eagle and does make you go through as much bigma to get a board made.
gEDA/PCB (Score:2)
In addition to Kicad (already mentioned) there's also gEDA and PCB (http://www.geda.seul.org/), which run on Linux, Unix, MacOSX, and Windows. There are even some geda-designed boards in space :-)
GnuRadio as an Labview alternative (Score:3, Informative)
I wrote the DAQ I use for my research using the GnuRadio framework (which is written in python and c++). It is relatively easy to write custom C++ modules for high speed computation. There is also a graphical programming utility called GRC for simple LabView style "programming". Unfortunately, the code still isn't very mature, but it is under very active development. It is difficult to argue with the simplicity of labview for a quick interface to NI hardware, but once you leave the academic world where licenses and hardware cost real money, the appeal is quickly lost.
When I get time, I'll play with comedi and the comedi module for gnuradio for working with NI DAQ cards.
GRACE (Score:4, Informative)
Re: (Score:3, Informative)
All things considered, I recommend IGOR (Score:3, Informative)
IGOR is a fantastic tool. Will it do everything? Just about. Its programmability makes it nearly infinitely flexible.
But, for me, the two features that make me shudder at the thought of dispensing with IGOR are the support and OS interoperability.
IGOR Pro is perhaps the best supported and best documented piece of software on the planet. (Yes, I have contributed a minor amount to said documentation.) The company stands behind the program completely, and the user community usually offers solutions to anyone's problem within minutes. And, the documentation is simply phenomenal: the comprehensive and highly linked pdf version of the manual ships with the software is also installed as a searchable, highly organized database that can be navigated many ways.
Then, of course, is the bonus that everyone gets with their purchase of IGOR: you get Mac and Windows versions as part of your purchase. (I've been told the Windows version runs well on Linux under Crossover.)
And, oh yeah, the cost is a fraction of Matlab's or LabView's.
ZOINKS!!! (Score:2)
From an experimental physicist (Score:2)
Scientific Linux Is for You! (Score:3, Informative)
Scientific Linux OS is written by the scientists and engineers at CERN!
Its base is from Red Hat.
It is a jewel of an OS and is exactly what you need.
Distrowatch.com probably has it listed and a path to downloading it. It is free.
Good luck and do something good that helps people!
What are you asking? (Score:5, Informative)
Professional experimental physicist here.
There are two main types of software physicists have to deal with: hardware interface, and data analysis.
Hardware interface is often the the tougher one: slow controls, data aquistion, environmental monitoring - these all need to interface to hardware through various drivers. LabView is an obvious candidate for table-top experiments, since it is possible to set up working control and readout systems more or less out of the box. There is really no good open-source solution for this for the same reason that open-source drivers took a long time coming to Linux: the user base is just too small to write the code.
My own experience is that it's far better to write your own code, using whatever drivers you can scrounge - it's far more efficient at getting what YOU want done as quickly as possible once it's running. However, the time to write and debug this code is extensive. It's particularly bad since often students will write this code and then disappear, leaving you with badly-documented half-working code.
However, this is basically true of many LabView installations as well.
On the data analysis side, there are many good packages that serve as starting points. ROOT (http://root.cern.ch) is an excellent package for doing event-based data analysis in nuclear and high-energy physics, including efficient ntuple storage and histogramming. It's really a toolkit, not a program, so it allows you to do your own analysis by writing your own code.
I'm not familiar with other big packages, but I know that I frequently use raw C, C++, gnuplot, perl, and python to do little jobs.
There are other tasks as well. Blogging software can be good for logbooks. Wikis are good for in-house documentation.
It really depends on specifics. But basically it depends on where your project falls on the quick-and-dirty vs long-life vs high-performance judgements.
Re: (Score:2)
Also include numerical computations and symbolic algebra?
Stephan
Sage (Score:5, Informative)
Perl Data Language (Score:3, Informative)
Perl Data Language [perl.org] is quite nice -- it has all the blended hackish glue-code goodness of Perl (which could be a plus or a minus depending on your personal style). CPAN (the big Perl repository) has a lot of free stuff for access to serial, parallel, and USB ports so you can control your equipment and acquire data. PerlXS (built into Perl) gives you nice access to C libraries, and PP (a meta-language that comes with PDL) helps you sidestep a lot of cruft if you have to use a C module to import large volumes of data.
PDL has built-in commands for plotting 2-D stuff via PGPLOT or PLPLOT, and 3-D stuff via GL. I use it for all my publications.
Some prefer NumPy or SciPy, the Python equivalents of PDL, but (IMAO) Python isn't as expressive as Perl, and the external libraries, while extensive, cannot compete with CPAN for completeness or hugeness.
Depends on your field, but.. (Score:2)
my researcher friends routinely use Octave, perl, I know of R being used... Not sure about data recording programs though, perhaps that is more specialised (or I just never needed them in my field). LaTeX, as auxiliary software, is pretty standard for writing up papers.
Of course, since it's Turing-complete you could just use emacs for everything... or vi... :-)
Long term support (Score:2)
Why and Why Not (Score:2)
I use both open and proprietary software in neuroscience, including more generalized signal analysis and statistics including some far left field non-linear non-integer stuff. Proprietary is far more likely to have the advantage of having a history of validation. If you can address validity yourself, use OSS. If you haven't approached calculation validity yet, look into it before you decide whether to attempt it. Statisticians and experimental mathematicians (yes, there are such beasts) relish arguing about
Things i have tried. (Score:3, Informative)
My different generations of meas and evaluation software stacks
Measurement software:
0) C
1) C/perl/java evaluation
2) C/java/jython evaluation:matlab
3) matlab/labview (connected by http+xml+xslt)
4) matlab with DAQ+instrument control toolbox
5) C/tcl evaluation:matlab; real time plotting: gnuplot
C was only used to bind non-preexisting daq functions. I dont write GUIs in C. For GUIs i prefer java, and the best free combinations i used were 2) and 5) would i be free to choose i would use 2), however in the environment i am currently working some co-workers write in tcl, so i found in more productive to learn tcl. (Not because of my productivity, but learn fast. However, one more conflict in the group could have killed the group. This selection was purely driven by social purpose - i gave to say: sucessfully.).
Otherwise the general idea would be that to use java and implement the GUI in java and the scripting languange in one of the many languages existing on the top of java. If you use a standalon scriptiong language for performance reasosn (i tested that once, but i could affot to go without it): python is a great choice for semi-numerical (and numerical) programs.
For evaluation i used and
1) perl
2) python
3) octave
4) matlab
1-3) each plotting with gnuplot. I know there are other solutions around, but when i tested them, gnuplot was always the rock-stable, feature-rich workhorse (think: LaTeX friendly output: epslatex - great in combination wit a makefile for you figures in a thesis!), with the only disadvantage of some things beeing historically grown.
4) is great, but do yourself the favor and alway try to keep the core functionality working in octave. I mean matlab is a great product, but you may prefer not to tie yourself to a package which costs than much money (think about what will happen after you leave your current univerity? do you still want to access you old data?). My current strategy is to keep the functions loading experimental data/evaluating it working in octave, and gui functions in matlab (only exception: i use the statistical toolbox moderatly. I could rewrite the function i use, but it would take a week. I am working in a reasearch company, and here it is more reasonable to buy the toolbox.
Oh. and by the way from time to time i am misfortunate enough to work in labview. it is a PITA. My personal experience say: rewriting the program in another language usually saved time.
Well (Score:2)
(You can only get a floppy boot image, for a network install AFAICS, and to install on the network, you need a login)
Maybe it's Alzheimer's but... (Score:3, Insightful)
Python(x,y) (Score:3, Informative)
I'm sure this will be lost among the sea of comments, but I'll throw it out there anyway. As a former LabVIEW/Matlab user, I have had tremendous success with the Python(x,y) [pythonxy.com] package.
All the development is done in Eclipse. You have a WYSIWYG editor built into Eclipse, with which to design Qt GUIs. The signal/slot methodology in Qt is similar to LabVIEW, but easier to work with once you've learned it, since you can make connections in text or the drag and drop interface, whichever is easier.
You can use the generic Qt widgets to display Matplotlib graphs, which use similar syntax to Matlab.
You have numpy/scipy, which are fantastic for processing data.
You have pyvisa/serial/parallel, which will control any instrument you have, or even ones you make/program yourself, such as FPGAs or MCUs.
You can use Python's pickling to store your data in rapidly-accessible modes (I've opened hundred million datapoint sets in seconds), or CSV/Excelerator to store smaller chunks of data in more human-readable modes. Python also provides a fairly simple database system if you need one, and you can always use Zope or Django if you want a web interface (though that will be harder to learn).
On top of all that, there are many more field-specific modules available in Python than in LabVIEW.
I've been using it for months now (I should emphasize that I had never even heard of Qt or numpy or any of those things before that), and I cannot wait to drop LabVIEW. If you think Perl is bad, try to debug a 5 year old LabVIEW program, developed by ten different people, each of whom was using LabVIEW because they don't know how to program properly. One of our old VIs, which only sends a string of commands over a serial port and then reads a DMM, weighed in at over 350 MB. No one can even figure out how it works any more. The same system, done with Python(x,y), came out to under 200 lines of code.
Re: (Score:2)
(2) GnuPlot Plots all that beautiful data....
The data may be beautiful, but the gnuplot plot won't. Use Matplotlib http://matplotlib.sf.net/ [sf.net] for that.
Re: (Score:2)