Solaris' Dtrace in Detail 212
paulkoan writes "The Register has a further details about the new Dtrace systems utility bundled with Solaris 10, along with pictures of the authors, and user testimonials. It also highlights Suns vague assurances that this (if it lives up to the hype) amazing utility may or may not end up in the public domain."
assurance of what ??? (Score:3, Funny)
so, we are assured of a and !a ???
Re:assurance of what ??? (Score:4, Funny)
No, we're assured of a or !a.
Although knowing Sun, they're probably trying for a and !a and wondering why it won't work...
^^ not || (Score:2)
And it still dodging around the issue. I do enjoy how project looking glass (cool but fairly useless) was used as an example of the trend of Sun to go open source. I guess, as people speculated, Sun decided to dig through their stuff, find something 'sorta cool' and throw it to the public saying 'see, we're trying.
a || !a = 1 (Score:3, Funny)
Re:assurance of what ??? (Score:2, Funny)
It also highlights Suns vague assurances that this amazing utility may or may not end up in the public domain." so, we are assured of a and !a ???
No, we are _vaguely_ assured of a *or* !a. I thought Sun made it clear enough! ;)
Sun has a long history of doing both at same time. (Score:2)
Sun has a long history of vacilating on whether and how to open their products. Hardware interfaces, SPARC-based hardware cloning, various software packages, and so on. Often this means half-opening, licensing and failing to renew, and so on.
It seems to be different every time. But the common thread is that they never commit. (I think the closest they came is Open Office where at least we have a fork.)
It was due to their refusal to open things sufficiently for me to
Re:assurance of what ??? (Score:2)
I hereby define statement a as:
Statement a is false.
a, or not a, that is the question...
And what is DTrace? (Score:4, Insightful)
Re:And what is DTrace? (Score:5, Informative)
Re:And what is DTrace? (Score:2)
So how about this wild ass attempt for an explanation:
Is dtrace simply a way of inserting wrapped, more-instrumented system calls to replace the standard ones, on the fly, perhaps using the same concepts as dynamic loaded libraries?
[I'm accustomed to memory debuggers that work by substituting their own heavily instrumented wrapper versions of malloc() for the vanilla ones.]
Re:And what is DTrace? (Score:5, Informative)
I've been using it for a little while, it's a very interesting tool.
Re:And what is DTrace? (Score:5, Interesting)
Re:And what is DTrace? (Score:2)
man fuser
Also, for the masochistic out there, it is possible to look at the pfiles output for every process and match inodes.
Re:And what is DTrace? (Score:5, Informative)
You can do a lot with cut sort and tr. Often they're all you need, but perl lets you solve problems those three tools can't even address.
I'd recommend grabbing a Solaris 10 beta and having a play with this thing. It's pretty amazing.
Re:And what is DTrace? (Score:3, Insightful)
Re:And what is DTrace - Shark I presume (Score:2, Informative)
If DTrace is better than Shark, it must be an awesome tool. Shark does everything this article describes and far, far more:
http://developer.apple.com/tools/shark_optimize
The 4.0 version of Shark shown at WWDC was even better. In many cases, Shark can actually give you specific directions on how to optimize your code: both at the source and instruction level.
Re:And what is DTrace - Shark I presume (Score:2)
Looking at that Apple blurb (and reports I heard from WWDC attendees), Shark is a performance tweaking tool - a profiler. If you had ever bought the heavy-duty Sun compilers, you'd have a functionally similar set of tools (collect, analyzer, and cc with various flags that generates annotated source). Admittedly not nearly as polished as Apple's tools (and not as cheap), but Shark is also several years newer - and having done exactly the
It's about time (;-)) (Score:5, Interesting)
Re:It's about time (;-)) (Score:5, Funny)
Did you get the memo about the cover sheet?
Re:It's about time (;-)) (Score:3, Funny)
Re:It's about time (;-)) (Score:3, Funny)
Re:It's about time (;-)) (Score:2)
Re:It's about time (;-)) (Score:2)
Hummph. Youngster Whippersnapper. I was here when they didn't need any improvement.
When I was young we had blue skys unless it was raining, and clear, dark skys at night, so dark and clear you could see the milky way from horizon to horizon. The "north star", Polaris, could be picked out of the c
terrrible article (Score:4, Insightful)
The article says what this D-Trace does, but yet, doesn't... Not to mention it's neither objective, review like, and even fails to mention alternatives or relative tools
Definately one of the poorest software article's I've seen in a while
Slashdot is making me bitter (-;
Complete CCTV Security Cameras [completecctv.com]
Re:terrrible article (Score:5, Insightful)
I must agree that the article is pretty positive about the whole thing. But a positive review is not necessarily a bought review. In fact, the article does contain an angle that is not likely to be sponsored by Sun: open sourcing it, and open sourcing solaris. They explicitly make fun of Sun's president, who is evading their questions about open sourcing DTrace. From the article:
So there you have it. DTrace may or may not end up in the public domain. Glad that's settled.
Re:terrrible article (Score:3, Informative)
The article gives sample uses, but it isn't a howto. I don't usually go to the register for technical howto's, do you? The article does link to more info on how to use the utility.
Not to mention it's neither objective,
Nor is it wholy pro-sun, it jabs the rep for not directly saying whether or not it will be open sourced and repeatedly makes comments about the hype ("It slices, it dices, it spins, it whirls").
and even fails to mention alterna
Re:terrrible article (Score:2, Funny)
Re:terrrible article (Score:2)
I agree with this. The article was very uncritical and did not place the software in any context.
IBM mainframe users have had this type of stuff since the early '80s. (The trace facilities SMF etc. were much earlier but the cool tools for analysing the output weren't really there util then).
Also there are similar tracing facilities available on AIX, although the user interface is classic IBM (obscure and picky) syntax.
This could be viewed as case of Sun playing catchup with its competitors, and, hyping u
Re:terrrible article (Score:2)
Re:terrrible article (Score:2, Funny)
indeed! (Score:3, Funny)
In fact, you can bet your life that it will or won't show up in public domain.
Now this is what I call a nice and reputable company! At least they keep what they promise!
Very Useful (Score:2, Interesting)
Re:Very Useful (Score:5, Informative)
The major difference: a huge amount of polish. DProbes can crash your system if used incorrectly; DTrace is guaranteed safe. DProbes reports just about everything; DTrace supports speculative tracing and predicates so you can narrow down the fire-hose of profiling data into the useful information. DProbes doesn't scale well across multiple CPUs; DTrace does. (This last one is just like Linux versus Solaris: one scales, the other doesn't. Sorry!)
So, yes. If IBM had spent the past four years tuning DProbes and turning it into a useful tool, they might have something. As it is, Sun spent quite a bit of time and money, and ACTUALLY CREATED a very powerful tool.
[rant]What is increasingly bugging me about open source advocates on Slashdot is how they'll look at any (ANY!) closed-source tool and say "oh, open-source tool XYZ does this / did this a few years ago / will make you walk on water". These advocates will cheerfully ignore that the closed-source tool has months or years of additional polish that have brought it from an internal company toy project into a commercially viable tool - that the open source tool of now is no more advanced than the closed source tool of years ago, except that most open source tools are stagnant and will never leave development and never obtain the degree of polish wide-scale use demands.
A message to open-source advocates: if you don't spend the time turning your hobbyist code into solid, professional code, the closed-source companies that DO spend the time will leave you in the dust - freedom of code (both free as in beer and free as in GNU) notwithstanding.[/rant]
other resource (Score:5, Informative)
just to save you all some trouble (Score:5, Insightful)
Solaris users could not care less whether Sun ports it to linux or not.
Sun are *not* evil because they don't immediately give away the source to a product which has taken them years and $$$s to develop and which gives them an edge in a competetive market.
Some people just prefer Solaris. If you prefer linux, that's fine.
Linux is not Solaris. (Score:5, Interesting)
Note that this is finally a tool to resolve finger-pointing *it's your app that's slow - no it's the kernel etc* . I guess IBM will find it advantageous to create a similar Linux tool for their systems, as they bill Linux as an application serving environment. So we should see it soon on our own boxen.
The sad thing is this proves once again that closed source companies retain an edge at innovating.
Re:Linux is not Solaris. (Score:2)
I think you're missing the point. It's competition that drives innovation. Sun are having a hard time keeping up with performance of their rivals in no small part due to the fact their cpu architecture is falling behind. In order to remain competitive they need to take advantage of every opportunity to squeeze more performance from their systems and apps. If that urgency didn't exist, would they have felt t
Re:Linux is not Solaris. (Score:2)
I disagree. Dtrace is the next logical step in performance monitoring. It is ahead of previous tools like gprof, gcov, Vampir, etc, and as such it represents real inovation. But is is not a revolution.
I can think of dozens of Open Source programs which have also taken the next logical step in some field, and are in my opinion just as innovative. These programs include Screen, Valgrind, Beagle and BitTorrent.
Re:just to save you all some trouble (Score:4, Informative)
Yes, it does (well, 85% of it at least), in the form of DProbes [ibm.com]. Dynamic Probes was ported by IBM from OS/2 to Linux way back in 2000, is distributed by SuSE and some parts are in 2.6 kernel out-of-box.
Re:just to save you all some trouble (Score:5, Insightful)
So I guess I'm saying, if you're 100% linux, that's cool. I certainly don't need to hear about j00r leet gentoo boxen in a Solaris thread. You know the kind of person I mean!
Dtrace slashdot (Score:3, Funny)
i/o preformance tuning (Score:5, Informative)
It's a hugely complicated problem and it doesn't have a simple solution except in the minds of markedroids.
Re:i/o preformance tuning (Score:2)
1) monitor for a period of time (1 day, one week, something typical that sees all normal usage patterns);
2) analyze collected data;
3) simulate moved (physical) data;
4) goto 2
Perhaps I'm missing something -- I realize this would be complex, but does not sound impossible.
S
looking glass (Score:3, Interesting)
I suppose we'll have to wait and see...
Re:looking glass (Score:2)
Microsoft perfmon (Score:4, Interesting)
It's a bit of a misnomer to characterize dtrace as the first application for doing this, although I will say it's among the first that I have seen that is scriptable (a huge advantage when troubleshooting problems remotely).
Re:Microsoft perfmon (Score:2, Informative)
Re:Microsoft perfmon (Score:3, Interesting)
Re:Microsoft perfmon (Score:5, Insightful)
Sun reminds me of H2G2 philosophers (Score:5, Funny)
``But who the devil are you?'' exclaimed an outraged Fook.
``We,'' said Majikthise, ``are Philosophers.''
``Though we may not be,'' said Vroomfondel waving a warning finger at the programmers.
``Yes we are,'' insisted Majikthise. ``We are quite definitely here as representatives of the Amalgamated Union of Philosophers, Sages, Luminaries and Other Thinking Persons, and we want this machine off, and we want it off now!''
``What's the problem?'' said Lunkwill.
``I'll tell you what the problem is mate,'' said Majikthise, ``demarcation, that's the problem!''
``We demand,'' yelled Vroomfondel, ``that demarcation may or may not be the problem!''
``You just let the machines get on with the adding up,'' warned Majikthise, ``and we'll take care of the eternal verities thank you very much. You want to check your legal position you do mate. Under law the Quest for Ultimate Truth is quite clearly the inalienable prerogative of your working thinkers. Any bloody machine goes and actually finds it and we're straight out of a job aren't we? I mean what's the use of our sitting up half the night arguing that there may or may not be a God if this machine only goes and gives us his bleeding phone number the next morning?''
``That's right!'' shouted Vroomfondel, ``we demand rigidly defined areas of doubt and uncertainty!''
Re:Sun reminds me of H2G2 philosophers (Score:2)
Not even to be modded up to +5 funny, aparrantly.
Interesting... (Score:5, Interesting)
I'd be very, VERY surprised if Sun allows Dtrace into the open source world, at least not for a while. If Dtrace really is the supertool it seems to be, and is actually and massively UNIQUE, it represents a reason peole will move to Solaris and buy Sun's hardware to do it. Maybe the closest thing to a Unix killer app that has existed for a while.
Now eventually (as in five years down the road) it will probably pay for Sun to open it up. If I were them, I'd milk it for all its worth on the "get people to move to Sun boxes" mantra, while the rest of the world trys (and probably fails) to duplicate the tool. Then, when Sun has gotten all the converts they are likely to, start making the tool even BETTER by opening it up and letting the world go to town on it. (GPL or something similar so Sun can incorporate back in the goodies.)
Of course, that's just an off the cuff theory by someone who doesn't know.
Re:Interesting... (Score:2)
Oddly enough, there doesn't seem to be a patent for DTrace.
Re:Interesting... (Score:4, Insightful)
Ya, sysadmins are always trying to optimize code in the core applications they bought from vendors. NOT.
It's not useless, but the examples cited were from application developers optimizing their code.
Re:Interesting... (Score:2)
First, as a sysadmin you could determine where your problems exist and provide proof to your internal cevelopment teams or to the vendor. I would find this priceless as most of us have been on the vendor seesaw before. Oracle blames SAP who Oracle and then both intimate it could really be the OS or Veritas...
Second, app developers can use this tool to dramatically improve the performance of their apps. You don't think db v
Re:Interesting... (Score:2)
Understanding and using Dtrace (Score:5, Informative)
Dtrace user guide [sun.com].
A collection of Dtrace scripts [tpg.com.au]
Re:Understanding and using Dtrace (Score:2)
Kprobes and Dprobes (Score:4, Interesting)
How does Dtrace for Solaris differ from Kprobes [ibm.com] and Dprobes [ibm.com] for Linux ??
Re:Kprobes and Dprobes (Score:2)
Re:Kprobes and Dprobes (Score:5, Informative)
Dtrace Usenix Paper [sun.com]
More examples in answerbook from the BigAdmin Dtrace page [sun.com]
Re:Kprobes and Dprobes (Score:5, Interesting)
You can find a small section comparing kprobes with DTrace in the DTrace USENIX paper; it also mentions some other differences (kprobes can easily crash your production system; that won't happen with DTrace).
New Tool? Maybe for Solaris... (Score:2, Informative)
Linux Kernel Instrumentation (Score:4, Informative)
DaGoodBoy
Re:Linux Kernel Instrumentation (Score:2, Informative)
Re:Linux Kernel Instrumentation (Score:2)
Can someone summarise what Dtrace is? (Score:2)
(any answers along the lines of it DTRACES your OS etc will make me come to your mothers house and explode my head all over her prize dinnerware set)
Well... (Score:2)
You could derive work-alike programs for top, sar, ps, etc. etc. using simple dtrace scripts.
And a lot of the statistics would be normally nearly impossible to get without debugging the kernel or poking around in
By putting it all
Re:Well... (Score:2)
Anyone know of a BSD gig going?
Exists already in Linux kernel (Score:2)
Its called OProfile.
While not identical to DTrace it lets you perform the same task.
And another advantage of open source - if you do find a resource hog, you can go in and "make it right".
Heck, for really silly bottlenecks you could fix it in hours/days rather than months.
Re:Exists already in Linux kernel (Score:5, Informative)
OProfile is useful for measuring system-wide resource consumers (for example, you can see what pieces of code are causing cache misses in the kernel when your apache process is in the kernel etc, or which user processes take up the most CPU time).
DTrace can also do something similar (though it needs a little more work yet). But DTrace does a LOT more than this. Imagine a system-wide (kernel, binaries, libraries) 'strace', where you can trivially choose what to print out, and what parts to strace, and under what circumstances. DTrace does even more than that.
OProfile can't tell you exactly why your system call is returning EINVAL. OProfile can't tell you why your application is causing cross-calls. OProfile can't tell you what processes are writing to what files, in real time. OProfile can't debug race conditions.
OProfile is a profiler: it does its job and nothing more. DTrace is, essentially, an instrumentation suite; one of its abilities is to function as a simple profiler.
You won't really get a notion of why DTrace is so useful until you try it.
Re:Exists already in Linux kernel (Score:2)
Other tools that do this. (Score:5, Interesting)
actually that's not true. Kerninst [paradyn.org] does exactly that.
Kerninst is a framework for dynamically splicing code into a running kernel, almost anywhere, anytime. Code can be removed and changed at will. Kerninst works on standard (unmodified) Solaris and Linux kernels, *no* kernel re-compilation is necessary.
Re:Other tools that do this. (Score:5, Informative)
This paragraph is copied from the Related Work section.
Kerninst is a dynamic instrumentation framework that is designed for use on commodity operating system kernels[13]. Kerninst achieves zero probe effect when disabled, and allows instrumentation of virtually any text in the kernel. However, Kerninst is highly aggressive in its instrumentation; users can erroneously induce a fatal error by accidentally instrumenting routines that are not actually safe to instrument.3 Kerninst allows for some coalesence of data, but data may not be aggregated based on arbitrary tuples. Kerninst has some predicate support, but it does not allow for arbitrary predicates and has no support for arbitrary actions.
Favorite quote (Score:5, Funny)
I'll have to start using that in my conversations about lusers. "I wouldn't call him stupid; let's just say he doesn't have a good relationship with his brain..." :)
Sun comming arround? (Score:4, Insightful)
In that sense, Sun is no Microsoft. They are a hardware company that provides services, plain and simple. The only reason why they are cold to Linux is because it pits Sun servers directly against x86 commodity PC's - otherwise they are all for it.
Once the dust settles on their bread and butter revenue stream, you can better believe that they will be open-source all the way. But right now they need to force some differentation with their hardware because in most cases they simply can't compete against an x86 farm in the server space. All the rest of the BS about new accomplishment is just propaganda, I would ignore it.
Re:Sun comming arround? (Score:2)
Find out about what DTrace is, can do and what it means before simply referring to it as 'BS'.
Re:Sun comming arround? (Score:2)
No
Sounds very cool... (Score:4, Funny)
I know I would pay big dollars, say like $28, for a box that ran infinitely fast.
This could be the diet pill for software bloat that we've all been looking for.
Re:Sounds very cool... (Score:2)
Would this *not* be an antidote to code bloat? I am rather puzzled why my parent post was slapped with -1 troll. Is bloat dear to the moderator? *hint hint*
Nice automated Gui? (Score:2)
Or I guess I'm asking, how complex can these functions get? Would it be feasible to tie it in with other things so I could do my own interface to it? Sounds like fun if I can...
A competing tool for Linux (Score:3, Interesting)
Code Review (Score:3, Insightful)
I was reading the examples of using DTrace to spot performance issues.
It seems to be that most of them could have been caught with code reviews.
Perhaps with tools like DTrace, you can spot performance issues more quickly. But I believe a good code inspection regime could catch these problems and more.
Does Dtrace benefit Scientific Computing? (Score:2)
Can it suggest areas of vectorization, loop unrolling, parallelization, etc?
Does it primarily give you stats on usage of functions, disk I/O, etc?
The numerical methods I use are probabilistic, making them very hard to debug sometimes.
(Imagine a bug occurring only
Solarisx86 port? (Score:2)
I am about to try out solarisx86 and my asus motherboard, nic, and video card all happened to be supported with Solaris 9 oddly enough.
I am interested to give this thing a spin.
Re: (Score:2)
Re:Lots of users == good. (Score:4, Funny)
So if some software isn't in the Gentoo portage tree you won't run it on your Solaris boxes at work?
This is proprietary software for a version of Solaris which isn't even out yet.
Remind me not to employ you as a Solaris sysadmin. Hell, remind me not to employ you to make the tea - we might have different cups at work and you wouldn't know what to do with them.
Re:Lots of users == good. (Score:2)
Re:Lots of users == good. (Score:2, Informative)
Which is what - you don't have an opportunity to play with it at home? But you do - you can get a Solaris 10 preview through their Sun Express program (linked in the article) and install it on another box, in VMWare, etc.
Re:Lots of users == good. (Score:5, Informative)
interesting experience (perhaps kind of OT) (Score:2)
So of course, I had to install Gentoo on it. I did. It took a while. I burned out a stick of RAM. I am afraid of Gentoo. But anyway, I got it up and running, but I had to disable tagged queueing on the disk to keep it from freezing up all the time. So here I am, with a
Re:Huh? Quantum leap? (Score:3, Funny)
"very small" is only your perception of the size of change. Moving between a base state and an excited state is a pretty big thing for an atom.
Re:Huh? Quantum leap? (Score:3, Funny)
Re:Huh? Quantum leap? (Score:3, Informative)
Quantum means 'a level, or fixed amount'. In Quantum mechanics this refers to the smallest observable physical effects. At which point changes in energy are in fixed 'Levels' (ohh like Qunatums) rather than smooth functions.
So in this article he is saying that this has reached the next level. And that you are a dimwit.
Thank you very much,
QED
Re:Huh? Quantum leap? (Score:2, Interesting)
"A dramatic advance, especially in knowledge or method...This term originated as quantum jump in the mid-1900s in physics, where it denotes a sudden change from one energy state to another within an atom. Within a decade it was transferred to other advances, not necessarily sudden but very important ones."
(As a side note, I did a google on quantum leap definition [google.com] and got a lot of results using the example sentence ""this may not insure success but it will represent a qu
Re:Huh? Quantum leap? (Score:3, Funny)
Re:Hmmm (Score:3, Funny)
I'm sure they did this to appeal to the disproportionately-large numbers of African-Americans in the field of computer science.
Not really (Score:5, Informative)
Re:Isn't this a profiler for non-profiled binaries (Score:2, Informative)
try reading some of the related links (like the bigadmin dtrace site, the usenix paper, the com.unix.solaris post, or the sys admin magazine article) and you'll quickly realize why this is a revolutionary tool.
the profiling facility is only one of the information providers that dtrace has.
other providers include:
- a function provider (allows you to monitor calls into any kernel function e
Re:So it's the SE toolkit (Score:2, Informative)
DTrace is not the SE toolkit. The SE toolkit used a similar method (little langugae scripts) to collect data that was made public by the kernel.
DTrace takes advantage of monitoring hooks placed throughout the kernel and in every system call, and it does things that SE never could.
Furthermore, one look at the DTrace manual and your boss will shit his pants. This is not PHB material by any means.
Chapter 1: The D prograsmming Language...