Next-Gen X Window Rendering For Linux 652
Bytal writes "Seth Nickel, a GNOME hacker, has an extensive treatment of the next generation Linux graphics technologies being worked on by Red Hat and others. For all those complaining about the current X-Windows/X.org server capabilities, things like 'Indiana Jones buttons that puff out smoothly animated clouds of smoke when you click on them,' 'Workspace switching effects so lavish they make Keynote jealous' and even the mundane 'Hardware accelerated PDF viewers' may be interesting."
In case of slashdotting... (Score:2, Informative)
Next-Generation Rendering For the Free Desktop
For the past half year or so Red Hat's desktop team has had people working toward making accelerated graphics rendering on the free desktop badass, but doing an ass job of actually talking about what they're doing in a larger public / GNOME context. They've been doing a combination of experimentation (from that cracktastic OpenGL compositing/window manager luminocity to xsnow for the Xcomposite generation) and knuckle-down no-holds-barred infrastructure work (like making Win32 GTK work on Cairo so GTK can move to cairo as the default backend). With RHEL4 kicked out the door we've been able to rebalance day-to-day work on GTK and X onto other people to give the nextgenren hackers free hands. Currently the full-time nextgenren team at Red Hat is Owen Taylor (gtk/pango maintainer), Søren Sandmann (x hacker), Diana Fong (visual designer), Kristian Høgsberg (x hacker) and Carl Worth (cairo maintainer).
I'm really excited because these guy's expertise is across a broad chunk of the rendering pipeline, from the toolkit down to the x server, which is going to give this effort the ability to work on this from a global perspective rather than optimizing the bits where we happen to have influence in. I'm doubly excited because other companies (well, Novell at least, but hopefully others will join) are starting to invest in this effort too!
I'm hoping to drag Owen into spinning this off into an umbrella effort (ala project utopia) to help maintain a coherent story/platform even as lots of people pour work into lots of different packages and distros. There are so many different ways to attack the X rendering issue that I'm a little worried about seeing a lot of fragmentation of effort and the result not being particularly coherent. I do hope people experiment with lots of different approaches, but I also really hope that in we can give developers a consistent platform for doing cool graphics on the free desktop. It would be a real shame to end up with the message in two years being "well, platform X has the feature you want, but you have to worry about also working with Y because X won't work well on distro Z". This sort of technology-choice morass can really dampen developers playing with this stuff and adding support all over GNOME, which is exactly the sort of quick-fiddling big-payoff stuff I think we'll see a lot of as soon as this stuff starts landing. In other words, lets push toward the point where people can feel confident and start hacking up cool things for this system inside GNOME.
What It Might Look Like
A really good system needs to have lots of pieces in place all hooked together....its not something that can be hacked apart and replaced by arbitrary random incompatible bits (though there are points of commonality, such as OpenGL or Render). For example the pieces in one imaginable architecture - by no means the decided-upon final one or anything - might look like:
* A sophisticated drawing layer (cairo using glitz/opengl or render as backends)
* Stock renderers built on top of that drawing layer (pdf/ps rendering backed by cairo - such as Alex Larsson's xpdf fork in evince, svg rendering backed by cairo, etc)
* A toolkit that agressively takes advantage of the features in the drawing layer, exposing them to applications and themes (gtk+)
* A window+compositing manager that can work closely with the toolkit but essentially takes the window contents as a static image in compositing (metacity with luminocity-like GL compositing manager features fused in to deal with window effects, synching up smooth resizing
overview of modern display systems (Score:5, Informative)
i'm hoping cairo/glitz will give quartz extreme a run for its money. now we just need to get started on implementing something similar to coreimage/corevideo!
Re:I think their efforts would be better spent on. (Score:2, Informative)
The XAA sucks sweaty donkey ballsack. KAA (or it's successor) needs to be firmly in place, with solid proof that it works (software implementation) before vendors are going to write drivers for it.
Getting the vendors to open up specs, or to write OSS drivers themselves is just gonna get you blue in the face, and besides, it is pretty orthogonal to actual technological growth.
Only programmers can do this work, any consumer or advocate can push to have specs opened up. How many GPU manufacturors have you contact this week?
Welcome to last week (Score:3, Informative)
Uhm, E17 anyone? (Score:5, Informative)
Granted, Enlightenment is a window manager that lives on top of the existing X protocol, but nearly every single piece of 'eye-candy' this guy mentions is already do-able in E17.
Sounds just like the themeing system in E17 to me... http://enlightenment.org/pages/systems.html [enlightenment.org]
Don't get me wrong, the things Seth describes sound cool, but the way he describes it makes it sound like they're the only ones with these ideas, when in fact Enlightenment 17 is already enabling most of what he mentions in this article. Sure, it's not a "production" release yet, but DR17 is certainly usable today, and has most of the features he mentions.
Heck, some things Seth talks about (Live window thumbnails) have been available in Enlightenment for quite some time (I know DR16 has them, and maybe earlier versions as well)
Re:"Hardware accelerated PDF viewers'' ? (Score:4, Informative)
Also gnome is very macintosh like and one of the early macintosh developers wrote nautilus if I recall.
No one is stealing anything. Even the menu bars on the top of the screen came from Xerox before Apple used them.
What I like about kde and gnome to some extent is that they are highly customizable compared to either mac/windows. The problem is the later versions of kde look a little cluttered as a result but you can make your desktop look like anything.
Also you can have kde put a menu on the top of the screen just like gnome and macos. I think you can add a task bar to gnome as well.
I think perhaps some new innovative idea's are needed instead of just borrowing existing ones. Perhaps a way to handle many apps running at once without the desktop looking cluttered is next.
But I believe(could be wrong) that Windowmaker,kde,gnome all use ghostscript which is a postscript clone. The original macos and nextstep used it. Windows has an equilivant but I do not remember the name since its been a long time since I admined Windows boxes.
Re:Inevitable comment about bloat (Score:3, Informative)
I can. You know all those Gnome and KDE themes that try (badly) to imitate Aqua? They copy the eye candy without understanding the reasoning behind it, or the value of visual cues. Too often, the result is badly misapplied pinstripes, distracting transparencies, and so on.
Re:"Hardware accelerated PDF viewers'' ? (Score:5, Informative)
I sobbed when I read this.
I wrote a really long post correcting this widely and wrongly held opinion some weeks back. I don't feel like finding it, or being that verbose again. So short versions.
No PDF, no OpenGL.
Quartz 2D is a display-list engine, but it is not a PDF interpreter. Rather, Apple wrote some very, very simple shims that quickly translate PDF files into Quartz 2D display lists and back. Nothing in Quartz 2D is represented in PDF format unless it's sitting in a file on the disk.
The windows are drawn on the screen by a piece of software called Quartz Compositor. A couple of years ago, Apple rewrote Quartz Compositor to take advantage of hardware acceleration. They did use OpenGL for this, but only in a very limited way. Each window is represented as a texture on a surface and fed to the graphics pipeline for compositing.
Quartz is amazing. Nothing else in the world comes anywhere close to it, despite what some very confused people seem to think. But you're really selling it short when you describe it as "PDF and OpenGL." Because it isn't.
Re:Battle has already been won (Score:4, Informative)
Re:"Hardware accelerated PDF viewers'' ? (Score:2, Informative)
I don't know how the Dock (from NeXT/OpenStep) influenced the taskbar. And I have no idea what you mean by the menu's having been highly influenced from NeXT (I haven't seen too many people that use a floating menubar like NeXT did, but again I'm not sure if that's what you mean).
NeXT wasn't the only company at the time to use Display Postscript. Sun's NeWS system used it as well, and there are a few others that I can't recall right now. And NeWS came out before NeXTStep.
Also gnome is very macintosh like and one of the early macintosh developers wrote nautilus if I recall.
Members of the original Mac team (Andy Hertzfeld, Bill Atkinson, etc) founded Eazel, who made Nautilus (and others). Many people worked on Nautilus, such as Pavel Cizler -- who wrote Tracker while at Be (and is now at Apple working on the Finder). I wouldn't say Gnome/Nautilus is very Macintosh like, both from a classic and OSX point of view. But, that's just my opinion.
No one is stealing anything. Even the menu bars on the top of the screen came from Xerox before Apple used them.
No. Smalltalk (Xerox) had no menu bar, either at the top of the screen, or at the top of the window. A right click (or 'yellow button', or whatever the hell they called it) in the window opened up a popup menu. You can see a picture of Smalltalk-76 in action at: http://users.ipa.net/~dwighth/smalltalk/St76/st76
Or, you can download Squeak, an open source Smalltalk implementation created by the people that created Smalltalk back at Xerox, and check it out for yourself. No menu bars at the top of the screen, or at the top of a Window. And Squeak was created in the mid-90's (so it's not like they didn't know about menubars).
But I believe(could be wrong) that Windowmaker,kde,gnome all use ghostscript which is a postscript clone. The original macos and nextstep used it. Windows has an equilivant but I do not remember the name since its been a long time since I admined Windows boxes.
I really don't understand this. If you're just talking about general use of Postscript, okay. If you're saying the original MacOS used Postscript as it's method of rendering to the screen (i.e. like Display Postscript, or whatnot) then you sir are on crack.
Re:overview of modern display systems (Score:3, Informative)
Display PostScript worked by embedding a PostScript interpreter right into the operating system. The system would run PostScript programs, rasterizing them to the screen, to produce screen output. The system would do exactly the same thing but route the output to an attached laser printer instead of the screen to produce printed output.
Quartz 2D is not, and has never been, "Display PDF." Quartz 2D is a display-list drawing API that uses a drawing model that's very similar to PDF. Apple included code in the OS that could trivially convert any Quartz 2D display list into a valid PDF file on disk and vice versa. But Quartz 2D is not "Display PDF."
Your section on Quartz Extreme gets a lot of important stuff wrong, too. It doesn't take Quartz Extreme to put a transparent window on top of another window. That's something that the very first builds of Quartz Compositor were capable of doing. Quartz Extreme offers nothing that Quartz Compositor didn't offer; it's just that Quartz Extreme does the same job with the GPU, while Quartz Compositor did it all in the CPU.
Seriously, man, this paper is pretty terrible. Even if your assignment is finished, I hope for your own knowledge you go fill in all the gaping holes.
Re:"Hardware accelerated PDF viewers'' ? (Score:3, Informative)
A lot, actually. Try viewing a PDF document made from scanned pages - simply moving from page to page in xpdf will take 5+ seconds.
Re:"Hardware accelerated PDF viewers'' ? (Score:3, Informative)
Well, even Motif Window Manager lets you iconify running apps
Re:"Hardware accelerated PDF viewers'' ? (Score:4, Informative)
According to Bruce Horn [vwh.net]: "Smalltalk had a three-button mouse and pop-up menus, in contrast to the Mac's menu bar and one-button mouse."
Re:"Hardware accelerated PDF viewers'' ? (Score:5, Informative)
I'll try to address these points, while avoiding being too technical (which is a pain sometimes
1) Remembering windows size/positions. This drives me nuts.
Okay, okay. You're probably right on this one. However, please consider that a linux desktop is not used like a Windows one, specifically
- people generally use several workspaces and lay out their windows on multiple "screens", so to say,
- you're not supposed to reboot an X terminal as often as a Windows workstation - you just lock it and leave it as is. This comes from older times, but still shows,
- typically, people just arrange their windows *once* and leave them that way. For a very, very long time. When time comes to reboot, they save their session, preserving their windows' position (okay, this does not work all the time) then log back in again later.
Indeed, the X Window System was not supposed to be used like an MS Windows desktop, and the differences still bite us from time to time (why does evolution remember the active pane but not its window position across sessions ? WHY ? Answer : because it's the window manager's business, not his, and e.g. Metacity doesn't support this quite right yet).
2) Hot keys. For the love of god can someone fix hotkeys in gnome! I was used to the alt key toggling the menu of whatever is the active app.
The Alt key is a modifier. It is not a "real" key. It is meant to be used in combination with another "real" key, just like Shift, Control, Super, Hyper, Fn, Apple, etc. It is not cross-platform. It is not standard. It is usually mapped to the Meta key under Linux, which was once used to set the high bit on characters you typed on older terminals. You don't expect something to happen when you press the Control key alone, right ? The same applies to the Alt key.
Use F10. One press of F10 activates the main menu, both on Linux and Windows. Another press dismisses the menu. I don't know about Macs (do they have an F10 key ?) but a real (though nonstandard) key like F10 is much easier to code for than a modifier like Alt.
Hope this helps,
Cheers
ko
Re:overview of modern display systems (Score:1, Informative)
At Apple during the development of Mac OS X, the replacement for DisplayPS was called DisplayPDF. Quartz 2D was called the "window server".
Re:"Hardware accelerated PDF viewers'' ? (Score:5, Informative)
It says, "Quartz does not use Postscript as its internal graphics representation language. Instead, it uses Adobe's Portable Document Format (PDF) standard which is a superset of Adobe Postscript."
That's just completely incorrect. Quartz 2D graphics are not represented internally as PDF. They just aren't. When a Quartz 2D graphics context is stored in memory, it's stored as a display list, very similar (conceptually) to the way OpenGL scenes are stored in memory. To convert the context to a pixel buffer for display on screen, Quartz Compositor (or Quartz Extreme, depending on hardware) renders and composites the graphics context, which results in a bitmap.
A Quartz 2D display list is very similar to PDF in the way regions are defined and paint applied to them; this makes it easy for PDF files to be converted into Quartz 2D display lists and vice versa. But it's equally true that the Open Inventor file format is similar to an OpenGL display list in the way that vertices and surfaces are defined. You would be wrong to say that OpenGL programs store scenes internally in Open Inventor format; you'd be equally wrong to say that Mac programs store their graphics internally in PDF format. It just ain't so.
Can an Open Inventor model be trivially read from disk and turned into an OpenGL display list? Sure. Can a PDF file be read and trivially turned into a Quartz 2D display list? Yes.
That's it. Okay?
Re:Fully accelerated FBDev across monitors? (Score:3, Informative)
Joe
Detailed description of Quartz/PDF (Score:3, Informative)
"MacOS X is the first operating system on the market that actually uses PDF-technology within the operating system itself. Apple calls this technology 'Quartz'. Quartz is a layer of software that runs on top of Darwin, the core (or kernel) of the MacOS X operating system. It is responsible for the rendering of all 2D objects. Alongside Quartz, OpenGL takes care of handling 3D data (used in games like Quake or Unreal as well as professional 3D applications like Maya) and QuickTime handles multimedia stuff (movies, sound,...).
Quartz
Quartz replaces QuickDraw, which was used within earlier versions of MacOS. Within QuickDraw, the native file format was PICT. With Quartz, this now becomes PDF.
Quartz performs a number of tasks that include include:
automatic PDF generation and save-as-PDF (disk and clipboard)
conversion of PDF data to raster data or PostScript. The fact that Quartz can rasterize PDF files means that even cheap inkjet printers can output complex files. Gone are the days when only the screen preview of EPS-files was printed on non-PostScript printers.
a consistent feature set for all printers
automatic on-screen preview of graphics
high-quality screen rendering
In short: Quartz implements a set of rules for describing how pictures and text are displayed and printed. Because Quartz uses the PDF drawing model for imaging, native applications can create and import PDFs without the need for outside programs.
Some people have been wondering whether Apple pays licenses to Adobe for the technology used in Quartz. Here is what an Apple employer had to say about this: The Quartz renderer and the PDF interpreter that Apple ships with Mac OS X are built with Apple code, with no external licenses, by Apple employees. Adobe just publishes a specification for how it's supposed to function. This gives Apple considerably more flexibility with regard to what Quartz and the PDF interpreter can be used for.
Adobe PDF versus Quartz PDF
Since Quartz uses PDF, one would assume that everything that is possible within a PDF file is also supported by Quartz. This is not the case. Quartz uses only some of the features of PDF, it is based on a subset of the full PDF specs.
These are some of the things that are used within both the official PDF specs and Quartz:
the PDF imaging model
Common colour spaces: grayscale, RGB and CMYK
Embedding of images (even though Quartz does not support masks)
And these are things that are feasible in PDF but that are not (yet?) implemented in Quartz:
Annotations
Colour management using ICC profiles
Forms
Actions
Bookmarks
Digital signatures
Security
DeviceN (used within PDF to offer improved support for images containing spot colours)
Embedded fonts
Form XObjects: in some ways the PDF-equivalent of an EPS, meaning a group of objects that are a sub-part of a page.
Transparency
In fact, one of the main differences between both systems is that the PDF specs are now at version 1.4 while Quartz adheres to a subset of the PDF 1.2 specs."
Re:Detailed description of Quartz/PDF (Score:3, Informative)
1. Dividing it up into Quartz and OpenGL is misleading. If you want to talk about it in terms of functional block diagrams, OpenGL and Quartz 2D (note: not just "Quartz") do the same job. They take instructions from a running program and turn them into patterns of pixels on the screen. But Quartz 2D is not responsible for all 2D drawing. QuickDraw can also be used to draw to the screen; pre-Tiger, QuickDraw is quite a bit faster than Quartz for doing aliased RGB drawing. QuickTime also renders directly into the window, bypassing Quartz 2D entirely. So saying that Quartz is responsible for all 2D drawing is just plain wrong.
2. Talking about "native file types" is also misleading. There is no native Quartz 2D file type. Quartz 2D display lists can be translated into PDF and back for disk storage, but that's not the same thing.
3. The whole thing about how Quartz rasterizes PDF files is bogus. When a PDF file is loaded into a Quartz 2D drawing context, it gets converted by the file I/O code into a Quartz 2D display list. This is a resolution-agnostic floating-point-based display list format that has absolutely nothing to do with either pixels or PDF. If this display list is destined for the screen, it goes to Quartz Compositor (or Quartz Extreme) which renders the display list into pixels. If it's destined for a printer, it goes to the printer driver. If it's destined for a file, it gets converted back to PDF format and stored on disk.
4. There is no PDF interpreter built into Mac OS X. That is, there is no piece of software that takes PDF input and spits out a bitmap. Quartz doesn't work like that.
Other than that, this comment is sorta-mostly-kinda correct. More or less.
Re:I think their efforts would be better spent on. (Score:4, Informative)
Re:"Hardware accelerated PDF viewers'' ? (Score:3, Informative)
Just because neither Apple nor Microsoft have "embraced" this concept doesn't mean we have to reinvent the wheel.
Re:Detailed description of Quartz/PDF (Score:3, Informative)
Have you considered reading Apple's developer documentation on the Quartz imaging system instead?
Is Arstechnica wrong?
In large part, yes. Please read this [slashdot.org]. (Wow. Déjà vu.)
Re:GTK needs Win32 to work on Cairo? (Score:1, Informative)
Re:Let's run through the list, shall we? (Score:5, Informative)
Well, an understanding of the topic we're discussing, for starters. I mean, I know what all the words mean, which is clearly something that you can't truthfully say. All you've done is pull quotes from marketing brochures! There's no evidence at all that you have even a passing familiarity with the basic concepts under discussion here.
you're going around claiming Quartz doesn't use PDF for imaging
Correct.
when every developer documentation from Apple directly states that Quartz uses the PDF imaging model
Also correct.
Gasp!
How can this be! How can Quartz 2D both be PDF and not PDF!? He's a witch!
Friend, in order to wrap your head around this topic, you're going to have to understand what the expression "imaging model" means. An imaging model is not a file format, and it's not an instruction set, and it's not an interpreter. It's not actually any type of computer software at all. Rather, it's a way of looking at things.
Back in the old days, we had QuickDraw. QuickDraw used a pixel-based imaging model. You drew to the screen by specifying coordinates in terms of pixels: integer coordinates, bottom-left origin, one pixel was exactly one seventy-second of an inch. Regions were translated literally by shifting bitmaps around in memory. That was the QuickDraw imaging model.
That worked great for drawing to the screen, but it didn't work at all for drawing to a laser printer. For drawing to a laser printer you needed a totally different imaging model. Which means you had to do one of three things in your program: Either you had to maintain an internal representation of whatever you were drawing in whatever form was appropriate for printing and then convert that to QuickDraw for on-screen display, or you had to maintain a QuickDraw representation and convert it at print time, or you had to do both.
But the advantage of QuickDraw was massive: You could draw right into video memory. Toggle a bit in memory and a pixel changed color on screen. Very efficient.
Quartz 2D is different. It uses an entirely different imaging model. Rather than representing on-screen graphics as bitmaps in memory, Quartz 2D creates a layer of mathematical abstraction. With Quartz 2D, you still have a bottom-left origin, but you're not longer on an integer plane. Coordinates are given as floating-point numbers. You don't deal in pixels, but rather in mathematically pure regions of the drawing plane.
You draw in Quartz 2D by defining regions. A region is a locus of floating-point coordinate pairs. For example, (2.1, 3.37), (6.29, 5.3), (7.889, 1.961) defines a triangle. You draw by telling Quartz 2D to fill that region with a certain color, defined by any of the supported color spaces. For instance, you might use RGBA, meaning you'd specify red, green and blue color components and a floating-point opacity value.
Sending these commands to Quartz 2D from within your program creates an in-memory data structure called a display list. This display list doesn't look like anything at all; it's just a sequence of bytes that are encoded to represent the scene you drew. The display list doesn't become anything until you send it to Quartz Compositor (or Quartz Extreme) to be rendered into pixels.
The fundamental assumptions behind Quartz 2D drawing -- the coordinate system, the color spaces, all the low-level details --are referred to collectively as the "imaging model."
PDF has an imaging model that is very similar to Quartz 2D's imaging model. Not identical, but very similar. That's because Apple's engineers were inspired by both PostScript and PDF when they created Quartz 2D.
Because Quartz 2D and PDF use the same imaging model -- the same set of fundamental assumptions --it's very easy to convert a PDF file describing a scene to a Quartz 2D display list that describes that scene. Or you can go vice versa, starting with a Quartz 2
Re:Let's run through the list, shall we? (Score:3, Informative)
In other words, it's easy to go from one to the other - it's trivial to convert a bunch of Quartz instructions to an equivalent PDF document and vice versa, even though the internal representations of the data are completely different.
Quartz isn't about applications sending actual PDF data across a pipe or socket into a renderer, it's a bit more sensible than that.
Re:"Hardware accelerated PDF viewers'' ? (Score:3, Informative)
you don't quite understand (Score:2, Informative)
X11 is a protocol. Xaw is not part of the protocol. It was "ditched" long ago. People still use it because they still have applications that depend on it, but that doesn't need to bother you.
stop interfacing with video hardware like it's 1980.
I don't know what that is supposed to mean. X11 has numerous server implementations that interface with hardware in all sorts of ways. Many commercial and workstation X11 implementations have had dedicated hardware acceleration for more than a decade. What more do you want?
If you are saying that XFree86's architecture is a bit dusty, well maybe you are right, but XFree86 isn't X11, it's one of many implementations.
NeWS did not use Display Postscript (Score:3, Informative)
In many ways Adobe/DPS were way behind NeWS.
Re:"Hardware accelerated PDF viewers'' ? (Score:1, Informative)