Seven Habits of Highly Effective Unix Admins

jfruh writes: "Being a Unix or Linux admin tends to be an odd kind of job: you often spend much of your workday on your own, with lots of time when you don't have a specific pressing task, punctuated by moments of panic where you need to do something very important right away. Sandra Henry-Stocker, a veteran sysadmin, offers suggestions on how to structure your professional life if you're in this job. Her advice includes setting priorities, knowing your tools, and providing explanations to the co-workers whom you help." What habits have you found effective for system administration?

Tmux

[matthiasvegh]

I discovered tmux (terminal multiplexer) a while back, and is a very potent replacement for screen, it supports splitting windows, having multiple sessions, sharing windows between sessions, customizable status bars etc. Try it out!

Using multiple shells

[Neruocomp]

When working on a problem, I usually have two or more shells open. I don't mean multitasking, but with more then one open, I can issue commands from one and use the others to monitor logs/etc.

Habits ...

[Xaemyl]

What habits have I found effective for system administration? BOFH spring to mind ...

Re:Number 6 Problem

[zacherynuk]

Indeed - not only that, but even if you are really good at keeping docs, an intranet log or similar - it still won't be read, understood or appreciated. Later on, with even the best of everyone's interests at heart the worst thing you could ever say is - "I documented this here, and explained it here and asked for feedback here and you said you read it..." Nothing like a few reference facts and common sense to drive a wedge between admins and users.

Re:Tmux

[evilviper]

For my use cases, I could not find a compelling reason to use tmux

Obviously if you've been limiting yourself to the features of "screen" for many years, you're not going to think you need the added features of "tmux"...

A big one is sharing:
"window can be linked to an arbitrary number of sessions". If you or somebody else has a screen session open, you don't have to detach it from their terminal to see what's on it. You can just attach it to your terminal as well. Works great when you've got a session attached to your desktop, then want to access it on your laptop/tablet/phone/etc. The tmux session will even change geometry to match the smallest terminal window.

Being more lightweight and responsive is good. Saner keys for some functions, like ctrl-a pg-up to access scrollback. And just the fact that it's still getting active development is an important feature.

Re:Tmux

[dissy]

Here it is in nicm@'s words:
"In particular, being able to share a single window between multiple terminals, with other windows in the same session but entirely separate. Adding this to screen was implausible"

Perhaps I am still misunderstanding the features of tmux (most likely in fact), but to say that is implausible to add to screen is misleading to say the least, since I have been doing exactly that in screen for nearly a decade.

On one terminal, either start a new screen session or -r to a detached session.
If starting a new one, try: screen -S LetsShare

On a second terminal, run: screen -list
You should see a list of screen sessions and their status (attached, detached, multi, etc)
If you used -S on start that will be the name, otherwise it's some tty.host.number string.

Now on that second terminal run: screen -x

Try to adjust both terminal sessions so you can see them at the same time. Type in either, watch in either. They are shared seemingly matching your tmux description.

You can change permissions per terminal so others can't type but will see everything you do (aka tutorial mode) using ^a *

Also for split/multiple windows showing on the same terminal, use ^a S (control-a capital-S)
To switch between split windows use ^a tab
Close a section of split window with ^a Q

The status bar problem is true and pretty annoying. I fixed it myself with a line in ~/.screenrc but of course I have to pretty much install that user config file on every new system I use which can get annoying.
If you want an always-on status bar showing window numbers and titles (^a A to change the title), add this to .screenrc (and hope slashdot doesn't munge it!)

hardstatus alwayslastline "%{= wk}%-Lw%{= BW}%n%f* %t%{-}%+Lw %-=%{= BW}%H%{-}%{-}"

Note the two "BW" bits? That's background blue and foreground white, and applies to the window with focus. Change B to R for red for example (production vs not-production in my case)

Here is my whole .screenrc file for copy/paste purposes: http://pastebin.com/kMkuFXi9 [pastebin.com]
No splash screen, always on status bar, 10k line scrollback history for copy/paste (^a [ and ^a ] ), and auto-open three windows with preset titles and commands running in them.

I don't mean to knock tmux in any way at all, having not used it (and I do plan to check it out now) - but hopefully these screen tips help out others here.

Re:Rebooting is not a fix

[pla]

For some reason, Windows admins have been trained to reboot immediately when things don't work well rather than to figure out why something is failing.

Because in the Windows world, I usually don't have the luxury of digging into the kernel's or driver's source code to figure out exactly why it has stopped behaving correctly. If it doesn't log any errors, doesn't export any useful diagnostic messages, doesn't outright crash on reproducible conditions, and just stops working "right", your avenues of further inquiry get very very ugly, very fast.

I can reboot a VM in well under a minute. For any nontrivial problem that happens roughly twice a month and a reboot makes it go away, it would take twenty years of rebooting to justify spending an entire eight hour day diagnosing the root cause.

And I say that as someone who (in the Linux world) has written his own kernel patches to work around buggy hardware. In Windows, just not worth the time; because even if you do successfully diagnose the problem, you may well have no ability to correct it.