Documenting a Network? 528
Philip writes "Three years ago I was appointed as a network manager to a barely functioning MS-based network. Since then I've managed to get it up and running — even thriving — but have been guilty of being too busy with the doing of it to document the changes and systems that were put in place. Now as I look back, I'm worried that I am the only one who will ever know how this network works. If I get hit by a bus or throw in the towel for any reason, I'd be leaving behind a network that requires some significant expertise to run. Ultimately, this won't be a good reference for me if they are trying to work out technical details for years to come. It looks like I'm going to have to document the network with all sorts of details that outside consultants could understand too (no, I don't want to be the outside consultant), especially since it's likely that my replacement will have less technical expertise (read 'cheaper'). Are there any good templates out there for documenting networks? Is anyone who has done it before willing to share some experiences? What did you wish your predecessor had written down about a network that you inherited?"
I know... (Score:5, Funny)
The Passwords.
Re:I know... (Score:5, Insightful)
Whether the comment was intended to be funny, I find this to actually be a serious issue...
Re:I know... (Score:5, Insightful)
Whenever an important password was changed, I would hand over the new envelope
Re: (Score:3, Insightful)
At which point you won't be getting the passwords anyway since you fired him/her? Good idea. ^_^
Re:I know... (Score:5, Funny)
If you think that is ironic, then note that the story poster is worried his ability to get a job if he dies in a bus accident.
Re:I know... (Score:5, Funny)
If you think that is ironic, then note that the story poster is worried his ability to get a job if he dies in a bus accident.
maybe he could work as a voter in Florida?
Re: (Score:3, Insightful)
I was taught that the proper way to store admin passwords for an emergency situation (such as admin gets hit by bus) was to
1. Write each password on an individual piece of paper
2. Seal each piece of paper in an envelope
3. Store the envelopes in a Safe/Deposit box that a limited number of people have access to, (company owner, CTO, CEO, IT Manager)
4. Put policy in place that requires passwords to be changed and re-recorded any time the envelope it is stored in is opened.
It may seem low tech, but it is probab
Re: (Score:3, Insightful)
Someone will still need the password to decrypt the store.
Public-key cryptography to the rescue!
You can encrypt the file in such a way that any of a set of private keys can decrypt it. You can also encrypt it in such a way as that a combination of keys are needed (i.e. 5 keys used in encryption, and you must have 2 in order to decrypt.)
You can also send that machine's logs to all 5 owners, so that an intrusion by any one will be more likely to be noticed (and it will be harder to tamper with the logs.)
You could probably even automate the entire encryption process
Re:I know... (Score:4, Insightful)
Didn't Bruce Schneier once say "If you think cryptography can solve your problem, then you don't understand your problem and you don't understand cryptography"?
I don't think this solution is nearly as good as the envelope system. In particular, I don't expect the company owner or the CEO of the average small to medium business to have the expertise keep their private key files and password accessible to them and no one else. That means that either you're actually relying on two out of the other three keys being available (if those two fail to keep their private key files accessible to themselves) or you have poor security (if those two fail to keep their private key files accessible to no one else), whichever is worse.
In contrast, all of the trusted people probably know how to keep a physical key reasonably secure and accessible, and if you use a deposit box at a bank there may be identity checks as well.
In general, if your solution requires detailed technical knowledge from non-technical people, your solution is broken. Pick something low-tech instead.
Re: (Score:3, Informative)
Sounds like that's the root of our disagreement - when I think of a small business, I think of one that doesn't have five IT people, much less senior IT people.
Re:I know... (Score:4, Insightful)
So what happens if said predecessor gets hit by a bus, has a heart attack or a stroke and can no longer tell you the passwords? Or, worst case scenario, the whole IT team gets taken out in a road accident on the way to a team building session for example? I've read a few "deserves to be fired" comments on /. and usually tend to agree (or occasionally get embarrassed because I think hmm, that's me!), but in this case you are being a fool.
Of course, if you are dead then you won't care if they have the passwords, but some of us actually like our places of work and even our colleagues, and want our place of employment to be able to chug along even in our absence.
Re: (Score:3, Interesting)
So what happens if said predecessor gets hit by a bus, has a heart attack or a stroke and can no longer tell you the passwords?
Boot single user and reset them?
Yes, I realize that doesn't scale, but it works fine in a lot of small environments.
Re:I know... (Score:4, Informative)
How exactly do you get a local admin account on a domain controller? I didn't think there was any such thing.
Re: (Score:3, Informative)
do remember that the Directory Restore password is diffrent than the domain admin password..
from directory restore you can change the domain admin.
Re:I know... (Score:4, Informative)
Reboot the DC into "Directory Services Restore Mode" (this is on Server 200 and above) and the local Security Accounts Manager is used again, not AD.
This is actually a way of resetting the admin password on a domain, if you ever need to. Boot from the Linux password reset CD (http://home.eunet.no/pnordahl/ntpasswd/bootdisk.html), blank the Administrator password (which resets the _local_ admin password), then reboot into DS Restore Mode. Log in and then copy cmd.exe over on top of logon.scr and reboot into "normal mode" (with AD functional).
Wait until the "screensaver" pops open and you have a command prompt that just opened that's running with SYSTEM privledges. From there you can run mmc.exe (or whatever else you like).
Oh and of course, this is exactly why you don't allow physical access to servers....
In reference to the PDC statement, I'm not aware of any way to use local accounts on NT 4 or below, but I think he meant just DC instead of "PDC."
Re: (Score:3, Interesting)
If you are primarily a Linux shop you might use Coda or NFSv4, but if you still have Windows clients you might also run Samba. Actually, Samba can serve CIFS and not just SMB; CIFS has extensions for Unix features. So it's not a bad choice for serving files to Unix, but it's no NFSv4. (In some ways, that is a good thing.)
Re:I know... (Score:5, Insightful)
If the predecessor does write the passwords down, he deserves to be fired.
That's knee-jerk stupidity, and you should be ashamed of your non-thinking fundamentalism.
Passwords do need to be written down, and stored in "escrow". I put the list of passwords in an envelope, lick seal it, sign and date the seam, and then seal it again with clear packing tape. Give it to the boss to put in his safe.
Yes, it's easy to open, but you'd know whether someone tried to tamper with it.
Re:I know... (Score:5, Funny)
That's knee-jerk stupidity, and you should be ashamed of your non-thinking fundamentalism.
I just assume that any event capable of destroying my ability to transmit said passwords to my successor also destroys any ability to give a damn about my job. Problem solved.
Re:I know... (Score:5, Funny)
So what was it like working for the City of San Francisco, anyway?
Re: (Score:3, Funny)
No no, you want to keep everything in your head, that way you can't be replaced by someone cheaper!
Re:I know... (Score:5, Insightful)
Re:I know... (Score:5, Interesting)
Yes, it's easy to open, but you'd know whether someone tried to tamper with it.
Try spraying the envelope with refrigerant. The paper becomes translucent when wetted and you can sometimes read what's inside, and then it dries without a trace (unlike wetting it with water, which swells up the paper fibers leaving the telltale signs of tampering.)
Learned this one from a history of the U.S. Black Chamber [wikipedia.org].
Yes... I use dust-off (Score:5, Informative)
the cans of compressed air in every office supply store? inverted they throw out a very cold liquid that does exactly what you describe.
Re: (Score:3, Interesting)
Re: (Score:3, Informative)
Yes, it's easy to open, but you'd know whether someone tried to tamper with it.
Try spraying the envelope with refrigerant. The paper becomes translucent when wetted and you can sometimes read what's inside, and then it dries without a trace (unlike wetting it with water, which swells up the paper fibers leaving the telltale signs of tampering.)
Learned this one from a history of the U.S. Black Chamber [wikipedia.org].
Does anyone not use "security" envelopes anymore? (The ones that are printed with a dense line pattern on the inside?) I didn't know they sold plain envelopes anymore, except for greeting cards.
Re:I know... (Score:5, Insightful)
Either he writes the passwords down, or he uses weak passwords that a human mind can remember.
Besides, a password is a security token. A piece of paper or a little plastic card with the password printed on it or a USB stick with SSH key or whatever saved on it are also security tokens. They aren't inherently less secure than memorized passwords; you simply have to secure the physical object by, for example, locking it in a safe.
"Passwords should never be written down" is an idiotic rule, right there with "never use goto".
Re:I know... (Score:4, Informative)
If the predecessor does write the passwords down, he deserves to be fired.
You can't always take it for granted. I've been on the cleanup-end more than once. Sometimes it's "engineered job security"/"they don't dare fire me", sometimes it's "I don't have time for that", sometimes it's just plain forgetting about a piece of hardware you set up the first week you started work there, and sometimes it's a legacy password issue. ("nobody's been able to login to that box since Phil left in '05") The rare treat is finding a mystery box that nobody knows what it does nor has any idea how to login to it. Too many managers don't understand the danger and consider it a waste of time or a bad risk to try to fix problems like that.
Sometimes it's an uphill battle when taking over, too. You want to document the settings on all the routers, but two of them are "legacy" password issues, and the router only supports hard reset (clears password, AND all settings) so you can't get the settings from it once you reset it, and you need the settings from it in case it gets reset. That's never fun, but you will find yourself in that catch-22 occasionally, and it's hard to blame someone for not fixing it because it's utterly unpleasant to deal with. (hint: get another router and program it how you think the mystery box is set up. swap. test. immediately swap the mystery back in. adjust the settings on the new one and test. Swap back out. repeat until you get it right, and don't reset the mystery immediately, keep it onhand for at least a month in case some uncommon thing requires a setting you haven't yet discovered)
Occasionally you can get lucky - contact the hardware vendor and see if the box has an undocumented soft reset. ("open it up and short together the two pads left of D-15, and you can login for 5 minutes with no password")
Re:I know... (Score:4, Interesting)
Or how about a new CTO pissing both admins off and one walking out as the other was fired for no good reason.
I've had to walk in behind something like that several times and reset the passwords or load the password hashes into some cracker in order to find the passwords. A lot of times, you can pull them from workstations the old admins used and they are easier to crack then the newer MS servers. The funniest thing is that the CTO usually wants me to stay on full time and gives me a dirty look when I won't do it because he made life so miserable that two other people walk off the job under his supervision.
Re:I know... (Score:5, Interesting)
This may sound funny, but I recently had the same experience. I took over the position of CTO of an electronic payment company, and after one week, I figured a lot of critical systems are missing root password, including Linux, AIX, HP/UX and SCO Unixware. No one knows the password, it's been changing hands so many times, and the people who were responsible for those machines have left, without leaving the passwords behind.
Those are critical systems that must run 24x7. We had to rebuild the system on new machines, re-route transactions to the new machines, and shutdown the old ones to recover (single user mode).
And that's a platform handling over 400 billion in transaction per year. Scary. But that's the easiest problem I have inherited, mind you.
Re:I know... (Score:4, Funny)
So that's what happened during the bank collapse!
Re: (Score:3, Interesting)
Re:I know... (Score:5, Funny)
Check the top drawer of the your desk where the paper clips are lying normally.
Oh? It is locked and you dont know where the key is?
Do what the guy before me did (Score:5, Funny)
Re:Do what the guy before me did (Score:5, Insightful)
The next step up from a Post-It, though, is a snazzy label-maker. My portion of the company uses these extensively to document our development lab (we do some NMSy stuff). Of course, it's not a production network, and standards are a little different.
Good News (Score:5, Interesting)
Your successor will never find any documentation that you leave behind (or if you show it to them they won't bother with reading it) and by the time they notice it they'll have already screwed things up to the point where the documentation will be obsolete. This means you can save yourself the trouble of doing the documentation unless that documentation is going to make you more effective while you're there.
Re:Good News (Score:5, Insightful)
This is more to the point. Most network admins have the attention span of a flea and won't read past the first sentence of anything you write; actually, I could probably expand that statement to include most people generally. Lorem ipsum dolor sit amet, consectetur adipisicing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non proident, sunt in culpa qui officia deserunt mollit anim id est laborum.
Re:Good News (Score:5, Funny)
You just thought I wouldn't catch a reference to Cicero's De Finibus Bonorum et Malorum.
Original Lation from which it was derived: ...neque porro quisquam est, qui dolorem ipsum, quia dolor sit amet, consectetur, adipisci[ng] velit, sed quia non numquam eius modi tempora incidunt, ut labore et dolore magnam aliquam quaerat voluptatem. Ut enim ad minima veniam, quis nostrum exercitationem ullam corporis suscipit laboriosam, nisi ut aliquid ex ea commodi consequatur? Quis autem vel eum iure reprehenderit, qui in ea voluptate velit esse, quam nihil molestiae consequatur, vel illum, qui dolorem eum fugiat, quo voluptas nulla pariatur?
At vero eos et accusamus et iusto odio dignissimos ducimus, qui blanditiis praesentium voluptatum deleniti atque corrupti, quos dolores et quas molestias excepturi sint, obcaecati cupiditate non provident, similique sunt in culpa, qui officia deserunt mollitia animi, id est laborum et dolorum fuga.
English:
Nor again is there anyone who loves or pursues or desires to obtain pain of itself, because it is pain, but because occasionally circumstances occur in which toil and pain can procure him some great pleasure. To take a trivial example, which of us ever undertakes laborious physical exercise, except to obtain some advantage from it? But who has any right to find fault with a man who chooses to enjoy a pleasure that has no annoying consequences, or one who avoids a pain that produces no resultant pleasure?
On the other hand, we denounce with righteous indignation and dislike men who are so beguiled and demoralized by the charms of pleasure of the moment, so blinded by desire, that they cannot foresee the pain and trouble that are bound to ensue; and equal blame belongs to those who fail in their duty through weakness of will, which is the same as saying through shrinking from toil and pain.
What was that about my attention span?
Re:Good News (Score:4, Funny)
Not if they did their jobs right :)
Re:Good News (Score:5, Funny)
...fellatio uber alles...
WTF???
Re:Better News (Score:5, Insightful)
Not really, because as most high level executives know, IT doesn't really do anything!
Re: (Score:3, Funny)
When I was part of an IBM sales team in The Old Days we used to scare prospects by asking to see their network diagram. Almost never appeared.
Then we'd ask who owned the network. That was good for laughs.
Finally we'd ask why the most important part of the network (the end user) didn't appear on the diagram (assuming they could produce one).
By the looks of it nothing's changed.
Re:Better News (Score:5, Insightful)
Re:Better News (Score:5, Insightful)
Yeah, I think that's more or less true. At one of my previous jobs, I had a guy try to imply that I didn't deserve my pay because I "wasn't doing much". When I asked him what I should be doing, he said, "It's just that you have a really easy job. The IT guy at my last job had it much harder. He was always running around, fixing things. You just sit at your desk because nothing ever breaks."
I can't remember now, but I think I might have done a literal facepalm right then. I said something like, "Has it occurred to you that, if you think none of our IT stuff ever breaks, I must be doing a good job? If the IT stuff at your last job kept breaking all the time, he was doing a worse job than me?"
get some help (Score:5, Insightful)
Re: (Score:2)
I concur, nothing like showing a fresh grad the realities of IT by making him document a network without assistance.
On another note, if you ever expect some form of job mobility or flexibility, theres nothing like saying 'heres the documentation, cya'.
Documenting teamwork (Score:3, Interesting)
Alternatively... (Score:2, Interesting)
Professionalism be damned, we're in the middle of a recession and you're wanting to make yourself dispensable? Go ahead by all means, but pass your boss my number :)
Re:Alternatively... (Score:5, Funny)
English only please... I don't understand this word "vacation" you have mixed in with the others.
What about? (Score:5, Informative)
Re: (Score:3, Informative)
In other words, the tool you recommend is Etherape?
Any other tools you would use to grab a snapshot?
@the submitter, what about a tool like spiceworks?
Re: (Score:3, Informative)
Etherape is gorgeous. I wish I had time to understand what it all means, but still I love firing it up and staring at the network traffic.
Another tool that I've enjoyed much more personal success with is cheops-ng. For documenting a networking, cheops-ng and a decent icon collection provides a pretty snazzy view of what NMAP can see. (NMAP being another tool I haven't had time to fully grok, yet)
http://cheops-ng.sourceforge.net/screenshots.php [sourceforge.net]
Re:What about? (Score:5, Insightful)
A summary diagram (dia/visio) is still pretty key, but nmap (or specifically the latest versions of zenmap (the gui frontent to nmap) provides a good detailed view of network topology and if you include the capture file, you can annotate specific computer details in the notes section (ofc you have to assume your replacement will be familiar with zenmap for that to be useful though)
Re: (Score:3, Informative)
has anyone ever used The Dude?
http://www.mikrotik.com/thedude.php [mikrotik.com]
i just got a work-study job with the campus adming at the community college i attend. hes been there almost a decade and has no network monitoring system, so he has no idea when something goes down until he gets a complaint or cant get something to work himself. i thought an interesting project during my time with him might be to see if hed let me help implement a network monitoring system, and ive seen a couple of people use The Dude before a
schematics (Score:5, Informative)
Basic network documentation:
I've found that starting out with the very basic physical layout and working your way up in complexity is greatly beneficial.
i.e. start out documenting network cable runs including cable type. follow it by switch layout. follow that by routers and vlan setups. follow that by the servers that provide basic network functionality(e.g. DHCP, etc...). If this is a windows network, that would likely mean detailing the domain controller setups. From their systematically document the systems in order of importance to the business, etc...
Also, visual diagrams are extremely helpful.
Let's be real (Score:5, Interesting)
Short answer: don't worry about it too much. Put together enough that it looks like you've done something then go have a beer.
You could have the most amazing docs the world has ever known - with passwords and clear instructions - ad the odds are about 20% that the next guy will even read them.
The next guy will figure that he/she knows much more than you as evidenced by the fact that they are there and you are not. And, the cheaper they are (read: inexperienced) the more likely this is to be the case. When things go wrong, they will blame you anyway.
So document away, but for YOUR sake so that if/when you are called in after the new guy horkens everything, you can have an easy time putting it all back together. But don't wait for the call... people will put up with almost anything when pride is on the line.
And go have a beer.
That is not real, is cynical and unprofessional. (Score:5, Insightful)
No wonder our field and many of our professions have such a bad reputation.
I have read only a few posts and two (moded up 5) say pretty much to ignore the issue.
In several networks I have worked with fundamental information was non existent. This translated in lost time, down time and actually losing money (if you lost your job in one of those companies recently, the indolent SAs or Network administrators may be partly to blame).
You never know who the next guy will be, if he is less experienced or capable then the documentation will be very valuable, if he is more experienced or capable then you would have saved their time to do some real work, after all they (and you) have not being hired to do forensics.
How a professional can hide behind the "let's be real" nonsense is beyond the pale.
Start with disaster scenarios (Score:5, Informative)
For me, visio's are great and everything, passwords too, but really the most valuable thing you can do is document single points of failure, outdated software/hardware, etc., license keys/expiration dates, cert expiration dates, personal support contacts you have and all vendor relationship details as well are essential. Do you use change control? If you do, go back and comment your changes, if not, do the best you can at explaining why things are the way they are. Get some open source software that is good at indexing data and create a searchable knowledge base from the information above. Don't concentrate on docs that can be found on the web at first because any admin worth their salt will know where to look for how to's, etc. Focus on the why's, the where's and disaster recovery.
My two cents...
Re:Start with disaster scenarios (Score:4, Informative)
Get some open source software that is good at indexing data and create a searchable knowledge base from the information above.
A Wiki?
Here is what I would get (Score:5, Informative)
1. Viseo overview of the network drawing with complex areas drawn out specific detailed viseo's (even a scanned sketch or paint drawing is better than nothing)
2. A spreadsheet with circuit ID's mapped to router and interfaces.
3. Document the trunk interfaces as well as the LAG's (Link Agrregation Groups, port channels, whatever you want to call it)
4. TACACS passwords / domain logins in a secure location (or radius or diameter or whatever you use)
5. Data center capacity as a function of 1. Rack Space, Cooling Capacity, Electrical Load.
6. Write brief knowledge articles describing any problem areas and explaining a history of anything you think would be hard to figure out easily. No need to go hog wild, just re-brand the RCA documentation you have. You do have Root Cause Analysis right?
7. Network protocol hierarchy map. Where are your major redistribution points, what is your routing strategy etc.
8. If you have a voice network document all your DID's, PRI trunks, Gatekeepers, Dial Plan, and any translations you use on h.323 gateways or how MGCP or SIP is configured. If you have a complex call center you should probably pay to have it professionally documented in down to the minute detail.
9. SSID's and BSSID's for any wireless you may have as well as passwords, 802.1x authentication methods, along with linking documentation.
10. Make the documentation part of your CMR process (Change Management Review) and incorporate it into the time allotted for a change.
I know these are just rough ideas and you should get many more ideas from all the smarter people on here than myself, but whatever advice you get I would say you would need to have the documentation update able via subversion, or some document control system and have some kind of review process for it, even if it means getting together over pizza with some of the other groups and asking them about their environment and getting pointers and possibly help on documenting it all. Documentation is a full time effort and IMHO there is no such thing as too much documentation. You would be surprised how good documentation can aid you in problem resolution down the road or aid vendor support in helping you resolve a major outage. The three basic principles of network care are document document document. :)
Cheers,
Anonymous Coward.
Re: (Score:3, Insightful)
The AC has a very good list, I'll see if I can add anything to it.
Network diagrams should be at a network, physical and datalink layers. Only the simplest networks can have all this information on a single diagram and have it be useful. Seperate the network drawing from the datalink and physical drawings as requred but be sure to leave enough detail to connect the drawings (Visio has a nice linking feature for this). Also keep a spreadsheet or database of assigned networks, IP ranges, and assigned static
Re:Here is what I would get (Score:5, Insightful)
circuit ID ? in that a farmer's synonym for "MAC address" ?
Not everything is ethernet.
Re: (Score:3, Insightful)
no, in the real world most businesses dont interface with the internet via cable modem or dsl. Ckt ids refer to the Telco designations for the connection to the outside world for your network.
Re:Here is what I would get (Score:5, Funny)
Actually, it's a simple test designed to separate the people who should be working with the corporate wide area network from the people who should be forcefully prevented from coming anywhere within fifteen metres of the comms cabinet, using a taser if necessary.
If you have to ask which group your answer has placed you in then I'm not going to tell you.
TiddlyWiki (Score:5, Informative)
On the side, I manage a small network, and I've also wondered the same sort of thing: if someone else needed to find their way around, where would they start.
A Wiki makes for a really nice way to document things, not least because you can include all sorts of cross references. For example, a list of servers, with links to the services they provide - and a list of services, with links to the servers. But Wiki's normally run on servers, which leaves your successor with a chicken-and-egg problem.
A bit of random surfing turned up TiddlyWiki [tiddlywiki.com], which is a Wiki in a single HTML file. A really elegant bit of engineering, and very handy for self-contained documentation. Since the entire Wiki is just a single file, it's easy to protect. I wound up with two: one with "public" information describing the general architecture and one with private information (including passwords). The private one you can put on a USB-stick in a safe, hand to your boss, or whatever seems appropriate...
setup a wiki (Score:5, Interesting)
At my last few companies and my current one that I work out, one of the first things I do is setup an internal only Wiki server.
Not only does this let me document everything I can about the network but I also try an train my co-workers in using it to document information they feel is important for their job too.
The effectiveness though seems to be related to the level of computer literacy of the user, i.e. my last company the software developers went crazy with it and documented everything they could think of. But other than them or us in the I.T. dept, no one use would hardly touch it at all.
Either way it's still a simple method for your to store notes, diagrams or any information about your network in an easy to find single location that you feel would be important to the company should you leave for any reason.
Why do people make things hard for themselves? (Score:5, Informative)
Why not use an automated too?
www.open-audit.org
Many documents. (Score:2)
Unlike software documentation, in a network there is more than one approach, pretty much a function of the amount of people that have to fiddle with the network at any given time. Usually there is physical laying of cables (where are they), box location and naming (labeling and Visio-sheets), peripherals (printers, faxes, phone system), box setup (OS and processes), server process configuration specifics and client requirements. And then there's that (what I think) very important document that describes w
Re: (Score:2)
Oh, and did I mention procedures, like backups ?
Self-Documenting? (Score:2)
Contact Numbers (Score:5, Insightful)
You likely don't run *everything* and the new person needs to know who to contact when the interaction between inside and outside fails.
I've been doing this for a while (Score:5, Informative)
My job requires me to do exactly what you're looking to do but for multiple companies/networks. Then, as soon as I'm done, I usually pack up and go or get hired in and fix the network.
Since I'm writing the Network Overview for managers AND potential future network managers I tend to write mine in the following format:
1) Synopsis of what the network does for the company, what general technologies they use (Windows AD vs *nix OD, thin clients vs Windows boxes, Cisco vs Brand X), and what the LOB software is.
2) Points of contact for the ISP and other providers (anti-spam, anti-virus, hardware, etc). Passwords for various accounts and services.
3) Logical network overview map (visio), containing firewalls, routers, switches, other devices, open/forwarded ports, IPs, what the servers do, what vlans are in place, Quick explainations for why (such as why vlan vs a seperate subnet).
4) Physical map of devices if the complexity of the network calls for it.
5) Software notes, what apps are critical for the business and which systems they rely on.
Then, for my specific job I have to do the following:
6) Licensing issues.
7) Network weaknesses/points of failure.
9) Other rec's.
Do what everyone else does. (Score:5, Insightful)
Draw a horrible diagram in Visio (or similar) of what's connected where without any indication how it actually works!
NeDi (Score:2)
Easy (Score:2, Funny)
Lots of flowcharts! (Score:5, Insightful)
Present client I am at I inherited a network of about 15,000 clients that was previously managed my a very incompetent IT department. Started by looking at the existing flowcharts and discovered that almost everything that was documented was wrong... Long story short I have been spending a fair bit of time reverse engineering their production environment so that we could accurately document it. Unfortunately we had come to the conclusion that we can use /nothing/ that the previous administrators left behind for documentation. You don't want someone like me coming in and looking at your documentation and declaring you incompetent, it can cost you your job.
You haven't detailed the size of your organization to know if you will need sign off from other departments or not. If possible try to get sign off so that they have a reference and you can create a standard that can be used to fix things and to ensure your designs don't get trampled by a new admin in another department. You really need to provide more detail on your environment for people to answer you.
I do most work in Visio, starting at 50,000 feet and working my way down. At this level I need to document network topology, server distribution and database server distribution. I work my way down from there using a zoom in style that has served me well for 30 some clients. Depending on the size, complexity and your area of responsibility you may need to flowchart anywhere from a 2-3 levels to potentially dozens of disparate processes. You haven't mentioned much about process development, I assume you want people to know how to do at least critical portions. Never write a process without flowcharting it, this will save you grief by getting people to focus on the process instead of a step by step set of directions. It takes someone fairly good to document the complex and make it look simple, that is your job at this point in time.
The bottom line is that your documentation should show dataflow for each critical system. As long as you can do this someone else can step in and work with what you have, even if they may not understand a given piece. One of the big advantages of flowcharting everything (especially processes!) is that this will readily show you weakness and holes that may have been previously overlooked. When flowcharting complex processes don't be afraid to have a single point represent an entire additional complex process that can be distictly referenced of it's own accord (as an car repair manual of mine once described the process to replace a crankshaft "Step 1. Remove Engine".) If you try to put to much detail in a given process you lose your audience and the value of the documentation.
Bottom line when I am done with a design document it covers server, network, database and client topology in varying levels of detail with dataflow. A typical design document I would turn over would be 150 pages with most of that broken down into different sections describing what was done, why it was done, the best practices followed for build, and best practices for lifecycle. The document typically does not get read by any one person, instead it would be a reference for a number of different departments that will each reference it according to their own needs.
Re: (Score:3, Insightful)
From the sound of the original poster, he's in a tough spot. Way too much to do, and not enough time to do it (sole network admin).
I think this Dilbert ca [dilbert.com]
Re: (Score:3, Insightful)
You are both on and off the mark. First, I'm an enterprise architecture consultant for a living, I've done lifecycle administration in the past and do some now. I've certainly done audits, even to the point of being brought overseas, but that was only about 20% of my work. Once the audit is done my job was typically to follow up with how to bring things up to par. Staffing, architecture, servers, licensing and bandwidth considerations all come into play and receive my recommendations. I am far more likely t
Re: (Score:3, Interesting)
I find the more long words in someones job title, the less useful the function they actually perform.
Good title you have there.
Documentation at your hands, and timestamped (Score:4, Insightful)
If you are in the server room, and you have:
A: a spreadsheet that your predecessor made.
B: a post-it note on the switch saying it what it does.
Which one do you trust?
For the physical/low-level network the documentation should be in the network. Just like source code should contain comments about this particular piece of code, a similar approach works reasonably for the physical network. I see no point in a having an outdated spreadsheet. It is more useful that the cables and ports are labelled and numbered, that there is a post-it note on a switch say where the links go, etc.
The grand overview should be in electronic form, though. A scanned hand drawing is fine. A photo of a whiteboard drawing is fine too.
For the logical network put comments whereever possible. On settings, VLAN configurations, server connections, account setups, ...
Again, the grand overview should be in electronic form.
I have found it useful that the information is timestamped, so you know when it was last valid.
Re:Documentation at your hands, and timestamped (Score:5, Funny)
> Trust spreadsheet
You pull the wrong cable.
It's suddenly dark.
You get eaten by a grue.
The MACK(TM) Truck Rule (Score:3, Informative)
Ah, you're not following the MACK(TM) Truck Rule.
The MACK Truck Rule (MTR for short) is a measuring stick which we use do determine if a solution is good for us. Basically, it's an objective measurement of the level of expertiese required to do something. Basically, the MTR has you ask yourself (Or your team) the following question:
If the person(s) responsible for a task was suddenly hit by a MACK(TM) truck, How much time would it take for somebody else, untrained, to complete that task if needed?
If that amount of time is unreasonable*, It doesn't follow the MTR. Notice the caveat for unreasonable; this is the subjective part. What' unreasonable for one may be reasonable for another. This needs to be decided for yourselves.
Documentation always helps difficult tasks pass the MTR. So can good support. I try to leave a readme in the place where the installer is for a difficult program. I'm now begining to use FreeMind to map out networks and servers. I have a good ticket system for all our repairs. Hopefully these things will make things easier the day I want to take a vacation.
--Pathway
Re: (Score:3, Interesting)
We used to have a similar 'BUS' rule (ie, what if so-and-so got hit by a bus) until someone we all knew got hit by a bus. That sucked, he was a good guy and we had just referenced that joke a week earlier.
Now we have the 'LOTTO' rule (ie, what if so-and-so hit the lottery and left the company to be independently wealthy.)
We all miss Dave. And most of us secretly wish he had documented his fucking code before getting hit by that bus.
Wiki Wiki Wiki (Score:5, Informative)
I have a nicely formatted template page with all those categories set out. I also maintain a page of IP address assignments and an inventory of harware specs of all the machines in the office (which is helpful in the cases of "We need to reproduce a bug that only happens on ____ processor with ____ video card" and of "We're getting new machines. Who is in most dire need of an upgrade?").
I write down everything in these, and find myself referring to them very often. My predecessor gave me a Word document with all his notes in it, which has been very useful, and I used that as a starting point for my pages. The wiki has saved me a ton of time, kept me organized, and serves as a great reference for me and for the inevitable next admin.
The only caveat is if the wiki (or the server it's on) goes down. This has happened once, and my instructions for fixing the wiki were... on the wiki, so extra troubleshooting for me. Thus, I find it good practice to maintain a hard copy of the wiki pages, especially the page that tells how to fix the wiki.
I'm running this on Redmine, which has proven to be bleedingly simple to use and administer, and much easier than trac, which we used before. It's especially nice having it on the intranet, as I'll just have a browser open to the wiki as I work on systems and refer to and update it as appropriate. It's very handy to document exactly how I performed a strange or experimental installation of some software that I'll want to replicate later without making myself crazy, and I'll take the extra few seconds to retype the commands I just used into the wiki from anywhere in the building, though I probably wouldn't do the same into a Word doc.
It's not so much the mundane day-to-day that I find that important to document. It's the weird fixes, the trouble spots, the command line parameters, the installation procedures, the changes that shouldn't have fixed it but did, and the horrific chain reaction situations that make one piece of software crash because a seemingly unrelated piece of software has the wrong version of the 64-bit library. Things that take 4 hours to figure out and 3 seconds to implement... those are the ones to document, and those are the ones that I'd be kicking myself 20 months later for neglecting to write down. In an afternoon, any schmuck could walk into the building and figure out which network cable goes where. Documenting the strange bits (and the frustrations), though, can get a malfunctioning mail server back up and running in 3 minutes instead of 3 hours (which, of course, is secondary to good administration keeping the server from going down in the first place).
The overlooked parts (Score:4, Interesting)
There are a few things that are often overlooked and outright forgotten when documenting networks. I had to take over a few networks, let's see what I usually miss:
Every admin remembers to hand over passwords. Except for the routers.
Routers and other "managed black boxes" are notoriously being left out from the list of passwords. Fortunately, more often than not it's the standard password because "nobody has to touch them but me anyway" (ignoring that, if people only touched what they should, passwords would be moot...)
Every admin remembers to draw you a network layout. They don't tell you WHERE those switches physically are, though.
In large companies (read: Lots of room to cover, independent of the number of people working there), this can indeed be a problem. Especially when there's not one single server room where everything is collected, when you have switches and routers hidden in cupboards and other "innocent looking" furniture, cables that appear out of nowhere and disappear into walls, without an indicator where they surface again. Or what purpose they serve, first of all.
What HAS to be documented is the reuse of resources
That's the worst of the "undocumented changes". When you find a switch that shouldn't be there, you know you have to investigate, you know something wasn't documented. When you find a certain box sitting where it is supposed to be, you don't investigate. You expect it to do what it allegedly does. If it does not and has been "recycled" to fulfill another role, the whole documentation goes out the window. Because now you start questioning EVERY piece of hardware.
Size? (Score:3, Insightful)
If it is large enough (which it doesn't seem to be) you should divide stuff up in modules. Start from OSI layer 1 and work your way up.
If it's small, you probably wind up merging loads of stuff into one document in which a serious amounts of stuff is considered to be "the network" although it isn't.
/. to get this information. You're not the first person that has to do this. Must be a slow day here.
Having said this, there are places to go other than
start with nmap, then wiki (Score:3, Interesting)
2. set up a wiki
3. paste the results of the portscan into the wiki
4. start writing about everything that showed up
i've actually done this before with a pretty high degree of success, pm me if you want some help setting it up
Documentation of Networks (Score:3, Interesting)
My favourite technique for making sure documentation is done and updated, get the new guy to do it. Then he/she has to go all around the campus, locating servers and getting serial numbers form all sorts of odd equipment and making sure all of the support aggreements are current and the contact details for the vendors are accurate.
The other favourite is if I find new equpment that has been installed and is not labelled or documented, I get the installer responsible to audit all similar equipment to make sure there are no other ones missed out. After haivng to crawl around dozens of risers and labelling or confirming all switches etc are correct and documented, they don't often make the same mistake twice.
We also have a password management system which also allows details like how to install the management console or the URL to access a system for management to be stored.
My answer to any question about "What is the password for X", or "what the hell is the name of the server for X applicaiton" is "Its in the store" Then if it isn't, we add it 8) Only takes a few times for the newbies to start looking up the information themselves first.
The other key file is a massive Visio document with a summary page with a managment style overview, and then a document with everyhting in it in layers like an electircal diagram or building plan.
Lay in the workstaitons VLAN, the switch management VLAN, the Servers VLAN, link to things that are self contained like all of the Firewalls and DMZ configurations.
etc.
Two things you must do (Score:3, Insightful)
First: You must make everything as self-documenting as possible. Label every server, every cable, every power lead to within an inch of its life. And establish processes which say "when a cable is moved or added, labelling is updated accordingly". If you don't have a labelling machine, buy one.
That deals with basic "what's plugged in where" and is far more likely to stay up to date than a spreadsheet or wiki page.
Second: Whatever you choose, it must be something which can scale to your needs and which you can live with.
It will need regular updating - and quite frankly, very few people are able or willing to regularly update a single 200 page Word document complete with embedded spreadsheets, diagrams and photographs. A wiki - or even Sharepoint, if that's your thing - may be better. But if you do take the Wiki route, make sure you keep hard copies of the documentation which says "If the sh1t hits the fan, this is what you need to do to recover".
Others have said "don't bother, your successor won't read it" - I say balls. Documenting is more than just helping your successor - it also helps you remember what is set up, clarify how things work and as part of the process you start to look at things and think "hang on a minute.... this document I've written describes something quite absurd. Are we really doing that?"
Whether or not your successor reads it is really not your problem.
DoDAF For the Masochists (Score:3, Informative)
Re:Department of Redundancy Department (Score:5, Funny)
Now I see this Microsoft bashing on Slashdot regularly and it's completely unfounded. I use a Windows-based PC and it has provided me with years of worry, virus and spyware free operation.
In fact, our PC's at work are WeOffer popular brand names drugs such as ViagraFrom $1.87, CialiFrom $2.38, SomaFrom $1.07, TramadolFrom $1.38, LevitrvFrom $2.52, Celebre, Zocor, Fosamax, Effexor, Zyrtec, Plavix, Premarin, Flomax, Paxil, Zoloft, Prevacid, and Evista. Now it's easy to get your needed drugs 0nline.
Re: (Score:2)
Re:Department of Redundancy Department (Score:5, Funny)
An old girlfriend tells you you need viagra, and you still dont't get it.
Windows != SPAM (Score:5, Insightful)
Attempting (even facetiously) to blame SPAM on Windows is wrong. If every copy of Windows on the Internet somehow magically disappeared, the SPAM problem would not abate. Bot herders and spammers would simply shift their efforts to other platforms.
If your doubt this, consider what the winner of this year's PWN2OWN [tippingpoint.com] contest had to say about why it's easier to target Mac OS X [zdnet.com].
BTW, this is not a troll, and I'm not a (Windows|Mac|Linux) evangelist of any kind. I just find kneejerk Windows bashing rather tiresome
Re: (Score:3, Informative)
SAFARI on OSX is it's easier, not Mac OSX.p>
OK fanboi, did you even read the link I referred to? Here's an excerpt:
Re:False Info (Score:4, Insightful)
"The network started horrible, here is how I cleaned it up" is a GOOD reference. I have killer references from two jobs I automated myself out of this way. Each time I got a more interesting more challenging better paying job by doing so.
Re:For the sake of job security... (Score:4, Insightful)
That's always.
Those who don't document don't have job security. They are an insecure leach sucking up a paycheck fearing -- and rightfully so -- the day they are going to be replace.
Those who DO document show their value to the organization, and should have no fear of being replaced. Their position is secure -- and should they go elsewhere -- they have something to show of and for their work.
I disagree with the parent vehemently and will say so based on years of experience as a techie, a techie manager, a manager of techies, an executive, and (thankfully) a techie again. You can never document too much, but those who don't cost the organization more in the long run each and every time.
Document. Document well and often. Ignore the parent.
Ehud
You ARE mandated to document it (Score:3, Insightful)
All you are doing is wasting your companies money paying you salary for doing what they probably don't care about.
Doing a good job isn't just about doing what you are told. Just because management don't care about something doesn't mean you get to not care too.
Sometimes you have to do the right thing and often the right thing is helping the next guy who gets your job so that everything says running.
Re: (Score:3, Insightful)
Re:I am shocked at the suggestions here (Score:4, Insightful)
I also try to stay away from documenting things in static formats. If you ask me, incorrect documentation is even worse than no documentation at all, and the fastest way to get incorrect documentation is to create a process that relies on a person doing all of the updating manually.
I know a lot of people love their spreadsheets and their diagrams, and maybe they update them religiously. Nonetheless, that process is *always* prone to error. And if a technician goes to a document for information and finds out that the document is wrong, the document loses its credibility. If that happens a few times, the technician will simply stop trusting the document, and it will just fade into obscurity.
If you want to document a system, look for ways to make the system document itself. Switches keep real-time lists of the MACs that are connected to them. Routers keep real-time lists of which MACs map to which IP addresses. Routers and switches will always tell you their current configuration if you ask, and you can automate the process of asking and storing and checking for changes. Most servers will tell you their serial numbers automatically if you ask them, so you should automate the process of asking them and storing that information. The same goes for what kind of hardware is in the server, where the server is attached to the network, etc.
So much information can be collected automatically rather than recorded by hand, and when you collect the information automatically, it will always be up to date. It will not matter if a tech decided to re-rack a server in the wrong place -- even if they didn't write it down, your network knows that it moved, and it will tell you if you ask it. So the next time you sit down to write a 200 page document describing the network, you should ask yourself a) how much will it cost in time and effort to keep this document relevant, b) how likely is it that the document will become out of date either through accident or negligence, c) how quickly will people abandon this document if it does become out of date, and d) aren't there huge parts of this document that could be totally dynamic instead of written in static text on a page?
Re: (Score:3, Interesting)
And exactly how long ago was that?