CERN Engineers Have To Identify and Disconnect 9,000 Obsolete Cables (vice.com) 169
An anonymous reader writes: CERN, home to the Large Hadron Collider, has grand plans to update the world's largest particle accelerator complex in the next few years. But engineers have identified a barrier to the upgrade: there's no space for new cables in the injectors that accelerate particles before they enter the LHC. In the past, when parts of the accelerators have been upgraded or added to, engineers would often additionally replace the cables that connected them. In the process, they would leave in place the old cables that were no longer in use. Now, a heap of obsolete cables are blocking the way to install new ones needed for the accelerator’s next big upgrade. To make space, CERN engineers have set out to identify and remove the old, unused cables. All 9,000 of them.
Market idea (Score:5, Funny)
Sell them to audiophiles. You have a limited supply of cables used in a unique, world-class esoteric application. That's a perfect match for people with deep pockets and shallow skulls.
Re:Market idea (Score:5, Funny)
Sell them to audiophiles. You have a limited supply of cables used in a unique, world-class esoteric application. That's a perfect match for people with deep pockets and shallow skulls.
Oh yeah, oh yeaah! Here's the hook. Since the cables have been bombarded by high energy particles, they've had all of their atoms lined up in a manner which results in less frequency domain delay, and a purity of sound unmatched by mere normal cables.
That and special gold plated fuses, zebrawood knobs and those special rocks will give you a kickass audio system.
Re:Market idea (Score:4, Informative)
Re:Market idea (Score:5, Interesting)
Confirmed: http://moodybluesattitude.yuku... [yuku.com]
Re:Market idea (Score:5, Funny)
Whenever the discussion turns to audiophiles, I have to post my favorite audiophile gear of all, the $150 cable elevator. (currently on sale for $119, but on back-order)
http://www.reddragonaudio.com/... [reddragonaudio.com]
Re: (Score:3)
Machina Dynamica is the best. Did you see the other products? They are all just as ridiculous as that bag of rocks.
One of the best is that he can improve your gear over the phone, using "quantum teleportation", genius.
Thinking it was a joke, I proceeded to order something and had to stop just before paying. So yes, he will really take your money, and according to some audiophile forums, the guy is serious about his stuff, using customer feedback to "improve" his products and all that.
Re: (Score:2)
My god, that wins hands down. I have a new favorite audiophile gear.
Re: (Score:2)
But does the Optigrab do anything for "negative dielectric field interaction"? That's what I want to know.
wire wrap cable testing (Score:5, Interesting)
I am reminded of the days of wire-wrap circuit boards. hunders of wires in a few colors at most forming a rats nest of interconnects on the back. All done by hand from post to post where you had to count pins by eye to find the right post each time. Chance of 100% correct wiring was geometrically vanishing.
The problem was not discovering the connections you had failed to make (which is easily done with a continuity tester) but finding the connections that were mistakenly wired the wrong pins.
So what you did was go find a filament transformer (these were high current low voltage transformers used to power the filaments in tubes). then you put one probe on one pin, and another probe on every other pin it was not supposed to be connected to. This is not as complex as it sounds since normally one pin is not connected to more than 3 or 5 other pins. So once you eliminate those, you can just slide the probe along the sides of all the other socket pins.
The current was so large that even a momentary connection would vaporize the wire if it was incorrectly wired. A continuity tester would not have worked well because the response time for the human to test all N^2 connections and look at the continuity tester was too long.
Re: (Score:2)
I am not sure what continuity tester you have experience with but all of my digital multimeters going back 20+ years produce a tone in addition to a reading on the display and respond within milliseconds. Running the test lead along a string of contacts or pins and listening works fine with them.
Re: (Score:2)
Re: (Score:2)
That problem is usually due to connecting them the wrong way round.
Re: (Score:2)
That problem is usually due to connecting them the wrong way round.
That's almost as bad as crossing the beams!
Re: (Score:2)
Said bombardment might also have transformed the cables into low-level radioactive waste...
Bad Idea (Score:3)
Since the cables have been bombarded by high energy particles,....
Re: (Score:1)
With so many cables, is CERN reinventing the web ?
Re: (Score:2)
Yes, looks like they are going to need to hire contractors to do cable-jiggling. I worked in a company that was replacing their old funky official yellow and blue ethernet cables with vampire taps. Staff had to go round in pairs in order to slowly replace the network. One jiggled the cable that was being removed from inside the eaves, basement or the attic of the building, then the other disconnected it and tied string to the end, so that they could then haul the new cable across.
Amazing... (Score:1)
9000 cable... and no labeling
Re: (Score:2)
Sounds like my rack.
Re: (Score:3, Interesting)
Re: (Score:2)
Indeed. If you let it get much above 1000 you're just a slob.
Re:Amazing... (Score:5, Informative)
i use patchsee cables. a bit more expensive but you'll NEVER disconnect the wrong cable again. you get a specially shaped torch which you shine at 1 end of the cable and the other end lights up. the torch has 2 modes - stable light and flashing.
there are other brands (e.g. evo6, belden) doing similar things but they tend to be overpriced and overly complicated (cable with its own buttons and batteries in jack boot).
Re: (Score:2)
Sounds great, except:
- Are the cables transparent along a run of 100 meters or more? I doubt it.
- Do they STAY transparent for thirty years?
That's the case with CERN: long old cables, covered in dust. Some are probably quite brittle from heat or radiation damage.
Re:Amazing... (Score:4, Informative)
patchsee cables aren't transparent. they have 2 strands of optic fibre running from one end to the other. you shine into one end, light comes out of the other (no need to even disconnect the cable). the longest patchsee cables i've seen were 30 metres. i don't think they make longer ones.
Re:Amazing... (Score:5, Interesting)
9000 cable... and no labeling
Sadly this is common...
As a lab manager I had to institute a rule that ANY cable that didn't have a label was going to be removed when found with no warning. Any cable which was incorrectly labeled, was subject to be connected to what the label said, or if that wasn't possible, the label would be removed and then the cable was pulled for not being labeled. Label content was defined and all where trained on how to make proper labels, and retrained when they came to ask why their system suddenly stopped working.
Maintaining a lab is a daily discipline, like cleaning house. You have to pick up after yourself as you go along or at the end of the day the mess is huge. Hey, where you born in a barn? Your Mom doesn't work here, clean up your mess!
No label = must not be important (Score:2)
As a lab manager I had to institute a rule that ANY cable that didn't have a label was going to be removed when found with no warning.
Back in the day of floppy disks we had a similar rule. If the disk wasn't important enough to justify a label it was to be considered blank. I feel the same way about labeling cables. If you aren't disciplined about this stuff it can get really out of hand really fast.
Re: (Score:2)
Re: (Score:2)
As described in the article, they have such tools, but the facility has been remodeled many times over the 60 years it has been in operation so the database is no longer 100% accurate. The database is therefore useful to check which cables *may* be redundant, but one still has to check...
Re: (Score:2)
There was a reason they used to call the underfloor cabling in a machine room, the "snake pit". It was impressive. You lifted up a floor tile with a spike clamp, and there are hundreds of cables of all lengths and colors just stretched, coiled and bundled up together.
Re: (Score:2)
We call them SCRATCH disks, a carry over from the tape reel drive days, SCRATCH TAPES.
I even do the same with my HDD's for temp downloads etc, I call them SCRATCH. The idea being, you just temporarly scratch the surface with data. Short lived data.
And SCRATCH monkeys, always mount a scratch monkey.
Re: (Score:3)
On the boring side: I'm pretty sure that scratch tape/disk derives from scratchpads, pads of paper used solely for doodling or intermediate calculations and stuff that you don't want to enter into your (paper) engineering notebook.
Re: (Score:2)
And why did you get taught how to write on sheets of slate probably brought by the hundredweight form a building contractor? Because to buy paper and pencils, or even paper, pens and ink, was a continuing expenditure, while slates and slate pencils were a capital expenditure, but not a continuing cost.
Re:Amazing... (Score:5, Funny)
They are all colour coded... All grey ones are cables.
Re: (Score:2)
Re: (Score:1)
LOL, yea we had those kind of issues too. Good luck, it's never easy and if the problem has been let go very long, it can be a LOT of work to get things back in order. I almost just gutted the lab over the Winter holiday break once, roll everything out into the hall and bring it back in one piece at a time.... Sometimes it's just easier to just rip it all apart and reassemble..
Re: (Score:2)
I'm pretty sure you don't randomly pull cables on the $B high energy machine, it's liable to either break the expensive toy or make it go bang in interesting ways. I do get your point about a need for discipline though =)
Re: (Score:2)
You hear this all the time in any business at all clients. About undocumented access rights, old systems, phone numbers, applications, etc. My response has been: "Its going to fail, accept it. Do you want it to happen randomly and you have no idea what happened. Or do you want it to happen predictably and you know why." Sadly many pick the former because "it hasn't happened yet."
Re: Amazing... (Score:2)
geez.. the jobâ(TM)s no done til you clean up after yourselves.
its like those bad designers that leave al. sorts of unused mislabelled layers in a file.
install the new, remove the cruft. maintain a clean system.
It comes down to funding (Score:2)
Cable work is often times contracted out. One contractor makes a bid including the time to remove the old cabling even though it isn't specced, another contractor leaves the work to remove the cabling out because it isn't specced. The lower bid is accepted and funded.
The customer then asks the winning contractor why didn't remove the cables, Response "Not in the requirements". Customer then goes back for additional funding which is denied and the old cabling never gets removed. Seen it happen many times
Re:Amazing... (Score:4, Insightful)
Yes, but there's a point that doesn't work any more. The original injectors at CERN are more than thirty years old; some sites are probably 40 or 50. You find me a cheap and effective way of labelling cables that doesnt' fall apart over that time span...
Re: (Score:2)
They have regular periods of maintenance and when you inspect the cables if you see the labels are becoming worn then you replace them. You don't need to have labels that last 50 years. You just need to do your maintenance properly.
Re: (Score:2)
Any cable which was incorrectly labeled, was subject to be connected to what the label said
Good thing you didn't have network cables labeled "mains"...
Re: (Score:3)
Re: (Score:2)
If nobody kept the original circuit design laying around, just set fire to boat then drill holes in bottom to put out the flames and Abandon Ship!
Of course you could keep taking the owner's money for years too....
Re: (Score:2)
Re: (Score:2)
The owner's an idiot and will spend many times the wiring diagram's cost trying to maintain this thing, but that's his call.
Oh well, smile and keep at it until he runs out of cash or you can find another job I guess... Good luck!
Re: (Score:2)
if they're leaving them in place this often, i must imagine the CERN accelerator's conditions are less like a server room, and more like the wiring harnesses of an aircraft, and so they've been making similar decisions.
in such packed conditions disconnected cables and wires are frequently left in place (and in aircraft, rarely completely marked because of both too little space and the FOD hazard). the amount of manhours needed to redo an the wiring harnesses to remove a cable every time is simply not worth
Re:Amazing... (Score:5, Interesting)
I don't know for sure but I bet this was part of a penny pinching cost analysis up front.
I recall when moving to a new site setting aside some time/budget to ensure that every cable was labelled (so, for example, we could trace ethernet from port on switch to patch panel to underfloor cable to floor jack to desk cabling to desk port) and set up a simple database to keep the details.
Work was killed off by accountants as an expensive luxury, after all cables didn't move often did they?
Fast forward to a minor flood under the false floor taking out some (but not all) systems. Fortunately some of them were in the finance and commercial group.
Suddenly it was "why can't you reconnect me NOW??". Money was paid for an 'after the event' recording of wiring by external people (which cost about 5 times the 'saving' up front).
Still at least it was better than a LONG time ago [Vax and VT220 era] when I saw one person labelling connections by yanking out an RS232 cable from a patch panel, waiting for a call "My terminal's died", asking which room they were in and making up a label and then plugging it back with "I think that may fix it" and getting pathetically grateful responses in return.
Re:Amazing... (Score:5, Funny)
So THATS where the PFY has been all morning. I thought he was in the tape safe again.
Re: (Score:2)
I wires an office a few years ago and labeled every single network jack, wires going to and from it, the punch down pannel, all the same for the phone and even the wires for the alarm.
Come back two years later to troubleshoot static in a phone line and someone painted over every single marking not in the wire closet and they were unreadable. To make things worse, they lost the book with the legend and office map showing the routine and locations.
Re: (Score:3)
9000 cable... and no labeling
They have angered the Flying Spaghetti Monster, and will be punished.
Those who fail to learn the lessons of history... (Score:5, Informative)
Praise Buddha that removing abandoned cables is now a code requirement in the US. I remember an old server room where the manager wanted to raise the floor 6" so they could fit in more cables. 12" apparently wasn't enough...
Re: (Score:2)
Re: (Score:2)
Interestingly... Or not... it had flooded so many times that they did eventually need to have a hazmat team decontaminate the space. Ant that is why the stock exchange is now a gym...
hope they are labeled (Score:2)
Comment removed (Score:5, Funny)
Lame (Score:5, Funny)
LHC.....No wireless. Cost more than Fermilab. Lame.
Re: (Score:3)
LHC.....No wireless.
Off course not. They study particles, they do not simply trust those particles are doing their designated job. Before you know it A SYN passes out as an ACK at the other end of the collider...
9kA Wireless Transmission? (Score:3)
LHC.....No wireless.
The magnets in the LHC require a ~9,000 amp current and the ability to dump it somewhere fast in the event of a quench. Care to explain how you plan to do that wirelessly? It's also worth pointing out that the part of the accelerator complex they are recabling was built in 1954, 13 years before Fermilab existed and 17 years before the first wireless packet network [wikipedia.org].
Re: (Score:2)
More importantly: a there are thousands of devices that monitor the beam. Plus the actual detectors, which generate multiple terabytes of data PER MINUTE. I'd like to see the wireless hub that can keep up with that.
Re: (Score:2)
CERN - Now hiring! (Score:5, Funny)
Pay: This is an unpaid position, but the experience you gain will look great on your resume!
Hours: Don't Ask.
Required Qualifications: At least 10 years experience working with high energy supercolliders, 15 years experience with networking enterprise-level systems, and a minimum of a Master's Degree in Theoretical Physics (student experience will be considered)
Position: Entry-level.
Re: (Score:2)
The pay at CERN is pretty good actually.
You can find many open positions for it at www.unjobs.org
Of course the competition is insane.
Re: (Score:2)
Also, they are hiring at many levels, and do a lot of training / internships, so while the competition is tough, its far from impossible. Which are paid reasonable to very good, with good benefits.
Have a look at:
https://jobs.web.cern.ch/ [web.cern.ch]
Re: (Score:2)
Re: (Score:2)
I have a feeling that Gordon Freeman answered a similar ad.
Just like the phone cables under NYC streets (Score:5, Interesting)
Re: (Score:2)
Tinkering (Score:2)
Re: (Score:2)
The entire scientific publishing industry would disappear without your massive overload of papers too.
Nah... That's not a massive overload of papers, it's just that the author lists now take more space than the content [wsj.com].
Re: (Score:2)
CERN is a pretty strong supporter of open access:
https://cds.cern.ch/record/195... [cds.cern.ch]
Needs a positive spin (Score:2)
The engineers should turn it into a game: count how many cables they can disconnect in a hour and then try and beat that record. They'll be done in no time!
Re: (Score:2)
They are planning it as 4 year long project. With quite a few people on the team, average amount of cables you are expected to disconnect per hour is between 0.1 to 0.2. Might be hard to gamify that...
Sadly, easy to believe (Score:2)
I've been in relatively new commercial buildings where they had to replace entire conduits because they were literally packed with cables (most of them inactive). 9000 actually seems low for CERN.
They just need to make sure they don't disconnect the cable that keeps the speed of light constant.
If they know they are 9000... (Score:3)
If they know they are 9000, that would suggest they have already identified them.
It's not like the New York City Subway, where a combination of age and bad record keeping in the early years, combined with the fact that it's 3 or 4 different systems that merged into one system has led to most of the engineers not knowing what's down there at all.
You ask how many unused cables are in the NY Subway, and you'll get a shrug. Nobody knows. Hell, my favorite is when they break through a wall and find track and a train that nobody knew about for 60 years.
9000 unused cables? Pfft. That's not that impressive.
Re: (Score:2)
Not necessarily. You could know how many cables there are in total (just count them), and how many devices there are (because you track your assets). That'll tell you roughly how many without giving even a hint which is which.
They think they have it rough (Score:3)
Imagine trying to deal with this [theverge.com].
In related news, someone had to shovel the shit... (Score:2)
In related news, someone had to shovel the shit out of the barns for some farm research at a university in Iowa last night.
In other words, yeah there's maintenance and cleaning work associated with almost any non-trivial research project; just ask a grad student if you need more examples. Swapping out 9000 cables sounds probably like a day at the beach in some lines of work.
I've replaced cash registers at gas stations - (Score:2)
I know what they're going through......
(I seriously had 30+ lbs of small random cables piled up in the parking lot of a Galveston gas station one time as I pulled them out)
Billion dollar machine, but no tone generator? (Score:2, Troll)
Re:Billion dollar machine, but no tone generator? (Score:5, Informative)
a device that could potentially eventually produce a particle that could turn our planet into a blob of stranglets
Potentially you could stop writing catastrophic nonsense, too.
a tone generator would make the job far easier
This is the least of your concerns. If you can merely safely access both ends of the cable, that's already a big win. Remember that different cables go between different locations, and some of these locations are unsafe to access during operation, some of them are unsafe to access at any time due to presence of radioactive dust, and many teams keep different things inaccessible at various times for all sorts of reasons. You're basically assuming problems that aren't there. They have mounds of documentation detailing the routing of these cables. The cables do have identifiers, and I can state this quite categorically. This is CERN bureaucracy, they sometimes serialize their pencils for all I know.
The "have to identify" phrase is simply sensationalistic wording. They can point to each of these cables on the plans, they know what labels they have, that's not an issue. The issue is to physically get to the ends of each cable run and find them among everything else that's there, without breaking any rules and without disrupting anything where downtime can cost a thousand Euros per second.
Re: (Score:2)
Re: (Score:2)
This is basic networking stuff, CERN has some of the smartest people on the planet working there...I expected better from them LOL
When I worked on the Google help desk in 2008, I had to walk a recent Stanford University graduate through the process of turning on the computer. I explained to him that a cubicle farm wasn't the same as a computer lab and that he will have to turn on his own computer. You're be surprised by how many computer engineers don't know had a PC actually works in the real world.
Damn (Score:5, Funny)
One more cable and they could have gone Super Saiyan on the task.
OK, so our lab isn't that bad after all! (Score:3)
It's amazing how much cabling gets forgotten about when you have a chaotic lab environment and new stuff coming in all the time (we do hardware evaluations and other systems integration work.) There's never any money left over for structured cabling once it's been spent on all the fancy new hardware. Even if we invested in structured cabling it would turn into an unstructured mess quickly. I have racks that look like those Magic Eye pictures; the only thing that will solve it is unplugging everything. I'm sure world class scientists can't be bothered to label anything if we can't!
Re: (Score:2)
We had similar problems when I was in the Navy. Under the deck in the Missile Control Center was a rats nest of cables... probably half of them unused leftovers from previous generations of fire control system. (When I served aboard 655, she was on her third generation.) The cables were simply terminated and left in place because once the cables were installed and the fire control system installed above them, there was no practical and economic way to remove them during later upgrades, modifications,
Re: (Score:2)
I'm sure world class scientists can't be bothered to label anything if we can't!
I'm sure they do, and cables can be traced. The problem is not tracing the cable, it's physically removing it. You can't just yank when there's several tonnes of copper lying above you as I imagine a long cable tray which has had new projects constantly added to it would look.
They are going to need ... (Score:2)
Get that new guy gordon freeman to do the work. (Score:2)
Get that new guy gordon freeman to do the work.
Obsolete? It's new (Score:1)
It seems they just got this thing on-line and up to full capacity in the last couple of years. Now it's already obsolete?
I must be getting old or something: stuff seems to change so fast it's obsolete before even being used. Should I get bell-bottom jeans? I might still have a pair from the last bell era.
Re: (Score:3)
The facility has been active for over 60 years, and the cables are in some of the oldest parts (the injectors). So yeah, some of this stuff is obsolete; If I'm not mistaken, they ripped out a lot of 60s control electronics during the consolidation last year...
Skip a job that needs doing (Score:3)
Isn't there's a puzzle about this ... (Score:3)
Book: Puzzles for Pleasure [amazon.com]
Chapter: "Wire Wizards", page 73
* https://books.google.com/books... [google.com]
Seemed like a good idea at the time... (Score:2)
Same Issue Smaller Scale (Score:2)
Oh, one more thing.... M
Decommission as part of a life cycle (Score:2)
To make space, CERN engineers have set out to identify and remove the old, unused cables. All 9,000 of them
And that, children, is what happens when decommissioning is not a concrete, well-thought off, first class phase in a system's life cycle.
Talk to an old Telco engineer (Score:3)
This is called "cable mining" in the telephone biz (Score:2)
And Ma Bell has been doing it for a century. Cable rack in the central offices gets crowded after just a few decades, otherwise.
There's precedent, there are specialized tools and procedures for error reduction, and worldwide there are at least dozens, perhaps hundreds, of people with lots of experience in this very specific field.
Pfft. (Score:2)
Re: (Score:2)