Reverse Engineering Doctor Who Into Color 171
Lanxon writes "In 1967, the BBC set about junking its Doctor Who archive: a moment sci-fi fans wish they could travel back in time to prevent. There are 108 vintage episodes missing, but since 1978 a number have been rediscovered as 16mm black-and-white films. The BBC shot many of these series in color, but made monochrome copies for countries such as Australia, where many TV companies were still broadcasting in greyscale. The reels had sat in archives since. Now, the Doctor Who Restoration Team, an independent group contracted by the BBC, is using a new technique to regenerate The Doctor in color."
color (Score:5, Funny)
Since this is the BBC, they shot *none* of them in color but many of them in *colour*....
Re: (Score:2)
Re: (Score:2, Funny)
I'm from North America. Canada to be precise. And I would like you yanks to learn to spell as well. =)
Re: (Score:2)
I'm from North America. Canada to be precise. And I would like you yanks to learn to spell as well. =)
Have you got a spell for that Potter?
Re: (Score:2)
But we already know how!
"as well". A-S-W-E-L-L. "as well".
color, colour, couleur (Score:2)
Re: (Score:2)
Re: (Score:2)
There's a reason why NTSC is said to be a shorthand for Never Twice the Same Color (obviously not colour :-))
Re: (Score:2)
"Perfection At Last"?
Re: (Score:3)
"System Essentially Contrary to the American Method"
Re: (Score:2)
Surrendering Eaters of Cheese Abetting Mis-compatibility
Re: (Score:2)
Re: (Score:2, Interesting)
The point being that 'the BBC' didn't destroy these tapes, at most, ONE or TWO idiots who worked for the BBC destroyed these tapes - and you can bet your bottom dollar that they were managers, arrogant tossers with an overblown sense of their own importance, who destroyed forever all the hard work of scores of people, which can never be replicated.
Thanks for that, BBC!
I notice we are never given the NAMES of the idiots behind these decisions. Somebody must know who they are.
Re: (Score:3)
There is an article on the history of video recording in the production industry [bbc.co.uk]. The tapes (Ampex VRX-1000) weren't destroyed as in thrown out, crushed or incinerated. They were re-used to record new programming until the iron oxide was worn out. Given the relative costs of storage and purchase of each cassette, limited budgets, tight deadlines, the fact that copies were made for distribution, no producer or accountant would have given a second thought to overwriting the tapes for new programming. The same
Re:color (Score:4, Funny)
Re: (Score:2)
But, I'm sure counting lower class Indians as English speakers despite the fact that they aren't being taught it is equally valid.
Re:colour (Score:3, Insightful)
Re: (Score:2)
Re: (Score:2)
Re: (Score:3)
Re: (Score:2)
..but as a Second (third, fourth... ) Language the rest of the world outnumbers the US...
The US version is not dominant, nor the is British English, what is often called International English is ...
This has many small variations, spelling is optional, and pronunciation very regional, but it is distinctly not what is taught in US schools or UK schools ....
Re:color (Score:4, Informative)
And given that the rest of you guys can't settle on a particular dialect, I think it's a bit arrogant to suggest that our version isn't the dominant dialect
We can't settle on a particular dialect here, either. I can barely understand someone from the NE seaboard; they seem incapable of pronouncing the letter R unless it starts a word. "Da dyam dwag is unda da cah!"
Folks in the south have too many Rs. "Warsh thayut thar winder!"
Then there's jive, ebonics, tex-mex, board-room, 133t5p33k, txt, and those are ones I can think of off the top of my head.
Re: (Score:2)
Actually, the US alone accounts for roughly half of all native English speakers in the world
That's only true if you only count first-language speakers. India has about as many English speakers as the USA, but most of them also speak two or three other languages.
Re: (Score:2)
Re: (Score:2)
Sure there is. And furthermore, you'll find huge differences between different social and ethnic groups even within those cities. There's an infinite variety of subtle and not-so-subtle differences which constantly cross-pollinate. Any attempt to ring-fence something and declare it "standard English" is doomed.
Ted Turner's Revenge (Score:2)
And now for the nerdery. (Score:5, Informative)
So the article was devoid of anything of particular interest other than some jargon. The jargon, on the other hand, led to fascinating little technique about reconstructing the color of the grayscale image from "chroma dots". The actual method was discovered by a BBC engineer, and you can read more about it here: colour-recovery.wikispaces.com.
Re:And now for the nerdery. (Score:5, Informative)
Re:And now for the nerdery. (Score:5, Informative)
http://www.iwillvoice.com/faqpage.html#q3.6 [iwillvoice.com]
Question 3.6 from that FAQ seems to be that which specifically refers to this issue.
Re:And now for the nerdery. (Score:5, Interesting)
Re: (Score:2)
They are human, for god's sake. Maybe they shouldn't be going into making corrections, but I am happy that they are there. They are working to try to help partially fix the massive mistakes the BBC unknowingly made so long ago. They are doing some yeoman like work, so please don't completely diss them.
Re: (Score:3)
There is a good argument to be made that they SHOULD FIX NOTHING.
Just cleanup the black & white prints they have and stop there.
People who are even interested in some 40 year old TV show will likely suffer the black & white versions.
"Fixing" works primarily has the effect of annoying those that care about the work the most.
Colorizing is a generally bad idea because of this.
Re: (Score:3)
As long as they haven't made the oh-so-painful mistake of interspersing shots of a cheap, flimsy set which looks like something straight out of the 1960's with modern FX shots - they did that with the Red Dwarf remasters (though that was 1980's set) and my God, it was appalling.
It was Red Dwarf (Score:3)
It was supposed to be appalling.
Re: (Score:2)
So the article was devoid of anything of particular interest other than some jargon. The jargon, on the other hand, led to fascinating little technique about reconstructing the color of the grayscale image from "chroma dots". The actual method was discovered by a BBC engineer, and you can read more about it here: colour-recovery.wikispaces.com.
http://colour-recovery.wikispaces.com/ [wikispaces.com]
a link tends to work better.
Rev the wrong thing (Score:2, Funny)
Can they reverse engineer the scripts instead? Color or black and white, those old episodes are damn unwatchable. We'd be better off giving Wikipedia descriptions of the episodes to the writing staff of Golden Girls. Those old droning 5-part episodes would be turned into 22.5 minutes of tightly scripted comedy starring Bea Arthur as the Doctor. And any of the other old hags as K-9.
Re: (Score:3, Funny)
108 of them are, at least.
Re:Rev the wrong thing (Score:4)
Re:Rev the wrong thing (Score:5, Informative)
You might consult the Doctor who ratings guide [pagefillers.com]. Look under "Televised Adventures".
Many people like Pyramids of Mars, and the Talons of Weng Chiang, though the latter isn't particularly culturally sensitive. Genesis of the Daleks is another keeper.
Personally, I started with The Power of Kroll.
Re:Rev the wrong thing (Score:5, Insightful)
Many people like Pyramids of Mars, and the Talons of Weng Chiang, though the latter isn't particularly culturally sensitive.
I agree, but there's a wonderful moment when Tom Baker exclaims something like, "Wait a minute, you're Chinese," as if that visually obvious fact had eluded him up to that point. Made quite an impression on my young mind, that an alien -- even a super intelligent one -- would be less capable of seeing our trivial differences. To be truly unprejudiced, we must see through better eyes.
Re: (Score:2)
Maybe he suddenly realized that Li H'sen Chang is Chinese as opposed to Japanese, Korean, Vietnamese, or some other instance of Asian ancestry.
Re: (Score:2)
So you're saying that it's better not to see color?
Re: (Score:2)
Personally, I'd start with An Unearthly Child and continue chronologically with the highlights.
I understand it may be too much for some people, like a DBZ marathon, but it's important to see the context and evolution. A brief skim through doctors 1-2, then settle in and watch most remaining Pertwee, then watch 4, 5, 6 and 7. Once finished go back and watch the ones you missed. Then start on the reconstructions and consider whether the new series is worth it.
Of course for this you need an experienced Who fan
Re: (Score:2)
Re: (Score:2)
I loved that episode. Sutekh was quite malevolent and wonderfully voiced by Gabriel Woolf who, incidentally, the BBC brought back to do the voice of the Beast in the 2-parter David Tennant story of The Impossible Planet and The Satan Pit.
Re: (Score:2)
The acting and special effects were fine... considering the time of the original broadcasts and the intended audience (children). I quite enjoyed it as a kid/young teen, though yes, going back to watch some of those episodes now is painful. The same thing happens to a lot of old shows, though, and if nothing else, the mythology of Dr. Who definitely withstands the tests of time.
Re:The old doctors (Score:3)
Ah, memories of watching Tom Baker followed by an episode of 'The Tripods' then to be lulled to sleep by 'The Star Hustler'.... On Fridays and Saturdays I could stay up WAY past 9:00...
I seem to remember that episodes starring his immediate predecessor weren't bad either.
Before that it was black and white. I think I only ever saw Dr #1 discover the daleks, and Dr #2 probably wasn't bad, but he did look like one of the Three Stooges...
I also remember not liking to watch Doctor Who anymore starting with the
Re: (Score:2)
When they started rerunning the old episodes in Australia a few years back I really enjoyed them. The acting wasn't real good, the fight scenes (fist fights etc) were so bad they were funny, and the strings holding up the dalek's spaceship were visible and it rocked side to side, but I still really enjoyed them.
Re:Rev the wrong thing (Score:5, Funny)
When they started rerunning the old episodes in Australia a few years back I really enjoyed them. The acting wasn't real good, the fight scenes (fist fights etc) were so bad they were funny, and the strings holding up the dalek's spaceship were visible and it rocked side to side, but I still really enjoyed them.
They had not mastered String Theory at the time.
Re: (Score:2)
When they started rerunning the old episodes in Australia a few years back I really enjoyed them. The acting wasn't real good, the fight scenes (fist fights etc) were so bad they were funny, and the strings holding up the dalek's spaceship were visible and it rocked side to side, but I still really enjoyed them.
I never understood this. The acting in Doctor Who has always been good.
Look at other "sci=fi" stuff released by teh BBC and you'll understand my point.
You want bad acting? rewatch Blake's 7 I had fond memories of that show during the 80's and found the acting to be so horrible when i watched it again recently that I couldn't believe i missed it the first time. But Doctor Who episodes? Nope, i rewatch them ever few years and don't have a problem with the acting at all.
Re: (Score:3)
I've been rewatching the original Star Treks, and their quality is all over the board. Some episodes have great writing from great writers (the episode I watched last night was written by Theodore Sturgeon), great directing, and great acting, but others are painfully bad.
TNG and Voyager had some episodes that were so scientifically wrong they were painful to watch (like when Paris hits warp ten and evolves... whoever wrote that episode knew absolutely nothing about evolution, and I think there's an equally
There's Good News and Bad News... (Score:5, Funny)
The good news is that they've figured out how to restore colour to the B&W negatives. The bad news is that it requires Kodachrome processing...
Re:There's Good News and Bad News... (Score:5, Informative)
Actually, the process by which they're recovering the colour data is very interesting:
http://www.insell.co.uk/colourisation/Recovery_of_Colour_Information_0-2.htm [insell.co.uk]
Re: (Score:3)
The good news is that they've figured out how to restore colour to the B&W negatives. The bad news is that it requires Kodachrome processing...
No problem there at all. Just use the Tardis and go back to the heyday of Kodachrome processing. For this...it really helps to use a Time Lords trick of thinking inside the box.
Re: (Score:2)
Re: (Score:2)
That's silly, if you're going to go to that trouble, you may as well go back to 1967 and stop the films from being destroyed. Or better yet, take them out of the bin and bring them back to the 21st century.
Sorry, can't. Since they were actually destroyed, messing up the timeline is a no-no.
Oh, but wait! We could copy them and put the originals back. Argh, can't do that either. Curses, RIAA.
Re:There's Good News and Bad News... (Score:4, Insightful)
No, but what if someone convinced the BBC to knock off 6 more copies [wikipedia.org], then sealed them up in a wall so they could be picked up later and sold to collectors?
Yeah, sure, they'd have "This is a fake" scrawled all over them, but since a copy by the original producer can't really be a fake...
Re: (Score:2)
That's silly, if you're going to go to that trouble, you may as well go back to 1967 and stop the films from being destroyed. Or better yet, take them out of the bin and bring them back to the 21st century.
Right now we are still trying to colorize Ted Turner.
not so new, but still cool (Score:2)
this isn't a new technique - TFA even says it's a refinement of a technique that's been used before.
damn cool though, to get the crap from the colour subcarrier that spilled into the luma image and re-generate the original from it.
good thing those old kinescopes were in focus!
Excellent (Score:2)
"Computer Science: it works, bitches!"
For a taste of recolored Who, see Babelcolour's videos [youtube.com] (hand-recolored, frame by frame)
Re: (Score:2)
Got a torrent?
Re: (Score:2)
"This video contains content from Sony Music Entertainment, who has blocked it in your country on copyright grounds." Got a torrent?
What a nice look into the future. We might have colours, but we're not allowed to enjoy the media because we live in the wrong place.
Re: (Score:2)
"Science: it works, bitches!"
FTFY.
Why couldn't they have lost the right ones? (Score:3)
Re: (Score:3)
Re:Why couldn't they have lost the right ones? (Score:4, Informative)
I did enjoy some of the Colin Baker episodes. Seriously, the guy did an amazing job considering the crap they were putting it through, and it's the producer John Nathan-Turner who would have been better lost, as he seemed determined to make the series die a slow death. Being forced to retake scenes requiring strong emotions multiple times just because "that prop in the background still isn't quite right" must have been soul destroying for the actors.
The good news is that Colin Baker is still doing Doctor Who via the Big Finish Productions [bigfinish.com], where he is given good scripts and is well liked among fans. Nicola Bryant seems to have settled into the role well too, and no longer sounds like she's about to burst into tears after every sentence.
Re: (Score:3)
I love the sixth doctor in the Big Finish episodes. BF has really allowed him to dig into the dark side of the character and do some complex stories that JNT would of never allowed. Colin Baker showed up at a really bad time at the franchise when they made some really bad decisions about the direction of the show (i.e... giving the Doctor too many unlikeable properties at once, too much domestic fighting in the Tardis, a horrible costume idea, etc..)
Re: (Score:2)
I liked Peri's accent. So enticing. So sensual. So.... sexy.
Do all America women talk like that? Because if they do, I'm packing my bags and heading across the pond!
Not a bad idea. (Score:2)
At least, if they restrict this to those serials that were originally shot in color. I would be a bit uncomfortable if the older, black and white originally, serials were colorized.
Re: (Score:2)
If you read the article, they're actually recovering some of the colors from the originally color serials. This means it doesn't have to be done all by hand. Hand-coloring the originally B&W serials would take a lot longer and require rather more artistic license.
Re: (Score:2)
Much more technical information (Score:3)
On the restoration processes used in the past can be found on the RT's website, if you dig around a bit: http://restoration-team.co.uk/ [restoration-team.co.uk]
Facts (Score:5, Informative)
The politics behind the Chroma Dot story is intriguing and in some places unpleasant. The instigator of the team was James Insell, and a method was created to perform the chroma dot extraction by a man named Richard Russell. Insell became a bit proprietorial over it all, and he and Russell parted ways, and now Russell it doing it alone. The original Colour extraction blog is here [wikispaces.com] but they don't seem to have made any huge advances since Russell left. There is some more info, plus a link to Russell's own work (including software download) on my own Dr.Who webpage here [paullee.com]
Re: (Score:2)
For their next trick... (Score:2)
more proof (Score:3)
Re: (Score:3)
It goes even deeper than that - from what I recall the episodes were destroyed (rather than lost) by the BBC as the TV production unions were charging huge royalties simply for storing the episodes, during the "home taping is killing the television industry" craze of the 70s. It is excessive copyrights "protection" that caused the destruction in the first place - and being so popular, DW is one of the few shows from that time that survived in any form (mainly thanks to piracy and actual theft as all officia
We've got the scripts, right? (Score:2)
This is the 2nd Color Restoration of a Doctor Who (Score:2)
B&W broadcast tricks (Score:2)
What most people don't realize is that K-9 was actually a zebra!
Re:technique (Score:5, Informative)
It almost is reverse engineering. The chroma subcarrier in a video signal has a center frequency picked to allow the sidebands to fall between those of the main lumanance (black and white) video. The spectrum of those extends out from the main visual carrier frequency (or up from D.C. for the baseband signal) at multiple of the horizontal scan rate. The goal was to add color broadcast information to an existing greyscale system while introducing a minimal amount of interference. Here people are figuring out what is going on from the visual interference.
The added signal amplitude represents the amount of color added/subtracted from the greyscale white, and the phase represents the hue. The phase of the signal is compared with a short burst (a minimum of eight cycles) sent just after the horizontal sync pulse prior to the start of video on each scan line. PAL, as used by the BBC, is very similar to NTSC, except the scan rates differ, the phase of the reference signal is inverted on every other line to help cancel out the effect of small phase errors on tint.
Basically, those trying to recover color from the back and white films of on-air video have to use a comb filter to pick off the frequency (precisely related to the inverse of the spacing) of the resulting dots that are there from the color signal. The position of the dots from left to right carries the phase information. Considering that the dot pattern is probably quite weak, the resulting color would be noisy. Depending on the filtering used, the bandwidth (detail) may also suffer. But it is still a good starting point to know what the colors were.
The dots aren't on/off like pixels. It's actually a sinusoidal intensity variation. I recall some older Zenith B&W sets had particularly good detail (and maybe some video peaking - enhancement) making it easy to see which programs were broadcast in color, and what parts of the picture were deeply saturated. In addition to a notch in the video response at 4.5 MHz to filter out patterns from the sound, some sets rolled-off or notched centered at 3.58 MHz (3.579545 actually) video response to reduce the interference. Better later sets (and color generally) used "comb" filters to separate the interleaved spectral components without those loss of detail seen with more primitive methods. Failure to filter color signals could cause wild colors/patterns on things like striped neck-ties when a shot zoomed in/out.
It's pleasing to see that there are still a few around that understand the old analog technology well enough to realize there were visual color cues remaining. Even those that understand the electronics well often don't associate a particular visual characteristic with the responsible signal attributes.
Although partial signal recovery is easy to envision with analog electronics, something along the lines of a GIMP/Photoshop plugin could work as well. Some might think of it as being similar to watermark detection.
Re:technique (Score:5, Interesting)
Re: (Score:2)
My mom told me once she was watching a black and white TV with her family, and someone walked on the screen with green hair. Everyone watching the TV instantly started laughing because the guy had green hair. I don't entirely understand your post, but it does verify that my mom was not crazy, and average people watching in those days could distinguish even if they didn't know what was going on.
http://en.wikipedia.org/wiki/The_Boy_with_Green_Hair [wikipedia.org]
Re: (Score:3)
The colour distortions NTSC and PAL suffer from on high frequency luminance changes are interesting and a fascinating insight into the world before digital. Every geek should IMO understand this because it's interesting and cool and easy to understand. I'll try and simplify the explanation...
The artefact in question results from a colour (analogue) TV trying to display a picture where the luminance (brightness to you and I) changes rapidly, i.e. with a high frequency. Viewers of old will be used to people w
Re: (Score:2)
In the era of the 1980 home computers, It was an advanced programming technique [atariarchives.org], on the same level as VGA programming in the 1990's, or GPU shader programming is now. It was more of technological curiosity, with only one or two games used it (Tetris), since the results varied from TV to TV. It did seem strange how using a 320x200 monochrome screen could generate red, green, yellow or blue pixels just because the pixel size was smaller than the sampling rate of the CRT tube.
Re: (Score:2)
Well, I've heard of people being able to determine the content of a wav file by just looking at the raw data, so...
Re: (Score:2)
"the guy had green hair. I don't entirely understand your post, but it does verify that my mom was not crazy,..."
No, she was colorblind, it was a redhead.
At least she was polite enough not to comment on his transparent trousers. You were really naive to think she was laughing at his green hair.
Re: (Score:2)
dumb question but why doesn't it just work? (Score:2)
Re: (Score:2)
If the colour subcarrier is there then why doesn't it just show in colour when displayed on a colour TV?
I think the point is that the color subcarrier isn't there; all you have are the errors from the color subcarrier bleeding into the luminance part and now those errors are being used to extrapolate and restore color to the film.
Re: (Score:2)
Because colour TVs don't interpret chroma dots to display colour, they use three different signals for RGB. Engineers would have had to build chroma-dot interpretation into the colour TVs.
Things don't just work. They have to be made to work. Are you in middle management by any chance?. Do you use the phrase "Make it so Number One"?
Sorry this is wrong (Score:2)
" ... Why doesn't it just work ... "
Because colour TVs don't interpret chroma dots to display colour, they use three different signals for RGB. Engineers would have had to build chroma-dot interpretation into the colour TVs.
Things don't just work. They have to be made to work. Are you in middle management by any chance?. Do you use the phrase "Make it so Number One"?
I know enough to know this is wrong, PAL uses a colour difference signal. Fine patterns in the luminance signal do show up as colours, check patterns would often show as strobeing colour on older colour TVs.
Re: (Score:2)
Re: (Score:2)
Note: the below applies to PAL, i'm not familiar with the details of NTSC but similar stuff probably applies there too.
For a color TV to receive a color picture a few things are nessacery.
1: the information has to be alternating in encoding on each line and at the right frequency,
2: the chroma signal must be strong enough
3: there must be a "colourburst" at the start of each line (outside the viewable area) to notify the TV of a colour image and to synchronise the oscillator in the chroma demodulator to the
Re: (Score:2)
I am sure this is a dumb question but why doesn't it just work?
From TFS: "There are 108 vintage episodes missing, but since 1978 a number have been rediscovered as 16mm black-and-white films. The BBC shot many of these series in color, but made monochrome copies for countries such as Australia, where many TV companies were still broadcasting in greyscale."
There is no color subcarrier in 16mm black and white film.
Re: (Score:2)
I am sure this is a dumb question but why doesn't it just work?
From TFS: "There are 108 vintage episodes missing, but since 1978 a number have been rediscovered as 16mm black-and-white films. The BBC shot many of these series in color, but made monochrome copies for countries such as Australia, where many TV companies were still broadcasting in greyscale."
There is no color subcarrier in 16mm black and white film.
Also from TFA:
"But when they made the black-and-white recordings, they didn't filter off the colour carrier [encoded as a 'chroma dot' pattern in each frame]
Re: (Score:2)
Because the film is distorted, and includes both interlaced frames in each image. So you scan it at a high resolution first (2k lines, for example).
So they need to de-distort. And as you will remember from your old CRT monitors and TVs, the image distorts according to the image displayed, so each frame needs to be de-distorted individually.
Then de-interlace. Then extract the information required. Then re-construct.
Re: (Score:2)
You mention the fact that pal alternates the phase on every line. IMO this is probablly among the hardest problems for those doing the recovery. This means each line has (roughly) the same color information but encoded in opposite ways.
When you are working directly off a video signal this is no problem, counting lines is easy. But when that video signal has been translated into a 2D analog image it means you are going to have to seperate the lines (which are probably bleeding together). I suspect this is a
Re: (Score:2)
Amazingly enough, the original implementation of 'Colour Recovery' was done in BBC Basic for Windows [blueyonder.co.uk] - an extension of the language used on the old Acorn computers in the 1980s.
That link is a much more technical overview of the process - the first time they used Colour Recovery, they used it as a supplement to more traditional computer colorization. They had an outside firm (Legend) do a hand-colorization of a black and white episode using an improved version of the old Ted Turner colorization process. Whil
Re: (Score:2)
All they really need to do is to give the prints to Turner [reelclassics.com].
I t6hink they are basically doing the same thing but extracting the original colour from vestigial chroma signals, rather than using what some artist thinks the colours were like..