Forgot your password?
typodupeerror
The Military Data Storage United States IT

Losing the War Data For Iraq and Afghanistan 62

Posted by timothy
from the if-you-save-everything-you-save-nothing dept.
cervesaebraciator writes with an excerpt from an analysis of a kind we're likely to see more of as ubiquitous sensors and cheap storage continue to proliferate: "'The Iraq and Afghanistan campaigns are unique in that they were the first wars to be documented electronically. The use of computers to track stabilization efforts produced enormous datasets in which important indicators were tracked, including daily electricity-production rates, georeferenced insurgent attacks, factory employment numbers, military spending on locally sourced goods and services and public opinion. [...] Army Secretary John McHugh recently admitted to members of Congress that thousands of records from the Iraq and Afghanistan conflicts are missing. [...] The problem is that much of the existing data were collected in an ad hoc manner that reflects the lack of planning for stability operations following both invasions. While certain data types were methodically maintained, others were kept by single individuals in more arbitrary ways—in some cases, on a single computer's hard drive, in a personal computer or within an e-mail account. As flash drives are lost, computers reformatted, files erased, and human and magnetic memory degrades, various data types have been and will continue to be destroyed." With apologies to Santayana, those who do not backup data sets of the past are condemned to repeat them."
This discussion has been archived. No new comments can be posted.

Losing the War Data For Iraq and Afghanistan

Comments Filter:
  • by Anonymous Coward on Tuesday August 13, 2013 @08:46AM (#44551425)
    The problem is that much of the existing data were collected in an ad hoc manner that reflects the lack of planning for stability operations following both invasions.

    With each generation the prior generation of technology often looks ad hoc or patched together. Given that these operations happened over a decade ago it's no surprise that the data was handled poorly by today's standards.
    • George Santayana once may have said "Those who cannot learn from history are doomed to repeat it." https://en.wikiquote.org/wiki/George_Santayana [wikiquote.org] and that would be a terrible shame consider the loss of both life and data.
    • The units brought their own network gear and staff, and it all departed at each RIP/TOA.
      Much harder to track down and prosecute anyone there for much of anything, for reasons of justice, or to slake our emerging lust for kangaroo courts.
    • by clodney (778910) on Tuesday August 13, 2013 @09:35AM (#44551861)

      The problem is that much of the existing data were collected in an ad hoc manner that reflects the lack of planning for stability operations following both invasions.

      With each generation the prior generation of technology often looks ad hoc or patched together. Given that these operations happened over a decade ago it's no surprise that the data was handled poorly by today's standards.

      I find it hard to believe that anybody is the least surprised by this. Look around your organization. Surely there is some guy down the hall who has taken it upon himself to keep track of something that is not required but that makes his job easier or piques his interest. After awhile people start to realize that he has a list of which customer has been sent which update, something which for some reason is not tracked in the CRM, but is sometimes very useful to have. He faithfully maintains the list for several years, until he moves to a different job. Turns out his successor does not find the information as useful, so stops collecting it. 2 years later it is hopelessly out of date and it gets deleted.

      There were *millions* of people involved in the Iraq and Afghanistan wars. That kind of semi-official record keeping had to have happened thousands of times. Suppose I work for a battalion of engineers doing electrical grid repairs. The CO has to make a report to Brigade every month on various metrics. Some staffer compiles the info every month for a Powerpoint. After the tour ends and the CO is no longer reporting to Brigade every month, why would I continue to maintain the data? Who is going to come asking for it? So I delete it. Now repeat that for thousands of records.

    • Re: (Score:2, Troll)

      by Jawnn (445279)

      The problem is that much of the existing data were collected in an ad hoc manner that reflects the lack of planning for stability operations following both invasions. With each generation the prior generation of technology often looks ad hoc or patched together. Given that these operations were handled by no-bid contractors, engaged by an administration rife with corruption and crony capitalism, it's no surprise that the data was handled poorly by today's standards.

      TFTFY

    • by hairyfeet (841228)

      Its not just that, according to a friend that spent 8 years in the military he said he was amazed when he started how much of the stuff was just thrown together by grunts because there was so much redtape and bullshit involved in going through channels that a lot of the grunts were just cooking up solutions on their own.

      Now I've never been in the military so i don't know how much was true and how much bullshit but from what he was describing to me it sounded a hell of a lot like what I went through when

    • by Anonymous Coward

      I was there and I can tell you first hand that the data (much of it classified) I created and used in 2004 was inappropriately stored and archived even by 2004 standards, which were not that far off from standards of today.

    • by greenbird (859670)

      With each generation the prior generation of technology often looks ad hoc or patched together.

      Apparently you've experienced a multi-billion dollar company tracking a critical financially function in a 40 tab Excel spread sheet. Today. This generation.

    • by gl4ss (559668)

      ...yeah, except invasions aren't really a new business by any standard.

      it sounds like the data was poorly kept by 1940's standards, for whatever reason.

  • " The problem is that much of the existing data were collected in an ad hoc manner that reflects the lack of planning for stability operations following both invasions. " bureau-speech for "We didn't plan well enough, so no one can be blamed."
    • Re:How convenient (Score:5, Insightful)

      by drinkypoo (153816) <martin.espinoza@gmail.com> on Tuesday August 13, 2013 @09:21AM (#44551727) Homepage Journal

      " The problem is that much of the existing data were collected in an ad hoc manner that reflects the lack of planning for stability operations following both invasions. " bureau-speech for "We didn't plan well enough, so no one can be blamed."

      Uh no. It's bureau-speech for "we planned well, so that no one can be blamed". They can lose any data they like as long as they claim that this is standard, and never get in trouble for burying any kind of evidence. Next to be lost: The NSA's records of who they've been spying upon.

      • Sounds like standard operating procedure. I, for one, hope nobody was surprised. The less said, the better.

      • by oodaloop (1229816)
        So when a dataset was saved on one laptop, that was planned from the beginning from someone at the top? Is your tinfoil hat on tight enough? You seem to have a lot more confidence in the competency and long term planning of our government officials than most people.
      • by hairyfeet (841228)

        Remember the malice/incompetence rule Drinkypoo, the military is just like any large org with PHBs and BOFHs and grunts that jury rig shit up just so they don't have to deal with the PHBs and BOFHs, I used to work with a guy that was ex military IT and from his stories you could have just replaced rank with corporate titles and his military stories and my corporate stories would have been interchangeable.

        Could some of it be because of malice or coverup? Sure but I bet a hell of a lot more of it was just

        • by drinkypoo (153816)

          Remember the malice/incompetence rule Drinkypoo

          You know, the saying is that it's just as dangerous to overestimate your enemy as to underestimate him, not that it's only dangerous to overestimate him.

        • I once heard the comment, "If the Soviet's had developed a laser that dissolved tape, NATO would have been f*cked."
          • by hairyfeet (841228)

            Well he was in Desert Storm and he said "The thing we had too much of was tape, both red and duct, what did we have too little of? Everything else". he said you'd be amazed how much shit was just improvised or jury rigged at the squad level because nobody had thought to provide it and you had a couple PHBs above you in the chain of command that were "trying to show their leadership skills" by basically cockblocking any and all requests.

            Like I said if you changed rank to corporate titles you really couldn'

  • by Anonymous Coward
    How much was lost, and how much purposely erased?
  • Don't panic (Score:5, Funny)

    by ebonum (830686) on Tuesday August 13, 2013 @08:56AM (#44551509)

    We can start another war and re-build the data set.

  • Ask the NSA (Score:5, Funny)

    by bengoerz (581218) on Tuesday August 13, 2013 @08:58AM (#44551525)
    I hear those guys have a copy of everything.
  • by Errol backfiring (1280012) on Tuesday August 13, 2013 @09:00AM (#44551539) Journal
    Just build some foam LARP missiles and play it again, preferably in some remote desert.
  • > cervesaebraciator writes
    > cervesaebraciator

    That is all.

  • by Anonymous Coward

    During WW2 the Germans did the same with their archives when the world started to understand what they were really up to.

    What they want to keep secret must be very ugly.

  • by Anonymous Coward
    I've had the displeasure to apply for Federal IT jobs. Their hiring system is so messed up, that anyone that makes it through is either a masochist or is a form filling savant. It's no wonder creative talented people stay away. Even if a creative type were to be hired, government "standards" would drive them insane.

    If the government can't get good people, it's no wonder they can't handle all that data.
    • by alen (225700)

      yeah, because if someone were to take a paperclip home from the office and the media found out and ran some story about employees stealing from the government people would be outraged.

      every time people get outraged, more rules are added. then people get outraged about the rules

  • Found them! (Score:3, Funny)

    by Sparticus789 (2625955) on Tuesday August 13, 2013 @09:27AM (#44551771) Journal

    Did you check Snowden's laptop?

  • by dkleinsc (563838) on Tuesday August 13, 2013 @09:50AM (#44552037) Homepage

    In wartime, militaries do some really nasty stuff. Ever since Vietnam, the US military has made it a policy to keep hidden from the public the facts about what they do, because they firmly believe that the US would have won the war had they not gotten into trouble with the US public for walking into villages and killing everybody [wikipedia.org].

    And no, more modern wars have been no different in that regard: For example, the US has used drones to launch missiles at weddings and funerals in Afghanistan. There's the infamous "Collateral Murder" video which shows US soldiers gunning down unarmed civilians trying to rescue wounded unarmed civilians. The US has acknowledged torture (by the definitions the US used before they got caught doing it) of often innocent prisoners in Gitmo and Abu Ghraib. There have been some very suspicious "suicides" of prisoners of war.

    I'm not saying war is never worth it, but you have to remember that in war all conventional morality is thrown out the window pretty quickly.

    • by Sparticus789 (2625955) on Tuesday August 13, 2013 @11:43AM (#44553455) Journal

      The CM video is a perfect example of playing Armchair General. The objects being carried by the "civilians" had the same profile as RPGs. Only from post-engagement analysis can you tell what they are. The video is a grainy night-vision camera from miles away. Nor can you identify the "children" in the video, they are literally small white blobs on the screen. If you watch the video, without preconceived notions, you can easily identify that it was legitimate engagement.

      According to the Rules of Engagement from the 2006-2007 time frame, if any person from a crowd of group of people commit a hostile act or show hostile intent, the entire group is displaying hostile acts or hostile intent. In that crowd which was shot by the Apache, one person in the group was displaying hostile intent (carrying an RPG, which has so legitimate self-defense purpose) so therefore, can be engaged. If it was an AK-47, they would not have been able to engage. I know this because a farmer across from my guard tower would carry his AK everywhere and we were specifically told that carrying an AK was not hostile intent. If it was a PKM (machine gun), the group would also have been displaying hostile intent. Another group of people showed up and were in the field of fire. Therefore, they become part of that group and are showing hostile intent.

      When you stop sipping on your latte's and get a dose of reality, the video is a legit engagement. Sorry for the innocent lives lost, but that is part of war. If you do not want to run that risk, then you should run away when you hear 30mm machine gun rounds land in your area, not drive towards the weapon fire.

      • by m00sh (2538182)

        the video is a legit engagement.

        Sure, sounds really convincing. Profile of an RPG (however you want to define what an RPG profile is) equals license to kill children.

        he CM video is a perfect example of playing Armchair General.

        And your explanation is the perfect example of spin. Yeah, there was an RPG profile in there somewhere and so everything is legit.

        • Oh, I didn't realize Hawkeye was a member of Slashdot. I am sure with your superhuman abilities you were able to identify a white blob on an infrared night-vision camera as a child. It could have also been the following: paper plate, basketball, laser disc, bowl of hummus, block of C4, 155mm artillery shell, or a Mac G4.

          It's not spin. It's called the Rules of Engagement. You have a problem with them, you should have voted. Otherwise, it is better to be judged by 12 (or a nation) than be carried by six

      • by dkleinsc (563838)

        The objects being carried by the "civilians" had the same profile as RPGs. Only from post-engagement analysis can you tell what they are. The video is a grainy night-vision camera from miles away. Nor can you identify the "children" in the video, they are literally small white blobs on the screen. If you watch the video, without preconceived notions, you can easily identify that it was legitimate engagement.

        Even if I concede those points, you still are in trouble, because of what happened next:

        Another group of people showed up and were in the field of fire.

        I'm going by my memory of it here, but IIRC the shooting had stopped, and there were a bunch of wounded and dead people on the ground. Some people, without anything remotely shaped like an RPG, drove up in a van and attempted to move the wounded people from the ground to their van. They did nothing that suggested a hostile act towards the helicopter, and were carrying no weapon that would be any kind of threat to a helic

        • As the AC pointed out, for a vehicle or person to be qualified as a non-combatant (limited to medics and chaplains), their vehicles must be marked with a Red Cross/Red Crescent. Chaplains wear their religious insignia on their uniform to denote their non-combatant status. As AC mentioned, the vehicle did not have the required markings to be an ambulance. Therefore, valid target. And, it also makes the parties driving the van, assuming they were acting as an "ambulance" guilty of war crimes.

          Following the

          • by dkleinsc (563838)

            By your logic, if an army invades a city, everybody in that city not wearing a Red Cross / Crescent / Crystal is a valid target. That is plainly incorrect: The Fourth Geneva Convention Article 4 defines civilians of an occupied country (which Iraq was at this point) as protected persons, which makes them not a legal target. In addition, Articles 16 and 17 make it clear that you're not supposed to shoot at wounded people who are not attacking you.

            What it seems like you're steadfastly ignoring is that everyon

            • "By universal agreement and practice, the law of war draws a distinction between the armed forces and the peaceful populations of belligerent nations and also between those who are lawful and unlawful combatants. Lawful combatants are subject to capture and detention as prisoners of war by opposing military forces. Unlawful combatants are likewise subject to capture and detention, but in addition they are subject to trial and punishment by military tribunals for acts which render their belligerency unlawful

      • by stymy (1223496)
        Rules of Engagement don't give soldiers license to commit war crimes. I thought things like that were established in Nuremberg.
  • War is necessarily an imperfect business. I'd rather devote resources to the MIA soldiers from Vietnam and locating our lost submarines from WWII than worrying about what happened to documents about Iraq and Afghanistan. People have to come first. Leave no document behind ranks way below leaving no man behind.

    • Documents can do things such as verify toxic exposure to Afghan and Iraqi burn pits.

      Their loss in past wars has caused vets many problems.For example, my friend worked VC-123Ks while TDY at Da Nang, but only has a single order amendment and no original order to verify boots-on-ground and exposure to Agent Orange.

      The MIAs are dead (any VC in his right mind wouldn't bother saving useless prisoners) and the submarines are already war graves. The dead are dead, rituals over the dead are nice, but hundreds of th

      • the VA is a whole can of worms of its own. The C-123 fleet, as you have said, has been a problem, but even so, the military has vehemently fought claims related to this despite even obvious documented evidence.

      • Anecdotal evidence only:

        Many people I was deployed with have been developing hypothyroidism and severe weight gain. Not a few pounds, but 50+ pounds which started after returning from Iraq. Even once they are on thyroid replacement medication, the weight gain still stays. Not trying to claim that my observations are statistically significant or are the de facto truth, but merely want to point out that breathing trash fumes 2-3 times a day for a year cannot be good for your health. And anyone who claims t

  • by Anonymous Coward

    we had a tradition where there would be a major accidental fire in a shop right on the verge of audit.

    Loosing some data on a war campaign, eh? Right after Manning and Snowden scandals? Must be an unfortunate accident, I am sure.

  • I found this essay some time ago on a forum, it was interesting so I saved it. This is not my writing (I 'cleaned up' some f words) but somewhat related as it deals with computers and the military:

    Weekly Rant
    I Love Computers

    This week I'm discussing the great efficiency attained with the use of computers in the military, since they've been in use. I for starters don't like computers (I do like this site so I can openly bitch to a relatively large audience and remain anonymous, since I'm still in. haha,

  • I don't believe it's a problem.
    Why do you want all that data? So we can fight the last battle better?
    I know it seems 'obvious' that keeping all that data--and then analyzing it repeatedly--seems useful. But it probably isn't.

    First off, it probably isn't 'data' so much as anecdote--a bunch of personal ideas from a limited perspective on what to do next. Do you really think there is 'data' on those click drives or is it more like ones and zeroes mostly attached to personal e-mails and off-color jokes?

"If a computer can't directly address all the RAM you can use, it's just a toy." -- anonymous comp.sys.amiga posting, non-sequitir

Working...