How will we write history when the data disappears?

Like a lot of people at present, I imagine, I have a little red dot hovering over Settings on my iPad and iPhone, inviting me to download an upgrade.  Despite the lure of little red dots, I’m ignoring it, because the upgrade would mean replacing my Google Maps app, which is fine, with Apple’s own Maps – which is apparently hilariously bad.

This is fine in the short term, just irritating – but what about the long term?  What about all the stuff – content, programs, operating systems, a perfectly good Google Maps app – that disappears?

‘Our online history is disappearing at an astonishing rate, creating a black hole for future historians,’ says Tom Chatfield on the BBC Futures program.  He gives examples from last year’s revolution in Egypt – photos, tweets, blog posts – that have just disappeared in the last year.  Chatfield points out the irony that it’s now possible to read online virtually any book or pamphlet printed during the 17^th century English Civil War – but not similar communications from last year’s Arab Spring.

Chatfield’s concern is with content, but there’s a similar problem with obsolete software.  Archivists deal with this nightmare all the time, for many government and business documents only exist in electronic form these days.  When a new operating system arrives, old documents can become unreadable.

At the Queensland State Archives a few years back, I was shown ranks and ranks of old computers running earlier operating systems so that, if necessary, a researcher can put their 7½ inch floppy disk into the appropriate slot to read off data from the 1980s.  But that strategy can’t last forever.

At Bletchley Park, where the very first computers were built to crack the Enigma Code during World War II, one of the ‘bombes’ has been rebuilt.  It’s in working order, and could crunch numbers again if required – but it is built using 1940s valves cannibalised from old radios.  Valves have a limited life, nobody is making them any longer, and one day they will all be gone.

Perhaps though, we can sometimes have too much information.  Researchers can follow the English Civil War in great detail if they have access to Early English Books Online.  EEBO is a wonderful resource.  Every book or pamphlet printed in England before 1700 has been digitised and is available in searchable form – although only by expensive subscription.

The braggadocia souldier: and the civill citizen (1647), in English Books Online, Thomason / 246:669.f.11[81], from British Library

A researcher can read everything anyone published during the Civil War – about religion or battles or witches or whatever.  The irony is that we can read far more than any of the original participants did.  No 17^thcentury library held so many books; no assiduous intellectual could find as many references to ‘women preachers’ or ‘Prince Rupert’ as we can, with the click of a mouse.  Sometimes, too much information can distort our perspective: a foot soldier in that distant Civil War had little knowledge of what was going on over the next hill, with no tweets, blogs, or emails to keep him up to date with distant events.  A foot soldier in Syria today has access to YouTube and Twitter – but they are ephemeral sources.

Meanwhile libraries are trying to keep up.  The Library of Congress plans to archive every tweet – though pity help the future historian who has to read them.  In Australia, the National Library of Australia has been archiving websites for a number of years through Pandora – but it is at best a partial process.  Links break, domains change, servers go off line.

Last week Pandora contacted me to ask permission to archive Historians are Past Caring.  I was quite chuffed, but I took a few days to get around to answering their email.  When I finally sat down to do so, I found that by mistake, I’d deleted the letter.  Says it all, really.

This time last year:
History at the Coalface, 28 September 2011