Document Freedom Day - March 31st, 2010
Paw Prints: Writings of the maddog
Today (March 31st) is Document Freedom Day, and I encourage everyone to talk about Document Freedom with all of your friends, and even your enemies.
In 1973 I worked for Aetna Life and Casualty, at that time the “largest commercial user of IBM equipment in the Free World”. We did not know what the government was using, and we did not know what the Russians were using, but other than that, Aetna was the largest.
Aetna had an on-site tape library of 500,000 12 inch diameter magnetic tapes, with another 100,000 stored in a salt mine someplace in Idaho for “long term storage”. The tapes were labeled starting at “1” and went sequentially up from there. When the retention period on the data expired we re-used the tapes (naturally), and some tapes wore out, but by the time I joined the company we were well into the upper six digits of tapes, and tape numbers like “965000” were not uncommon.
To keep track of these tapes we used a small mainframe computer, with an on-line program that would allow you to type in the tape number, or a dataset name and the date, and the system would tell you everything about the tape: tape number, dataset name, who created it, what date it was created, retention period, tape length, tape density in bits per inch, number of tracks on the tape and where it was currently located. The tape numbers were written on the outside of the magnetic tape and also written electronically on the tape header.
In 1973 Aetna was using nine-track magnetic tape at 9600 bits per inch. The “tracks” went longitudinally down the tape, and each 1/9600th of an inch there was another bit. Previous to this Aetna had used different tape densities (and you could ask for lower densities to be read or written) and even fewer tracks (seven track tapes, for instance). The seven track tapes would require a special tape drive, but they had not been used for a very long time.
One day I was finding out where a series of magnetic tapes were located for a project that I was working on and I typed the tape numbers into the mainframe. Eventually I was finished, and curious, I typed in the tape number “000001”. The machine came back:
TAPE NOT LOCATED
I typed in “000002”.
TAPE NOT LOCATED
Being persistent (some people unkindly say I am “stubborn”), I typed in “000003”:
TAPE NUMBER 000003 IS LOCATED AT IDAHO SALT MINE, 500 FT, 7-TRACK, 128 BITS PER INCH, RETENTION PERIOD 999999, DATASET NAME: INCORPORATIONPAPERS
Astonished at this, I went to my boss and said “Tom, how in the world are we ever going to read this magnetic tape in the future?” Tom looked at me wisely and said, “No problem. If we ever have to read that tape we have a seven-track tape drive wrapped in bubble-wrap at the salt mine too.”
I let that sink in, and then asked, “Yes, but where is the computer system we will hook that tape drive to, the operating system software that we will run on that computer system, and the operators that will know how to run that operating system? Are they out there in the salt mine too, wrapped in bubble wrap?”
It was questions like these that did not endear me to my management, but Tom just winked at me and told me not to worry about it, so I imagine that today tape “000003” is still in that salt mine, still waiting to be read by the tape drive and the non-existent computer system and the operators that are either retired or dead.
All of this, of course, brings us to Document Freedom Day and the necessity of having truly open document formats and multimedia codecs. We have the storage capacity to keep on-line all of the necessary documents for the re-creation of our society, but only if we are able to decipher them and read what they say.
While we still may be able to pick out the bits of Word Star and recreate the text of the document, some day the complexities of modern-day word processing programs will cause us to lose some image, some text or even a single word that might change entirely the meaning of some document. Will we be able to decipher the macros used by accountants in writing their Excel spreadsheets?
Today is a good day to talk to your boss or your dean about the necessity of having open standards for documents. For more information about why this is necessary, you can read up on the issues. You are also welcome to see my little video on printing and the freedom of information and please remember that every day is Document Freedom Day!
Tapes and longevityAfter Dave's comment about tape life I looked for "magnetic tape degradation" and found this 1995 report:
that reminded me more about tapes than I ever wanted to remember.
BUT, the report said that they considered tapes to have a life of 20-30 years, however, they were finding the tape life was actually a lot shorter than that.
They also recommended unwinding and rewinding the tape every three years.
Tape degredation>To my knowledge and understanding outside of R&D and other special projects there is no format where the >data will not be lost due to physical degradation of the media.
Media can degrade over time, and in the early days of magnetic tape there were issues of "bleed-through" where the magnetics of one layer could (over time) affect the other layers. This required unrolling the tape every once in a while and re-rolling it to put a different part of the tape on top.
I heard that more modern tape cartridges have gotten around the issue of "bleed-through", and of course the tape cartridge also cuts down on contamination and wear and tear on the tape.
The real issue is still in "planning", and understanding that no media lasts "forever", but in this industry 20 years might mean that the media is still intact but there is no mechanism to read it, and if you can read it, you can not understand what it means.
Degradation of storage mediaThis is an interesting article. I am all for open standards and know the pain of retrieving data from important documents that have become corrupt.
The knowledge of legacy systems discussed here is an important point. However, more important to this issue of having the requirement for the tape reader and knowledge of the system etc... is if/when someone goes to recover the data, will the tape still contain any?
To my knowledge and understanding outside of R&D and other special projects there is no format where the data will not be lost due to physical degradation of the media.
Kernel king admits his tone has alienated volunteers, but says the demands of the process require directness.
New flaw in an old encryption scheme leaves the experts scrambling to disable SSL 3
Lennart Poettering wants to change the way Linux developers talk to each other.
Enterprise giant frees itself from ink and home PCs (and visa versa).
Mozilla’s product think tank sinks silently into history.
TODO group will focus on open source tools in large-scale environments.
New tool will look like GParted but support a wider range of storage technologies.
New public key pinning feature will help prevent man-in-the-middle attacks.
Carnegie Mellon researchers say 3 million pages could fall down the phishing hole in the next year.
The US government rolls new best-practice rules for protecting SSH.