Web 2.0, Enterprise 2.0 etc are hot terms and some people have already started talking of Web 3.0! However, in the ECM space, I think there are quite a few challenges of 1.0 (or even pre 1.0) that need to be addressed. For example, Content Archival is an important stage in content lifecycle. Concerns like compliance are driving the need to implement ECM systems to build long term (greater than 20-30 years) content retention systems. However, consider the challenges that this poses that vendors have not yet cracked:
- Can the content created by your favorite application be modified 20 years later? The macros written in some of my Lotus 123 spreadsheets 8 years back are totally useless today because my company no longer used Lotus. Constant upgrades to software versions limit compatibility with previous versions.
- The hardware and/or operating system 20 years later will be completely different from today and even if you could find a copy of Lotus 123, will you be able to install it on that hardware?
- Most media have a limited shelf life – CDs go bad in a couple of years. Some media types are already obsolete – I can’t find a way to open my documents stored in floppy drives any more.
I don’t think the above issues related to software and hardware obsolescence are going to be resolved soon. This is where standards could play an important role. So instead of using proprietary formats, you should push your vendors to follow standards for storage, metadata, content access, security and so on. This will ensure that it’ll be ralatively easier to read your content 20 years later (point 1 above).
Also, it helps if you keep cleansing and migrating often – Migrating from version 1.0 -> version 2.0 -> version 3.0 -> version 4.0 is probably easier than migrating from version 1.0 -> version 4.0
I totally agree with what your saying Apoorv. On the technology side this is solved through open and, more importantly, well-understood standards such as XML. This will increase the popularity of the Open Document Format as well the pressure on Microsoft to relax the restrictions on Office XML (I believe its being submitted to ECMA). CMSs all play a part in the storage and archival management side of the problem.
My prediction: In 5 years every document and email in a Fortune 500 will be archived in a CMS, DMS, and/or RMS. The only way true “archival” is through open standards. Ask any librarian.
What will push this change through ? The recent change in US corporate law around Federal Rules for Civil Procedure.
Why ? Its cheaper for Fortune 500 companies to archive the content then it is to pay the new fines introduced with recent changes in US law for not being able to present potential evidence during any of on average 40-50 litigations a Fortune 500 company faces, every year.
Archival is often neglected, because people care only for the present and not the future. Few customers will understand how it contributes to performance and productivity, especially in the unrealistic schedules that CMS projects often attract. Also, much of the information becomes obsolete, so there is no incentive to keep it except for legal requirements.
In fact, in a way, you can view system migrations/upgrades as a boon for the organisation – the only time when there is a formal review of all the data stored in CMS: what to keep, what to reorganize, what to purge, etc.
I won’t call it a challenge of ECM 1.0 – it’s more like a human failing: we loathe to get rid of things we had, and yet we lack the discipline (and librarian training) to organize it properly from the start. Knowledge management is like financial planning and networking, one of life’s essential skills that should be taught early in life, but picked up only along the School of Hard Knocks …
Forget Web1.0, archival is a long standing problem. As a first hand experience while working for a media company – they wanted to bring their archives for last 40 years into the archives search. When asked to show the data, they managed to show the last 8 years archives in PDF form – which where on a hard disk, and older than 15 years archives as micro-films.
They just didnot have drives to load the tapes in which their 15-8 year old archives were kept. Did digitization push them a step backwards ?
Will one be able to read PDF documents 20 years from now ?
Will fonts and styles exist as we use them now ?
Since that experience, I dont think that we still have a replacement for optical microfilms which have stood the test of time to reproduce content as is – with formatting, layout, images etc.
Even the Nasa’s moon video hunt is desperate as the engieers who know how to read the format may not be around for long.
That media company – for records – decided to retain everything as microfilms + have everything on the hard disk ( and not tapes) – and have a XML version of the content ( without formatting) on the disk. Whether that is the best we can do – I dont know.
I tend to believe that HTML will have a longer life than any other format of today and that the only way to store documents is to keep them on primary media – which is hard disk today – and make sure they remain so and are migrated to new storage and new file system every year.
When we look even further and look at civilization records – 2000 and 5000 year olds – only three formats have survived
a) Those cast in stone
b)those immortalized by folk stories, songs and dances. Will the films and hard disks survive that long?
c) Religion
Its a big issue, older than Web1.0 and I am sure there are millions of research dollars being spent on it.