Project extracting complex HCL Notes documents with a large number of attachments, links, pictures and attachments in nested tables into XML and pdfs for import into SharePoint.
Challenge
To migrate HCL Notes databases with highly complex documents to a pdf format that keeps the structure and fidelity of the original documents. Each doc has a large number of images, links and attachments in nested and tabbed tables in a Rich Text field that needed to be migrated to a new platform.
This was the challenge set to us by Cheshire Police.
Rich Text content. Great for users. Tricky for migration!
A bunch of HCL Notes databases had allowed the police to gather information into a single, free format, Rich Text field on documents. They would copy and paste a template document containing a nested tabbed table, and then add inline images, attachments, links and formatted text into the appropriate sections. This gave the users flexibility when adding data, but how do you now export it?
Notes databases slow
Some documents had become unstable being susceptible to becoming corrupt and the HCL Notes client would often freeze or be very slow when accessing a document.
Duplicate documents
Duplicate documents had been created by users to mitigate this issue resulting in two or more documents relating to the same subject matter.
Sensitive Information
Due to the sensitivity of the information, the project needed to export the data while retaining the original format. Some of the data would be archiving and only viewed from the file system, other more recent data would need to be imported into Microsoft SharePoint.
Migration options
The options open to the migration team were
- Keep Notes databases on servers that were due for replacement
- Copy and paste / re-key information into a modern system
- Ask 360 Systems to help automate the HCL Notes migration project - which is what they did!
HCL Notes Migration Project
We undertook the project taking the following steps
Analyse content and layout
As the users were given free rein when entering data into these applications, 360 Systems began by completing document analysis to list each of the elements used, identify the typical document and any exceptions. The analysis identified nested tabbed table, inline images and attachments, links and formatted text. There were exceptions where users had not used the tabbed table or had added additional tabs from the normal set.
HCL Notes to XML + pdf toolset
360 Systems' HCL Notes to pdf toolset was configured to take the content analysis and extract each element of every document. The trick was to keep the original format entered by the user so that it could be viewed as it had been in IBM Lotus Notes documents. Attachments and inline images were shown next to the content describing them, formatted text (colour / size) and links would be retaining.
The resulting pdfs were ready to be stored on the file system, sent to archive or imported into SharePoint.
XML metadata enables intelligent import
To provide a mechanism for the extracted Notes to pdf documents to be imported into Microsoft SharePoint, each pdf was searchable and had related file attachments and XML metadata in formats that were understood by SharePoint.
Standardised migration process for remaining HCL databases
For important historic information, the benefit of running a quick, accurate migration project far outweighed the potential risk of leaving things as they were or doing a manual migration. The data was extracted from each of the HCL Notes databases over the course of a couple of weeks.
HLC Notes Migration Services
If you have a tricky migration project give me a call or send in a question and the team at 360 Systems will help you out.
Richard Evans
Projects Director
