Project to extracting complex Lotus Notes documents with a large number of attachments, links, pictures and attachments in nested tables into XML and pdfs for import into SharePoint.

Lotus Notes to SharePoint Migration using XML and searchable pdfs

Challenge

To migrate Lotus Notes databases with highly complex documents to a pdf format that keeps the structure and fidelity of the original documents. Each doc has a large number of images, links and attachments in nested and tabbed tables in a Rich Text field that needed to be migrated to a new platform. 

This was the challenge set to us by Cheshire Police.

Rich Text content. Great for users. Tricky for migration!

A bunch of Lotus Notes databases had allowed the police to gather information into a single, free format, Rich Text field on documents. They would copy and paste a template document containing a nested tabbed table, and then add inline images, attachments, links and formatted text into the appropriate sections. This gave the users flexibility when adding data, but how do you now export it?

Notes databases slow

Some documents had become unstable being susceptible to becoming corrupt and the Lotus Notes client would often freeze or be very slow when accessing a document. 

Duplicate documents

Duplicate documents had been created by users to mitigate this issue resulting in two or more documents relating to the same subject matter. 

Sensitive Information

Due to the sensitivity of the information, the project needed to export the data while retaining the original format. Some of the data would be archiving and only viewed from the file system, other more recent data would need to be imported into Microsoft SharePoint.  

Migration options

The options open to the migration team were

  • Keep Notes databases on servers that were due for replacement
  • Copy and paste / re-key information into a modern system
  • Ask 360 Systems to help automate the Lotus Notes migration project - which is what they did!

Lotus Notes Migration Project

We undertook the project taking the following steps

Analyse content and layout

As the users were given free rein when entering data into these applications, 360 Systems began by completing document analysis to list each of the elements used, identify the typical document and any exceptions. The analysis identified nested tabbed table, inline images and attachments, links and formatted text. There were exceptions where users had not used the tabbed table or had added additional tabs from the normal set.

Lotus Notes to XML + pdf toolset

360 Systems' Lotus Notes to pdf toolset was configured to take the content analysis and extract each element of every document. The trick was to keep the original format entered by the user so that it could be viewed as it had been in IBM Lotus Notes documents. Attachments and inline images were shown next to the content describing them, formatted text (colour / size) and links would be retaining.

The resulting pdfs were ready to be stored on the file system, sent to archive or imported into SharePoint. 

XML metadata enables intelligent import

To provide a mechanism for the extracted Notes to pdf documents to be imported into Microsoft SharePoint, each pdf was searchable and had related file attachments and XML metadata in formats that were understood by SharePoint.

Standardised migration process for remaining Lotus databases 

For important historic information, the benefit of running a quick, accurate migration project far outweighed the potential risk of leaving things as they were or doing a manual migration. The data was extracted from each of the Lotus Notes databases over the course of a couple of weeks.

Lotus Notes Migration Services

If you have a tricky migration project give me a call or send in a question and the team at 360 Systems will help you out.

Richard Evans

Projects Director