Something has been happening recently, and it’s good news for good governance.
After five years, the Australian Department of Finance, Digital Transformation Agency (DTA) and National Archives have all harmonised on something that has been a requirement of the International Standards since 2011. All evidence of business is a record, and all records must be managed. In any format, and any system.
This has been the ‘rule’ for a long time, but has not been the reality. For the whole history of digital records management, only the “one percent” have been managed formally: that is, document-type items, manually classified, stored in an EDRMS. But now, the main Federal departments in charge of setting direction how we manage information have called time on the traditional EDRMS.
There have been technologies on the market for some time now which connect business systems to traditional EDRMS, or act as an EDRMS that sits behind a business system. But these are now evolving one step further. Let’s look at a few systems which have started making the change to true in-place records management recently:
Microsoft Records Management
Microsoft has now in-housed the capability previously offered by other vendor add-ons, to classify and manage the lifecycle of records. This is a really important step in recognising that there are more regulatory obligations on information than just security. The approach still can’t work per the Australian and International Standards, as it can only apply one label. Per 16175-3, systems must be able to manage a many-to-one relationship where multiple disposition classes may be linked to a single electronic record, or where applicable an aggregation of electronic records. We know that records are about more than one thing, and those things change as the record changes. We also know that we need to dispose based on the longest-retention Class, so we can’t just apply one Class and hope for the best. Even if we can work out the longest retention Class consistently, we still need to apply it manually or using supervised Machine Learning using this new tool, which cannot scale. And, of course, we are still only managing O365, not ‘all evidence of business’.
RecordPoint
Originally a SharePoint system, RecordPoint can now manage SharePoint, Office 365, shared drive, DropBox and Box, in-place. This is better coverage than Microsoft, so we are getting closer to managing all evidence of business with a solution like this one, but it is still not ‘all in-place data, records and content’. The limitation that affects adoption is similar to Microsoft’s: the classification relies on a rules engine, and/or on supervised Machine Learning. The goal of Finance and DTA is to make record keeping invisible to users, which we can achieve with solutions like this, which apply the rules in the background. The issue is that the impact just transfers to the records team, who need to create (and maintain) rules for each save location (e.g. each folder in each library), and can then only apply a single retention policy to anything that goes into that folder or library (which doesn’t comply with the ‘aggregation’ sentencing requirements of the Standards). Because of the high impost of this work, solutions that use rules engines (aka rules trees or file plans) often do end up impacting the user base after all, because governance teams have to take away the ability for users to create their own libraries and folders for example, because the process has to be centralised to get the governance correct. RecordPoint is developing an AI approach now for automatic classification, used at this stage just to support the manual classification. However at this stage that system does rely on supervised Machine Learning, which again has a high impact on governance teams to both set up and sustain.
M-Files
This platform is now entering the Asia Pacific market, and is a manage-in-place solution that also assists with security and discovery, as well as document management. It uses AI, classifies based on content, and does not migrate records, so is definitely aligned with the Federal Government direction. It works across many more systems than Microsoft or RecordPoint, but it does require a connector for each one, which can be costly. It’s currently limited in that is does not automate sentencing and manage the disposition lifecycle, so is not really a records-management system, but it does allow organisations to view related information across silos through a single pane of glass. It uses metadata to label documents, which does have the same limitations as other ‘tagging’ solutions like the Microsoft one, in that there are only so many properties that can be added. This makes it hard to identify not only the ‘most obvious’ Class, but also the ‘longest retention’ Class, which is not always the intuitive one. It also makes it hard to apply additional rules, for example around freezes and holds, which have to supersede the standard retention. The AI in M-Files reads data and provides suggestions, which again all have to be manually validated and applied by the governance team, which is very hard to do at scale.
So what comes next? We have forged a path in true in-place records management, which works across all systems, with no user impacts.
- It is possible to manage all evidence of business, but not by using connectors or agents: there are just too many systems, and they change too frequently, to buy and maintain separate add-ons for each one.
- It’s also possible to use AI and ML, but not with a supervised model, as it just puts too much burden on records teams.
- It’s possible to sentence all content and manage its lifecycle, but not using rules engines or file plans that can’t handle nuance and complexity.
True in-place management is the logical next step for all records and information management systems, and there is a clear and proven path forward, which is now endorsed by the regulators. It will not be long before more technologies are working across more types of data, with less burden for everyone. And this is the best thing that can happen to the management of information risk and value not just here in Australia, but right across the world.