Delivering the Expected - Solteq Developer Blog

GDPR tools in Microsoft Dynamics NAV

Servers

GDPR is really hot topic recently. I might dare to say that this abbreviation does not even require decryption - we all know what those four seriously-sounding letters stands for. All companies are affacted somehow. As long as you process personal data of EU citizens, it does not matter how you do it - whether your software was written in Java or C#, whether it is desktop mobile or web app, hosted in Azure or on your own PC, is it a newest or very old version of software - you must obey the regulations. And although the regulations are the same regardless of platform, software providers may help their customers by providing the specifc tools - dedicated for their products/solutions - that can facilite GDPR compliance process.

This blog post focuses on the ERP system Microsoft Dynamics NAV and the tools that Microsoft recently published to support some GDPR areas.

Better late than never

Although the GDPR was made in April 2016 and the two-years transition period was given, Microsoft released tools and whitepapers materials in April 2018. Well, better late than never.

The tool released by Microsoft is available only in the latest version of the NAV (latest cumulative updates for NAV 2015, 2016, 2017, 2018 and Business Central of course) and what makes the things a bit more complicated, is a fact that there is hardly any documentation provided by Microsoft explaining how to use the tools. Mentioned whitepapers are only describing the process on the very general level, or rather the main message is that it says everything is covered, but the document does not give any specific instructions how to use the tools. So even when you read whitepapers and all (two or three) related articles on docs.microsoft.com, when you open NAV (let’s say newest version), chances are you have no clue where to start.

Some NAV professionals blogged about the toolkit, but seemingly they just briefly explain DataClassification property and give some screenshots of the tools, but still - we were lacking something more comprehensive to help us understand the ‘GDPR Toolkit’ in NAV (disclaimer: ‘GDPR Toolkit’ is not an official name). We were looking for materials that could explain if and how the DataClassification is linked to data privacy. How this works (how it was build by Microsoft) and is it really good - in case not, what are the possibilities to improve it? So, as you can see, reading just some headlines and these laconic materials will result in more questions than answers. That is why we decided to go through this topic throughthogly, on the technical level.

Two layers of data classification

We can distinguish two layers of data classification, as shown in the picture below.

Two layers of data classification

Both layers apply not only to tables (columns in tables) provided in standard product released by Microsoft, but all other fields added by ISVs (add-ons) or partners (customizations) should be reviewed and classified accordingly.

While Data Sensitivity applies to personal data only (which is subject of GDPR), Data Classification provides more generic level that goes even beyond the scope of GDPR. Fields can be classified in Data Classification processes as different types (as is shown in the next section below).

It is possible to skip the Data Classification process and perform only Data Sensitivity Classification. However, if Data Classification is not done, all fields by default are labelled as ‘personal’ (CustomerContent) and thus there are more fields to browse through when assigning relevant sensitivity level (GDPR scope). Therefore, it is recommended to do Data Classification first and then detailed classification (Data Sensitivity classification).

Data Classification

Data classification is done by assigning the (new) property for the table or field. It is described in the MSDN blog. (The table below and text in this section is copied from that blog)

DataClassification property

There are following classification option values available:

Data classification Description Example
CustomerContent Content directly provided/created by admins and users. Customer generated BLOB or structured storage data. Customer-owned/provided secrets (passwords, certificates, encryption keys, storage keys)
EndUserIdentifiableInformation (EUII) Data that identifies or could be used to identify the user of a Microsoft service. EUII does not contain Customer content. User name or display name (DOMAIN\UserName). User principle name (name@company.com). User-specific IP address.
AccountData Customer billing information and payment instrument information, including administrator contact information, such as tenant administrator’s name, address, or phone number. Tenant administrator contact information (for example, tenant administrator’s name, address, e-mail address, phone number). Customer’s provisioning information
EndUsePseudonymousIdentifiers (EUPI) An identifier created by Microsoft tied to the user of a Microsoft service. When EUPI is combined with other information, such as a mapping table, it identifies the end user. EUPI does not contain information uploaded or created by the customer (Customer content or EUII). User GUIDs, PUIDs, or SIDs. Session IDs
OrganizationIdentifiableInformation (OII) Data that can be used to identify a tenant, generally config or usage data. This data is not linkable to a user and does not contain Customer content. Tenant ID (non-GUID). Domain name in e-mail address (xxx@contoso.com) or other tenant-specific domain information.
SystemMetadata Data generated while running the service or program that is not linkable to a user or tenant. Database table names, database column names, entity names

Following rules apply:

To view the data classification on all fields, you can do one of the following:

Data classification on the table level can be seen in the Table Metadata (ID 2000000136).

Our observations

Apparently the DataClassification property on the table level has no special meaning. Only the data classification on the field level is used.

Data Sensitivity

As already mentioned, data classification does not provide enough categorization regarding privacy and data sensitivity. It does not cover all GDPR requirements as such. That is why there is another layer of classification available.

Sensitivity Description
Sensitive Information about a data subject’s racial or ethnic origin, political opinions, religious beliefs, involvement with trade unions, physical or mental health, sexuality, or details about criminal offenses.
Personal Information that can be used to identify a data subject, either directly or in combination with other data or information.
Confidential Business data that you use for accounting or other business purposes, and do not want to expose to other entities. For example, this might include ledger entries.
Normal General data that does not belong to any other categories.

Data sensitivity classification applies only to fields, and what is worth to note – only to fields that are classified (by DataClassification field property) as one of the following type: CustomerContent, EndUserIdentifiableInformation or EndUsePseudonymousIdentifiers. In other words, a field classified as AccountData, SystemMetadata, OrganizationIdentifiableInformation is not subject of data sensitivity classification.

Data Privacy Toolkit provided by Microsoft

The toolkit is available from the NAV Client (both: Windows and Web), but it is accessible only for IT administrators – only users within profile which has Role Center Page ID = 9018. Considering the standard profile set provided by Microsoft, it is IT MANAGER profile.

In the Data Privacy navigation pane there are following views (pages) available:

Data Classification Worksheet

Data Classification Worksheet

This page lists all fields that are available for classifications. Classification is done and maintained separately in each company.

When the Data Classifications Worksheet is accessed for the first time in the upgraded database, it shows no fields to be classified.

How does the Data Classifications Worksheet work from the technical point of view?

Data Classifications Worksheet is built on the (new) system table Data Sensitivity (ID 2000000159). When the Set Up Data Classifications is executed or when the Sync Fields function is called, system iterates through Fields virtual table and copies the fields to the Data Sensitivity table. However, the field is added to Data Sensitivity table only if DataClassification property value is one of the following: EndUserPseudonymousIdentifiers, EndUserIdentifiableInformation, CustomerContent. This action is executed in the context of the current company, as Company Name is one of the fields in the Data Sensitivity table.

Data Subjects

Data subject is one of the three main roles in GDPR. It is defined as ‘an identified or identifiable natural person’ and for the purposes of the scope of the GDPR that data subject is covered, regardless of their nationality or place of residence within the EU, in relation to the processing of their personal data.

Data Subjects

Data Privacy Utility

Data Privacy Utility is the assisted setup that guides the administrator through the process of exporting the data subject’s data. This tool always creates the configuration package, and if the export option is selected, Excel file will be generated.

Data Privacy Guide

You can select only one entity to export. For example: you can export one customer, or one employee. Consider following scenario. John Smith might be at the same time Customer and Employee in your database, but at a one time you can export either only John Smith customer’s data (and related tables, e.g. sales invoices) or John Smith employee’s data (and related tables). It is not possible to do these two things in a one round. Another example would be, that you cannot do a batch export for multiple records – you cannot export data for instance for all customers’ registered in the current year. It is always limited to one data subject (one export is John Smith as a customer, another is John Smith as an employee and yet another is Anna Smith as a customer).

Before exporting the data, it is possible to generate the preview.

If the export option was selected, then the Excel file can be downloaded from the (Administrator’s) Role Center – in the Report Inbox.

Created package in RTC

Our observations about this tool:

Chances are that when you try to export the data and import it back using configuration package export/import features, you will face some issues. Two main issues are following:

How to add new data subject?

In the custom-tailored solution, it may be necessary to register more Data Subjects than only the few provided by Microsoft. For instance, in the retail solution usually there is a table called Member (or similar) storing loyalty member information. In order to add a new entity to the Data Subjects list, development is required (this should be programmed in the source code).

Deleting the subject’s data, more detailed examples and how to install the tools into existing solutions

There is no a dedicated function (button) to delete requested data subject’s data. The only way how to do is, is to export configuration package to Excel, then clear the data and import the package back.

In my humble opinion, there is a room for improvement in this area. So, if someone from Microsoft is reading this, please consdier my comments :)

Unfortunately, for this blog post it would be too much to present here all our findings and examples we tested and documented. We already went through the pilot upgrade project (to upgrade custom db to newest version to have data classification in place). If you are interested to learn more about standard NAV data classification toolkit or need some help in this aread, Solteq is happy to assist you. Please use our contact form to drop as an request.

About Solteq

Solteq is a leading provider of Microsoft Dynamics NAV and LSRetail solutions and both NAV & LSRetail are strategically important solutions in Solteq’s portfolio.

Solteq has delivered several demanding Enterprise Resource Planning (ERP) projects and upgrades during the past 30 years specializing in Microsoft Dynamics -portfolio.

Solteq’s strategy is to be world’s smallest international digital commerce provider. We want to be small enough to care and nurture our customers but large enough to carry through international implementations.




Join us?

See what we have to offer - Careers