Scheme member data quality

Assessing member data quality

Once you have identified the data items that are essential to efficiently run your scheme, you need to measure the quality of this data. This will help you identify any actions that are needed to improve its quality.

Frequency of data quality assessment

Our general code requires you to assess the need for a data review exercise at least annually. How often you measure the quality of your member data in practice will depend on the size, nature, scale, and complexity of your scheme.

Member data degrades and is subject to change. Therefore, most schemes will need to carry out a review at least once a year to make sure their data stays high quality. Larger schemes and those with a complex benefit structure may need to review the quality of member data more frequently.

Unless your scheme has triggered a wind up, you will need to report scores for your common and scheme-specific data to us in every scheme return. In the sections called calculate common data and calculate scheme-specific data, you can find out how to do these calculations.

Certain events may trigger the need for additional data reviews, for example:

• where a decision has been made to buy-in or wind up a scheme (transferring to another scheme or buying out)
• where there has been a change of administrator or administration system/platform
• where the scheme is affected by bulk activities, such as a merger, an acquisition, section disposal, or de-risking exercises
• where the scheme needs to comply with new legislative requirements or carry out a remediation exercise that involves significant data, for example pensions dashboards, guaranteed minimum pension equalisation
• where data improvement work has been completed

Scope of data quality assessment

The scope of a data quality assessment will depend on your scheme’s circumstances. For example, a small scheme that is closed to contributions or future accrual and processes a relatively small number of transactions will have fewer changes to its records over a year. Therefore, the data assessment may be scaled accordingly. A scheme that has a high turnover of members or one that processes a large number of transactions is likely to require a more comprehensive data assessment.

Understanding your scheme’s needs is important when assessing the quality of your member data. Perfect data quality may not always be achievable. Therefore, focus should be given to ensure the data is as fit for purpose as it can be. Once you’ve decided what actions to take to assess and improve your data, you will need to work with your administrator, and any other identified service providers, to ensure these actions are taken in a timely manner. To find out how to improve your data, read the section called data improvement.

There will be instances when you may have exhausted all reasonable measures to improve the quality of specific data items, for example due to historic issues or data loss. In this situation, you need to challenge your administrator on whether all measures are truly exhausted, and if so, to agree, record and implement a consistent and fair policy for these situations.

Data quality assessment dimensions

Data quality dimensions are the characteristics against which you measure quality. Just checking that each field has data is not sufficient. You, or your administrator on your behalf, should assess your member data from different dimensions to ensure that it is of sufficient quality and determine if improvements are needed. You should know what tests your administrator runs when assessing the quality of your member data.

In the government’s news story, meet the data quality dimensions, it recommends looking at six core data quality dimensions as defined by the Data Management Association UK. It is good practice to ensure that your scheme member data has been assessed against these data dimensions.

Completeness

The member data you need should be in your administration records and ready to be used. This doesn’t mean that every data item must be populated for every member, but rather that it is there if it should be. For example, you would need to hold the expected retirement date for active and deferred members, but not pensioner members - for them, you should hold their actual pension commencement date.

Accuracy

Member data should reflect reality, from the moment it is collected and stay accurate over time. For example, by confirming date of birth against identity documents. While some information (for example, date of birth) is static, other data changes over time. You should have processes in place to capture this updated data. This may be provided by the employer or the member, or you may identify issues when using the data (for example, getting gone-away responses to mailings shows inaccurate address information).

Uniqueness

Duplicate data items may indicate issues. For example, unless your scheme operates with more than one record per member, two records with the same National Insurance number may indicate an error. This is particularly relevant when data sets have been combined.

Consistency

Your data items should not conflict within your records or with other data sets. For example, the date a member joined the scheme should be later than their date of birth and before the date they expect to retire.

Timeliness

Your data should be available when expected and needed. This is particularly important for processing retirements and transfers, meeting disclosure requirements, or providing information to members through dashboards. For example, you need members’ up to date pensionable salary information to be available so you can issue annual benefit statements.

Validity

Your data should be configured to the expected format, type and range. For example, all email addresses must have an @ symbol, UK postcodes must appear in the Royal Mail postcode list, and National Insurance numbers must follow HMRC rules on length and permitted letters. There are also specific requirements on how data must be formatted when it is sent to pensions dashboards.

Examples of assessing the quality of your member data

Your administrator will assess the quality of scheme-specific data for a member in a number of ways. An example of how the assessment could be done is as follows:

• Completeness: checking there is a record present in all relevant fields for this member. If any records are missing, the administrator should look into why.
• Uniqueness: checking there is no duplicate member record for this individual. If this member has multiple benefit records or employment records, these should be linked together under that member.
• Timelines: checking when the data was last collected, updated or reviewed for this member. Active and pensioner records will typically be updated monthly, while an administrator may only review and update deferred member data once a year.
• Validity: checking all relevant data items are in the correct range and format. For example, the sum of the transactions made, equals the total amount of contribution received in any given month.
• Consistency: checking the records held in other data sets (for example, data provided by other providers, or data held by your actuary or investment manager) is the same as the records held on your administration system. If different data sets conflict, look into why and decide which one is more reliable and should be used. Update agreed data in all data sets and check that the new data will not be unintentionally overwritten through automated data feed.
• Accuracy: checking that data held against this member reflects reality at a reasonable level. For example, that the value of their pension accurately reflects what they are entitled to under the scheme rules, taking into account salary and contribution information, investment returns and other relevant factors.

Data report

You should obtain a report on the results of the data quality assessment from your administrator. You should review this report and raise any queries with them. You must make sure you are satisfied that the report provides a true assessment of your scheme’s data quality. It must also provide sufficient information to understand the impact it has on your strategic goals and to make relevant decisions.

The data report can be a stand-alone report, an automated report from the system, or included as part of your administrator's report. Your administrator may provide multiple data reports for different purposes. For example, they may provide an automated report which allows you to monitor the data quality as you need, a regular report that relates to day-to-day data cleansing work, or a specific data review report for certain scheme events.

As a minimum, you should receive a data quality report from your administrator once a year, which includes the data score for your common and scheme-specific data and sufficient contextual information to enable you to understand:

• which data items are being tested, for which members
• how they are being tested
• whether there are any systemic issues which need to be addressed, for example, if issues are associated with one particular employer or membership type
• the impact of any identified data issues on the scheme
• any actions that could be taken to rectify the identified data issues

Submit data scores to us

You must send us the scheme’s data scores in each scheme return. You need to submit separate scores for common and scheme-specific data.

You should ensure that the score you submit is a true reflection of the quality of your member data. Your data score is the percentage of members in the scheme that you hold complete and accurate data for.

As you may need different data items for your members depending on the benefit type (defined benefit or defined contribution) or member status (active, deferred or pensioner), you may wish to segregate your members to test the different groups against the relevant data items. This will help prevent false negatives from being included in your scores. You will then need to combine these scores to provide a single overall score for the scheme.

Example calculation – common data

A data review of a scheme with 100 members shows that 40 addresses are missing but the rest of the data is present and accurate. The common data score is 60% as 60 members have full and accurate common data.

Example calculation – scheme-specific data

Scenario 1

If a data review of a scheme with 100 members shows that 40 members have at least one scheme-specific data item missing, the scheme-specific data score is also 60%.

Scenario 2

A scheme has 100 members, of which 60 members are active members and 40 members are pensioners. Different data items are included in the scheme’s specific data for active and pensioner members. To ensure that the data score provides an accurate reflection of data quality for the whole membership, the administrator calculates the scores separately for active and pensioner members, and then combines the individual scores. The data score for active members is 50% and the data score for pensioner members is 75%. 50%*[60/(40+60)] + 75%*[40/(40+60)] = 60%. The overall scheme score would therefore be 60%.

Scenario 3

A hybrid scheme has 100 members, of which 60 members have defined benefits and 40 members have money purchase benefits. The data score for members with defined benefits is 50% and the data score for members with money purchase benefits is 75%. 50%*[60/(40+60)] + 75%*[40/(40+60)] = 60%. The overall scheme score would therefore be 60%.

Scenario 4

A hybrid scheme has 100 members, of which 60 members have defined benefits and 80 members have money purchase benefits. The data score for members with defined benefits is 50% and the data score for members with money purchase benefits is 75%. 50%*[60/(80+60)] + 75%*[80/(80+60)] = 64%. The overall scheme score would therefore be 64%.