Quality of human resources staff ethnicity data in health in Aotearoa New Zealand: an audit of “not stated” staff ethnicity and a data linkage assessment

In Aotearoa New Zealand, health disparities by ethnicity are well documented.¹ The recent New Zealand Health Status Report confirmed the persistent life expectancy gap between Māori (the Indigenous population in New Zealand) and European/Other ethnicity of 7–8 years.² Accurate ethnicity data are essential to quantify and monitor health inequities under Te Tiriti of Waitangi, the foundational document of New Zealand.³ Ethnicity data in health have been widely used to measure and monitor Māori health needs and health outcomes, to ensure funding is adequately allocated for improving Māori health, to prioritise and deliver targeted health interventions and to improve Māori workforce representation.

Expectations for ensuring high-quality ethnicity data, which need to be timely, valid, reliable and useable, are clearly documented in New Zealand. Ethnicity is defined by Stats NZ Tatauranga Aotearoa (Stats NZ) as “the ethnic group or groups that people identify with or feel they belong to. Ethnicity is a measure of cultural affiliation, as opposed to race, ancestry, nationality or citizenship. Ethnicity is self-identified, and people can belong to more than one ethnic group.”⁴ Recently, the Te Kāhui Raraunga Māori Data Governance Model guidance has been published,⁵ which has a strong focus on high-quality ethnicity data and analysis. The required Standards for collection, recording and output of ethnicity data have been set out in the Ethnicity Data Protocols for the Health and Disability Sector.⁴ The Standards draw from the all-of-Government guidance from Stats NZ. Specifically, the Stats NZ ethnicity question that is used in the census⁶ is used to collect ethnicity data.

Health Ethnicity Data Standards apply to staff (human resources [HR]) demographic data as well as to patients. HR demographic data are used for the reporting of workforce diversity and for monitoring workforce ethnic diversity targets. Health New Zealand – Te Whatu Ora reported that Māori currently contribute 8.5% to the national health workforce, which is lower than their representation in the total New Zealand population (17.3%).⁷ Achievement of diversity in the health workforce, often described as ethnic diversity proportionally representative of the population served, is a goal of many countries including New Zealand.^8,9 Previous New Zealand research has highlighted the crucial role Māori health workers play in establishing meaningful relationships and providing culturally appropriate care to Māori patients and their whānau.^10,11 Ethnically diverse health workforces can also promote patient and doctor satisfaction, access to health services and continuity of care. Māori continue to be under-represented in the health workforces.¹² Growing and strengthening Māori health workforces remains a key Māori health priority that supports whānau aspirations and wellbeing.¹³

In other parts of the health sector, ethnicity data quality issues have been identified and quantified in various datasets.¹⁴ Through ethnicity data quality improvement projects, a data quality metric in primary care data was developed using the category of “not stated” ethnicity, with a threshold of >2% considered to indicate potential poor data quality.¹⁵ As part of a set of Māori-focussed workforce development activities in two health districts in New Zealand, the quality of staff (HR) ethnicity data was assessed, including examining the category of “not stated” staff ethnicity. The two data quality assessment projects reported in this paper were initiated alongside a quality improvement project mapping the HR staff ethnicity data processes and systems. The parallel quality improvement project identified a range of issues with HR database vendors and HR processes, such as HR staff being unaware of the Standards, HR processes not using the census Standard question and HR software not being designed for compliance with the Standards in data recording or output categories (Appendix Text 1). Therefore, data quality issues were anticipated with the HR ethnicity dataset. Our projects aimed to assess the quality of HR ethnicity data by 1) re-collecting staff ethnicity from staff with “not stated” or missing ethnicity on HR records, and 2) linking HR ethnicity data to National Health Index (NHI) ethnicity data.

Methods

Ethnicity Data Standards classification

Under the Ethnicity Data Standards⁴ and based on the Stats NZ classification system,¹⁶ individuals respond to the ethnicity question with their self-identified ethnicity or ethnicities. The responses are classified and recorded, and systems should be able to record multiple ethnicities. The classification or coding of ethnicity is hierarchical with four levels (Appendix Table 2). Depending on collection systems, responses can be coded into the most detail at Level 4 (containing 180 categories and five residual categories).¹⁶ Information may be coded into Level 2 (21 categories and five residual categories) or Level 3 (36 categories and five residual categories) if Level 4 recording is not possible. The five residual categories include: do not know, refused to answer, response unidentifiable, response outside scope, and not stated.

Output options are guided by the analytical requirements and include prioritised output based on policy prioritisation of Indigenous and minoritised groups (Māori>Pacific>Asian>Other).⁴ Level 1 is used for prioritised output purposes only and contains six high-level categories and one residual category: European, Māori, Pacific peoples, Asian, MELAA (Middle Eastern/Latin American/African), Other ethnicity, and Not elsewhere included.¹⁶ Residual categories are not usually reported separately and may be incorporated into the “Other” category. Total response option is also possible, which reports against all ethnic groups identified.

Study design and data sources

This is a cross-sectional observational project including two components: 1) re-collection of staff ethnicity via survey from staff with “not stated” or missing ethnicity on HR records in the Auckland and Waitematā health districts’ HR databases,¹⁷ and 2) linkage of staff ethnicity data of the two districts’ HR databases to the NHI database as a reference. The NHI¹⁸—a unique identifier assigned to everyone who receives healthcare in New Zealand—database includes identifiable data such as name, sex, ethnicity, date of birth and address. The HR ethnicity data are collected directly from the individuals at the time of application for a position and at the time of hiring, and they have no linkage with the NHI ethnicity data. The reporting of this project followed the STrengthening the Reporting of OBservational studies in Epidemiology (STROBE) statement (Appendix Table 1).¹⁹

Study procedure and data analysis

Re-collection of staff ethnicity via survey of staff with “not stated” or missing ethnicity on HR records

All staff were notified that a project to check staff ethnicity was being undertaken via the weekly staff newsletter and via all manager briefings which were conducted prior to the project commencing. Processes for questions and complaints were established.

HR members of the research team extracted staff ethnicity for all employed staff in Auckland and Waitematā districts’ HR databases on 30 April 2018 and 2 August 2018, respectively, and identified those who had “not stated” or missing ethnicity. The investigators then emailed these staff, inviting them to participate in an online voluntary closed survey hosted on the SurveyMonkey platform.²⁰ Staff who declined to participate were not contacted further. Other participants were sent a further email 2 weeks after the initial invitation that reminded them to complete the survey or to update their own personal details on the HR Kiosk platform. To maximise response rates, trained Māori community health workers using a standard script made up to three phone calls within the following 6 weeks to staff who did not respond to the two emails, and they conducted the survey verbally over the phone. Staff who were busy and asked for a call-back were re-contacted at a suitable time for them.

The Standards compliant ethnicity question⁴ was used to re-collect staff ethnicity (Appendix Figure 1), with options if the question was left un-answered: “I don’t know my ethnicity”, “I do not want to state my ethnicity”, and an open comments box provided at the end. For the analysis, all ethnicities were recorded and coded at Level 4, then prioritised to Level 2. Level 1 was used for summary results (Appendix Table 2).

Where staff confirmed their ethnicity through the survey or the online HR staff details portal, HR databases were updated with this information.

Data linkage of staff ethnicities to NHI ethnicity data

Personal identifiers and HR recorded ethnicity were extracted from the Auckland and Waitematā district health boards’ HR databases in June 2017. These data were supplied to the Ministry of Health – Manatū Hauora via secure electronic transfer (ftp server). The data were then probabilistically matched with the Ministry of Health – Manatū Hauora’s NHI database by a Ministry of Health – Manatū Hauora analyst, and ethnicity data were extracted from the NHI records. The ethnicity data were extracted at Level 4. The linked dataset, after removing the NHI and identifiable information, was returned via the ftp server to the project team for analysis.

The HR and the NHI ethnicity records were compared and presented in three categories, based on the Ethnicity Data Audit Toolkit (EDAT) data quality assessment:²¹ “exact match”, where the ethnicity data between the two datasets matched exactly, regardless of order; “partial match”, where at least one ethnicity matched between the two datasets, regardless of order; and “total mismatch”, where none of the ethnicities recorded in the two datasets matched, regardless of order. The results of data linkage were presented at Levels 4, 2 and 1. When comparing across the two datasets, where some codes in the Level 4 field in one or other dataset were recorded as Level 2 codes, it was considered a mismatch. For comparison at Level 2, all codes were mapped to Level 2. A prioritised output was used for Level 1 comparison.

Agreement analysis was performed, and kappa statistic was calculated. A p-value of <0.05 was considered statistically significant. Data analyses were performed using Microsoft Excel and Stata 16.

Ethical considerations

The regional Māori workforce body (Māori Alliance Leadership Team; MALT) supported the Māori data sovereignty assessment using the Te Mana Raraunga principles and approved the project. Locality approval was granted for the staff survey, which was consented. The data linkage project was approved with a waiver of consent and was conducted with a privacy impact assessment (PIA) and Ministry of Health – Manatū Hauora data governance approval process, which included ethical and legal review. Additionally, the data linkage project was approved by the Waitematā Privacy and Security Governance Group and the Regional Privacy Advisory Group.

Results

Re-collection of staff ethnicity via survey for staff with “not stated” or missing ethnicity on HR records

Of the 17,539 staff, the proportion of records with “not stated” ethnicity was 15.1% (1,493 of 9,877) at Auckland and 6.4% (488 of 7,662) at Waitematā. Among those, 727 Auckland staff and 122 Waitematā staff responded to the survey, with response rates of 48.7% and 25%, respectively (Table 1). Overall, at Level 2, 49% of respondents (n=417) self-reported NZ European followed by Other European (14%; n=116). At Level 1 European was the most common ethnic group (64%; n=540) followed by Asian (15%; n=127). Pacific peoples and Māori each contributed approximately 5% (n=43 and n=38, respectively). Approximately 8% (n=64) remained in residual categories (Table 1).

View Table 1–3, Figure 1.

Data linkage of staff ethnicities to NHI ethnicity data

All 17,539 staff ethnicities in the Auckland and Waitematā HR systems were matched with the NHI ethnicities (Table 2). Of the 9,877 Auckland staff, 276 (2.8%) in the local HR system and 537 (5.4%) in the NHI dataset had more than one ethnicity recorded. Of the 7,662 Waitematā staff, 592 (7.7%) in the local HR system and 442 (5.8%) in the NHI dataset had more than one ethnicity recorded.

Overall, at Level 4, more than half (54%; n=9,504) had exactly matched ethnicities, and 41% had total mismatched ethnicities, with about 5% partially matched. At Level 2 two-thirds (67%; n=11,731) showed an exact match, and at Level 1 86% (n=14,988) showed matched ethnicities.

The linkage of ethnicity data between the HR datasets and the NHI dataset at Level 1 showed an overall agreement of 86% (kappa 0.77, p<0.0001) (Table 3). Auckland staff ethnicity showed a slightly better agreement than Waitematā staff ethnicity (86% vs 84.5%). The NHI data recorded more European and Asian, whereas the HR data recorded slightly more Pacific and Māori (Table 3). The two datasets showed the highest agreement level for Asian (93%), followed by European (86%), Pacific (84%) and Māori (83%) (Appendix Table 3). Specifically for Māori, 730 (67%) were identified in both HR and NHI, while 206 (19%) were identified only in the HR systems, and 149 (14%) were identified only in the NHI system (Figure 1).

Discussion

This project has assessed the quality of ethnicity data in the Auckland and Waitematā health district HR systems by initially calculating the proportion in each district with “not stated” or missing ethnicity data. The proportions of 15.1% and 6.4% “not stated” ethnicity data in the Auckland and Waitematā local HR systems, respectively, are both above the target of less than two percent set in the primary care data quality audit,¹⁵ suggesting that ethnicity data quality in the HR systems could be improved and that this indicator is useful in identifying data quality issues in the HR dataset. The project then sought to re-collect ethnicity data from staff members with “not stated” ethnicity. The survey received 849 responses, including 38 Māori who were not included in previous workforce monitoring activities.

The linkage of HR ethnicity data to NHI ethnicity data showed a good agreement of 86% overall; however, significant discordance remained. For example, 206 Māori were identified only in the HR systems, and 149 Māori were identified only in the NHI system. This finding confirmed the ethnicity data quality concerns and provided support for the quality improvement project to address the system issues contributing to poor quality in the collection, recording and output of HR ethnicity data, including updating the pre-employment forms to meet the data Standards. The parallel quality improvement activity was successful in improving the “not stated” indicator, and this has been maintained over time (Appendix Figure 2).

Previous Māori scholarship and research have examined ethnicity data quality across different sources, mostly focussed on patient ethnicity data in primary and secondary care sources and on the NHI. This scholarship often takes an explicit anti-racist lens to highlight the underlying systemic nature of policy- or process-level structures in order to systematically address these.^22,23 In our project—which was positioned as assessing, documenting and addressing systems-level factors in the HR process (followed by quality improvement activity)—we used the NHI ethnicity data to assess the HR ethnicity data quality. The other sources of ethnicity data could have been used for comparison, such as the New Zealand census (not available at the time, although it is now via the Integrated Data Infrastructure managed by Stats NZ) or primary health organisations (PHOs) data. NHI ethnicity data, having a coverage of 98% of the New Zealand population and recorded at health encounters,²² is assumed up-to-date and most widely used to report health outcomes by ethnicity.¹⁴ However, all datasets, including the NHI, have known ethnicity data quality and misclassification issues, which are well described and have not changed substantially over time. The 2018 Census²⁴ found that 13% of the population identified more than one ethnicity, whereas our project identified only 3–6% in the local HR systems and 5–6% in the NHI records. A recent study confirmed prior findings that Māori were undercounted by 16% on the NHI compared to the census ethnicity records.²³ The misclassification was more pronounced for Māori males, showing the net undercount of over 20% in most age groups.²³ Other local audits also documented the undercount of Māori on the NHI.²⁵ PHOs have been collecting ethnicity data more comprehensively recently, with changes in the primary healthcare funding in New Zealand in the early 2000s. A review on the studies of PHO ethnicity data quality by Cormack et al.¹⁴ noted varying kinds of misclassification disproportionately affecting some groups, often Māori. However, one of the most recent studies reported that the PHOs recorded higher Māori than the NHI (5.9% vs 4.0%) and less missing ethnicity data (7.2% vs 17.2%).²⁶ Our project indicated that 149 Māori were identified only in the NHI and not in the HR database at the Level 1 prioritised category, suggesting that Māori may also have been undercounted in the HR system, although this is likely to be an under-estimate given the NHI undercount.

The discordance in ethnicity data across different data sources is attributable to non-standard ethnicity data collection. For example, before the move to the census approach in 1996, which allows recording of up to three ethnicities, the health sector had collected only one ethnicity. Some local data recording systems still collect only one ethnicity.²⁷ As a result, often only one ethnicity is recorded where people self-identify multiple ethnicities; for example, if a person is both Māori and European, they tend to be recorded as European where systems record only one ethnicity. Non-compliant ethnicity recording may also contribute to the conflation of nationality with ethnicity as discussed in the previous report on Ethnicity, national identity and ‘New Zealanders’.²⁸ Ethnicity data in the Health and Disability sector workforce have been collected since 1991, when an ethnicity question was included with the annual medical Workforce Survey, alongside the demographic information at the time of registration. However, ethnicity data collection approaches still vary, such as a free-text box and multiple-choice check boxes, among the health professional organisations.¹⁴ The lack of the use of the Standard ethnicity question and compliant recording of multiple ethnicities may significantly impact the quality and comparability of data in reporting the ethnic workforce diversity, alongside context including the framing of the instructions. This analysis suggests that for current workforce diversity analyses, and for broader policy moves such as the move to an administrative census in New Zealand, a level of caution should be exercised in relation to ethnicity data quality and likely misclassification or undercounting.

Several efforts to improve the quality of ethnicity data have taken place in New Zealand health settings, including primary and secondary care. Achievement of high-quality ethnicity data requires compliance with the Standards, education and training, quality assurance with regular auditing, and quality improvement to vendor software. The ability for staff to see their recorded demographic details and update their data in an electronic portal may also be useful. Recording of ethnicity data became more standardised after the release of the 2005 Statistical Standard for Ethnicity and, more recently, the Ethnicity Data Protocols for the Health and Disability Sector.⁴ Ethnicity Data Audit tools have also been developed to improve the ethnicity data in PHOs²¹ and in hospitals,²⁹ along with several other indicators to monitor the quality of ethnicity data. The quality of ethnicity data held in databases has often been audited by providers and researchers locally or regionally, although overall responsibility and accountability to ensure high-quality ethnicity data have been unclear.³⁰ Our project, alongside other audits, highlights the need to establish well-defined roles and actions to improve ethnicity data quality, as outlined recently in the action plan report for Te Aka Whai Ora – Māori Health Authority.³⁰ It also reiterates the ongoing importance of improving ethnicity data quality as a key enabler of planning, monitoring and action towards health equity, including workforce diversity, with the ultimate aim of improving whānau health outcomes and experience.

We assessed the quality of ethnicity data of all staff recorded in the HR systems of two major health districts in New Zealand, Auckland and Waitematā. Our project contributes to the very limited research that addresses the HR ethnicity data quality issue. However, the project had limitations. We noted that response rates were variable across the two districts and that less than half of the staff who had no ethnicity stated in their HR record responded to the survey, despite follow-up contacts. The HR ethnicity data were compared to the NHI ethnicity data as reference, where we acknowledged that the NHI had documented issues with ethnicity data, with known under-representation of Māori. The differences between datasets might also reflect that ethnicity self-identification can change over time³¹ and that people may identify or respond differently in differing contexts—for example, in employment versus for healthcare. The fact that the majority of those whose ethnicity was not recorded in the HR records were non-Māori might also reflect something important about levels of cultural competence and/or race/ethnic consciousness, and dominant culture within the employment context.³²

Conclusions

A diverse health workforce contributes to more equitable outcomes and better patient experience. Monitoring and improving workforce diversity is a key goal of many health systems, including New Zealand. Research is very limited on health staff ethnicity data quality. The results of these two projects show the ongoing need for improved quality of workforce ethnicity data for both monitoring purposes and for planning workforce development activities. Comprehensive quality improvement activities would include compliance with the Standards for data collection and recording in HR systems, as well as training and HR process improvement.

View Appendix.