In 2016, frustrated researchers at the RAND Corporation were moved to publish a debunking of “the most prevalent myths” riddling the critiques of patient experience survey data used to measure and improve healthcare quality.
Full article available to subscribers
In 2016, frustrated researchers at the RAND Corporation were moved to publish a debunking of “the most prevalent myths” riddling the critiques of patient experience survey data used to measure and improve healthcare quality.1,2
In the Aotearoa New Zealand context, criticisms of patient-reported experience data are widespread, and equally frustrating. Few of these criticisms are, however, published in the peer-reviewed literature, possibly for the reasons we outline below in debunking these myths—they don’t hold sufficient water. However, this received wisdom has come to our attention, and tends to take the form of the following chestnuts, many common to other contexts, here deployed without evidence: that Aotearoa New Zealand’s patient experience survey response rates are too low to tell us anything of use, and that consequently measures are not robust, valid or reliable. That surveys undersample and are not valid or reliable for Māori and Pacific peoples.3 That surveys provide data, sometimes insights and opportunities, but no solutions.4 That none of this relates to improving outcomes.
Let’s put some of these myths to rest, but first: why involve patient experience and the patient voice in design and delivery of care in the first place?
The consumer health movement is not new, with its roots in the de-institutionalisation of mental health,5,6 rise in disability rights movement7,8 and special interest consumer groups.9 Similarly, the calls to better understand if and how these movements, and the involvement of consumers more generally, have led to a safer and higher quality health system remains a perennial subject.10,11,12 In a 2018 narrative review in BMJ Quality & Safety, 11 leaders in patient safety from the US and the UK wrote, “overwhelming evidence indicates that collecting patient feedback and including patients as equal partners in their care supports improvement in both patient experience of care and clinical outcomes.”13 A systematic review of 55 studies published in the BMJ in 2013 found “consistent positive associations between patient experience, patient safety and clinical effectiveness for a wide range of disease areas, settings, outcome measures and study designs … patient experience is positively associated with clinical effectiveness and patient safety, and support the case for the inclusion of patient experience as one of the central pillars of quality in healthcare.”14 Another 2014 review found, “better patient-reported care experiences are often associated with other aspects of healthcare quality, specifically:
Yet despite evidence for the benefits of patient and public involvement in designing, planning and co-producing healthcare services, and a broadly supportive policy context in the UK and internationally, a 2016 BMJ Quality & Safety paper found that progress in improving consumer engagement in production of healthcare services has been “patchy and slow and often concentrated at the lowest levels of involvement.”16
Te Tāhū Hauora has championed consumer and whānau engagement since its inception in 2010, and this work has now culminated in the development and embedding as secondary legislation of the Code of Expectations for Engagement with Consumers and Whānau for health entities17 (the Code) in section 59 of the Pae Ora (Healthy Futures) Act 2022.18
The Code sets the expectations for how health entities must work with consumers, whānau and communities in the planning, design, delivery and evaluation of health services. Between 2021 and 2022, Te Tāhū Hauora alongside the Health Transition Unit developed the Code, and under the Act there is now a statutory requirement for health entities to give effect to it, and to report annually on how it has been applied.
Part 2.2 of the Code states that health entities must use “lived experience, including consumer experience data to inform improvements in health services with a focus on reducing health inequities, particularly for Māori, Pacific peoples and disabled people.”19
For health service providers, use of patient experience survey programme data is a critical element in fulfilling this statutory expectation. Te Tāhū Hauora provides an implementation guide20 for health entities to learn how they can implement the Code, including use of survey programme data.
The first critical distinction to make in understanding patient-reported survey data is the distinction between patient satisfaction and patient experience.
Both satisfaction and experience surveys include multi-choice items.
Patient satisfaction surveys ask questions related to “How did we do?” with responses from “very poor” to “very good”. However, patient experience surveys ask questions related to what happened, with responses such as “always” to “never”. From this, we can derive actionable knowledge of what did or did not actually happen in a healthcare experience for large groups of people, rather than a non-actionable measure of how people felt, in a given place at a given time, for in many ways unknowable reasons.
Consider your own personal mood when confronted with the HappyOrNot™ terminal at an airport baggage carousel. You have a choice ranging between a green smiley face and a red angry face. But how much is your satisfaction with your baggage handling mediated by your prior expectation of a terrible experience, for example, or a particularly awful flight, or some fantastic news you received when you turned airport mode off on your phone at disembarkation?
Patient satisfaction responses are considered subjective and biased because satisfaction is personal and related to expectations and other unknowable influences. It is not an objective reflection of the presence or quality of a service or aspect of that service.21 Patient satisfaction responses also tend to be overwhelmingly positive and are easy to manipulate with framing of questions.19
Thus, rather than a question like “How did we do?”, patient experience surveys ask questions such as “In the last 12 months, was there ever a time when you wanted healthcare from a GP or nurse, but you could not get it?” A large number of responses thus gives a sense of barriers of access to care, and who experiences these. Another example, from the hospital inpatient survey, is, “Were you told what the medicine (or prescription for medicine) you left the hospital with was for?” This question reflects quality of discharge and communication, and using the experience explorer (see below), and other questions around discharge, services can easily see if they are providing the same quality of discharge information to all people, and how well they are doing in comparison with other parts of the country.
Patient experience is a more objective construct and survey questions that address specific aspects of care can prompt specific actions to generate improvement. Patient experience is also an intrinsic element of most generally accepted frameworks of healthcare quality, including the Triple Aim, a framework developed by the Institute for Healthcare Improvement that has been adopted as a set of principles for health system reform in multiple countries and organisations worldwide,22,23 the Quadruple Aim (building on the Triple Aim with the inclusion of workforce experience of providing care)21 and the National Academy of Medicine’s early and seminal six domains of quality developed in the US.24 For these reasons we believe patient experience surveys to be superior to satisfaction surveys.
But what of the accusations that Aotearoa New Zealand’s patient experience survey programme’s response rates are too low and sample sizes are too small to make accurate judgments, and that these samples are skewed in composition due to the nature of those the survey reaches, and who chooses to respond? What of the accusation that the length of the survey causes people to give up on it?
The accusations don’t hold water.
Te Tāhū Hauora’s patient-reported survey programme25 is in fact the second largest government survey programme in Aotearoa New Zealand we are aware of (larger than the New Zealand Household Disability Survey and the New Zealand Health Survey, and second only to the New Zealand Census). The programme surveys the experiences of adult inpatients in hospital settings and adult primary care patients (an adult hospital outpatient survey has been more recently established). Combined, the former two surveys collected data from over 154,000 respondents between May 2022 and February 2023.
The adult hospital inpatient experience survey (established 2014) and the adult primary care patient experience survey (established 2016) were also reviewed and substantially refreshed in 2019 and 2020.26 Cognitive testing was performed to understand and improve how patients understand and interpret questions and instructions in the survey and involved multiple in-depth interviews, with a particular focus on Māori and Pacific peoples’ responses.
The survey is a major repository of data. During the early period of the pandemic many national programmes, including survey rounds, were paused as workforce focussed on the pandemic response. A one-off COVID-19 specific survey to assess impacts of the pandemic response on patients’ experience of access to primary care during Level 3 restrictions was conducted,27 and surveying began again with the refreshed instruments. The data repository for the hospital inpatient survey now contains about 35,000 responses from August 2020 (after the refresh during pandemic restrictions). The primary care patient experience survey now covers over 90% of general practices, with data from 315,000 responses since the 2020 refresh. On average, 3,000 hospital inpatients respond to the adult hospital inpatient survey every quarter, and 35,000 primary care patients respond to the adult primary care patient experience survey every quarter. Every quarter this data pool grows, keeping pace with a changing system and context.
So much for response size. But what about response rate? Booker and colleagues cite comprehensive reviews of comparable surveys in primary care internationally and found response rates vary between 10–61%.28–31
The hospital inpatient survey’s response rate compares well, in the middle of the pack at 25%.
The primary care patient survey response rate, at 16–17%, is not so big, so we compensate for this by ensuring the large sample and comprehensive reach to practices (over 90%) mentioned above.
Response rates are important for three reasons, which we deal with in turn as follows:
View Figure 1–2.
Criticisms of so-called “opt-out” surveys draw attention to non-response bias, where those who choose to respond to a given survey are systematically different in some key way to those who choose not to. To understand if non-response bias existed in this large collection of survey data, a study was performed comparing the responses from a sample of initial hospital inpatient survey respondents (n=910) with the responses of a sample of non-responders who were followed up and invited again to take the survey (n=163). The study found no significant differences between the two groups, and that “responders to follow-up have similar experiences of inpatient care in New Zealand to initial responders.”31 There are of course limitations to this study, including potential bias in the follow-up responder group limiting the ability to ascertain the true extent of non-response bias.
So, there is evidence that non-response bias does not exist, and it is furthermore true that response rates per se are less important than representativeness—a large response rate from a group who are very different from non-responders will produced more biased results.33 Hence the importance to the Aotearoa New Zealand patient experience survey data of Amhed et al.’s finding that “response rates are only weakly associated with non-response bias in surveys that adhere to high standards of survey methodology”.33–35 Our patient experience survey adheres to best-practice survey methodology promulgated by Statistics NZ and international researchers36,37 to minimise attitudinal response bias. Measures are tested and validated, and weighting is used to adjust for patient characteristics that are likely to affect individual perceptions and expectations to enable comparison across healthcare providers.38 Representativeness we discuss below.
The last point of contention (again unevidenced), that the length of the survey causes many respondents to give up, is also regularly monitored. The completion rate of the surveys is over 90%, suggesting the majority of those who start the survey finish it, but the programme is constantly looking to improve. Completion time, drop-outs, item non-response and patient comments about the survey are regularly monitored. This helps identify questions that people find hard to answer (issues of cognitive load and response burden), which is a more important issue than the number of questions per se.39 As part of the annual review process, we also analyse responses so we can remove questions that are tightly linked to others. Another way we are reducing length is to move questions into annual modules to be asked once a year rather than every quarter.
The primary focus of the review and a constant in the evolution of the programme has been on improving participation for Māori and Pacific peoples, including increasing their response rates. We look next at whether this has succeeded.
Some have claimed that patient experience surveys are not valid or reliable for Māori and Pacific peoples due to low response rates. The significant work invested into improvements in this area has meant that, in the four quarterly surveys administered between early 2022 and early 2023, the historical equity gap in primary care survey response rates between Māori and non-Māori/non-Pacific peoples has disappeared.
Māori response rates now average 17% annually, while non-Māori/non-Pacific response rates now average 16% annually. Implementation of the successful improvements seen in the primary care survey are now in process for the hospital inpatient survey.
There were two key aspects of increasing participation rates: increasing the number of patients who are invited to take part, and increasing the proportion of patients who take part once invited.
The primary care survey now uses representative sampling to ensure a sufficient count of Māori and Pacific people are invited to the survey. In practice, this means a two-week sample of people identifying as being of Māori or Pacific ethnicity, and one week of all other ethnicities.
A range of other methods to increase both the numbers invited and the proportion of those who subsequently take part have addressed the entire pathway of survey response, including email address collection rates, culturally appropriate invitations, testing of the survey instruments with focus groups of Māori and Pacific peoples, and using a zero-data rated website, which allows access without using respondents’ mobile data allowances.40
Can patient experience in fact be acted on to improve the quality of services? It is clear to most that data help identify issues and areas that can benefit from further exploration and development of insights that prompt quality improvement action, but where do providers start in using these data for quality improvement? Te Tāhū Hauora provides resources specifically designed to assist providers in using survey results for quality improvement in both adult hospital inpatient and primary care settings41 to assist in the development of strategic, actionable insights and guide improvement.
Results from the surveys are also currently used across the country in multiple ways. At the national level these data are used to publicly report on the quality and safety of services, for example, through the annual Window on the quality of health care reports published by Te Tāhū Hauora. The latest Window report published in June 202342 provided an update on effects of the pandemic on our health system and included a focus on experience of care for disabled people during pandemic-affected periods.
Survey feedback also informs policy and monitoring of how health services are performing. The interim Health and Disability System Review drew on patient experience survey data to highlight areas for improvement in the health system restructure process, for example in reported barriers to accessing primary care.43 Data from the surveys where respondents report prescription cost as a barrier was used in the debate on Budget 2023’s change to remove prescription costs. These insights to inform high-level policy can only be obtained through nationally consistent, robust and valid surveys.
At local and regional levels, providers use survey data to monitor what is working well and what could be improved. For instance, during COVID-19 lockdowns, some hospitals were reporting their survey data to staff to show that, despite all the challenges associated with providing care in personal protective equipment and lockdown, patients were continuing to report positive experiences and were grateful for the care provided. Examples of specific quality improvement work drawing on survey findings are those which have been pursued at four district health boards (DHBs)—Northland, Waikato, Bay of Plenty, and Nelson Marlborough, addressing patient awareness of medication side effects and condition management post-discharge.44,45 Survey data for quality improvement work has also been embedded in the annual System Level Measures Improvement Plans of alliances in every district from 2016 to 2022. Examples are available from Auckland, Waitematā and Counties Manukau alliances for the 2020–2021 and 2021–2022 financial years.46,47
Health providers participating in the survey can access their own local results through a secure online portal. These dashboards enable people to access, cut and use the data in different ways, by local regions, demographic variables, question and domain, according to their needs and interests.
Providers and researchers interested in investigating the data are able to and encouraged to get in touch (contact, in the first instance, the corresponding author Catherine Gerard).
Both the adult primary care patient and adult hospital inpatient surveys now also have publicly available data explorers48,49 to make access to and investigation of experience data public, easy and intuitive. The explorers also allow data to be filtered by survey topics and questions, locations, periods and demographic elements of responders such as age, gender, ethnicity and disability status.
See Figure 2 for an example screenshot of the adult primary care patient experience data explorer, showing most-positive responses to the question: “Did the healthcare professional involve you as much as you wanted to be in making decisions about your treatment and care?”
Te Tāhū Hauora has administered national patient experience surveys since 2014. Since its inception, the programme has continuously grown and evolved, delivering valid, reliable, relevant, systematic and practical patient experience surveys and resulting data reflecting patient experiences of specific aspects of care. The patient-reported data and trends are translated into relevant metrics and indicators, easily accessible, to give providers a guide to how well their services are being delivered to their populations.
Te Tāhū Hauora patient experience survey data supplies coordinates and a compass to guide more patient-centred care. These surveys place patient feedback at the heart of quality improvement and health system responsiveness. The myths debunked above, we suspect, derive simply from suspicion premised on unfamiliarity.
To this we would add: thanks to the survey, we have heard from a representative sample of New Zealanders about the experience of healthcare they have had. We have this data, and we have a statutory imperative. We are now legally and ethically50,51 bound to listen and to use the results for improvement.
Patient-reported data derived from surveys places patient feedback at the heart of quality improvement and health system responsiveness. Such surveys are not without critics, however, who contend that there are better ways to collect feedback. Criticisms assert that response rates are too low and measures are not robust, valid or reliable, that patient experience surveys are neither valid nor reliable for Māori and Pacific peoples and that such surveys do not contribute to improved outcomes for patients.
We debunk these myths in the context of the Te Tāhū Hauora Health Quality & Safety Commission (Te Tāhū Hauora) patient experience survey programme. We explain the centrality of a strong consumer and whānau voice in a twenty-first century health system, and that listening to and acting on this voice—including use of patient-reported data—is now a statutory requirement for health entities under the Pae Ora (Healthy Futures) Act 2022. We describe the different surveys in the programme and explain the differences between patient satisfaction and patient experience. We address sample size and response rates, including representativeness in the surveys of Māori and Pacific peoples’ experience. We look at how survey data can be used for quality improvement and to guide us toward providing equitable, culturally safe care.
We assert that, contrary to criticisms, the programme delivers valid, reliable, relevant, systematic and practical patient experience surveys and resulting data, with guides for improvement, and that we are both legally and ethically bound to listen to and use these results to improve the healthcare we deliver.
Catherine Gerard: Assistant Director, Health Quality Intelligence, Te Tāhū Hauora Health Quality & Safety Commission, Wellington, Aotearoa New Zealand.
Inga O’Brien: Senior Advisor, Patient Survey Programme, Health Quality Intelligence, Te Tāhū Hauora Health Quality & Safety Commission, Wellington, Aotearoa New Zealand.
Carl Shuker: Principal Advisor, Publications, Health Quality Intelligence, Te Tāhū Hauora Health Quality & Safety Commission, Wellington, Aotearoa New Zealand.
Jo Swanson: Survey manager, Health Quality Intelligence, Te Tāhū Hauora Health Quality & Safety Commission, Wellington, Aotearoa New Zealand.
Richard Hamblin: Director, Health Quality Intelligence, Te Tāhū Hauora Health Quality & Safety Commission, Wellington, Aotearoa New Zealand.
Catherine Gerard: Assistant Director, Health Quality Intelligence, Te Tāhū Hauora Health Quality & Safety Commission, 17–21 Whitmore Street, Wellington, Aotearoa New Zealand.
Nil.
1) Anhang Price R, Elliott MN. Debunking Myths About Patient Experience Surveys [Internet]. Santa Monica, CA (US): RAND Corporation; 2016 [cited 2023 Nov 14]. Available from: https://www.rand.org/pubs/commentary/2016/05/debunking-myths-about-patient-experience-surveys.html.
2) Anhang Price R, Elliott MN, Cleary PD, et al. Should health care providers be accountable for patients’ care experiences? J Gen Intern Med. 2015;30(2):253-6. doi: 10.1007/s11606-014-3111-7.
3) Fink JW, Paine SJ, Gander PH, et al. Changing response rates from Māori and non-Māori in national sleep health surveys. N Z Med J. 2011;124(1328):52-63.
4) Reeves R, Seccombe I. Do patient surveys work? The influence of a national survey programme on local quality-improvement initiatives. Qual Saf Health Care. 2008;17(6):437-41. doi: 10.1136/qshc.2007.022749.
5) Gooding P. From deinstitutionalisation to consumer empowerment: Mental health policy, neoliberal restructuring and the closure of the Big bins in Victoria. Health Sociol Rev. 2016;25(1):18-32. doi: 10.1080/14461242.2015.1134261.
6) Crowe M, O’Malley J, Gordon S. Meeting the needs of consumers in the community: a working partnership in mental health in New Zealand. J Adv Nurs. 2001;35(1):88-96. doi: 10.1046/j.1365-2648.2001.01825.x.
7) Winter JA. The development of the disability rights movement as a social problem solver. DSQ. 2003;23(1):33-61. doi: 10.18061/dsq.v23i1.399.
8) Scotch RK. Politics and policy in the history of the disability rights movement. Milbank Q. 1989;67 Suppl 2 Pt 2:380-400.
9) Coney S. Effective consumer voice and participation for New Zealand: a systematic review of the evidence. Wellington (NZ): New Zealand Guidelines Group; 2004 [cited 2023 Aug 22]. Available from: https://www.moh.govt.nz/
10) Roseman D, Osborne-Stafsnes J, Amy CH, et al. Early lessons from four ‘aligning forces for quality’ communities bolster the case for patient-centered care. Health Aff (Millwood) 2013;32(2):232-41. doi: 10.1377/hlthaff.2012.1085.
11) Hamilton Lopez M, Holve E, Rein A, Winkler J. Involving patients and consumers in research: new opportunities for meaningful engagement in research and quality improvement [Internet]. Washington DC (US): EDM Forum Community, AcademyHealth; 2012 [cited 2023 Aug 22]. Available from: https://academyhealth.org/sites/default/files/Involving%20Patients%20and%20Consumers%20in%20Research_%20New%20Opportunities%20f.pdf.
12) Holden RJ, Valdez RS, Anders S, Ewart C. The Patient Factor: Involving Patient and Family Stakeholders as Advisors, Co-Designers, Citizen Scientists, and Peers. Proc Hum Factors Ergon Soc Annu Meet. 2020;64(1):622-26. doi: 10.1177/1071181320641141.
13) Gandhi TK, Kaplan GS, Leape L, et al. Transforming concepts in patient safety: a progress report. BMJ Qual Saf. 2018;27(12):1019-1026. doi: 10.1136/bmjqs-2017-007756.
14) Doyle C, Lennox L, Bell D. A systematic review of evidence on the links between patient experience and clinical safety and effectiveness. BMJ Open. 2013;3(1):e001570. doi: 10.1136/bmjopen-2012-001570.
15) Anhang Price R, Elliott MN, Zaslavsky AM, et al. Examining the role of patient experience surveys in measuring health care quality. Med Care Res Rev. 2014;71(5):522-54. doi: 10.1177/1077558714541480.
16) Ocloo J, Matthews R. From tokenism to empowerment: progressing patient and public involvement in healthcare improvement. BMJ Qual Saf. 2016;25(8):626-32. doi: 10.1136/bmjqs-2015-004839.
17) Te Tāhū Hauora Health Quality & Safety Commission. Code of expectations for health entities’ engagement with consumers and whānau [Internet]. Wellington (NZ): Te Tāhū Hauora; 2023 [cited 2023 Aug 22]. Available from: https://www.hqsc.govt.nz/consumer-hub/engaging-consumers-and-whanau/code-of-expectations-for-health-entities-engagement-with-consumers-and-whanau/.
18) Pae Ora (Healthy Futures) Act 2022.
19) Te Tāhū Hauora Health Quality & Safety Commission. Code of expectations for health entities’ engagement with consumers and whānau. Part 2.2 [Internet]. Wellington (NZ): Te Tāhū Hauora; 2023 [cited 2023 Aug 22]. Available from: https://www.hqsc.govt.nz/consumer-hub/engaging-consumers-and-whanau/code-of-expectations-for-health-entities-engagement-with-consumers-and-whanau/.
20) Te Tāhū Hauora Health Quality & Safety Commission. Te aratohu tikanga: Code of expectations implementation guide [Internet]. Wellington (NZ): Te Tāhū Hauora; 2023 [cited 2023 Aug 22]. Available from: https://www.hqsc.govt.nz/consumer-hub/engaging-consumers-and-whanau/implementing-the-code/.
21) Dunsch F, Evans DK, Macis M, Wang Q. Bias in patient satisfaction surveys: a threat to measuring healthcare quality. BMJ Global Health. 2018;3(2):e000694. doi: 10.1136/bmjgh-2017-000694.
22) Institute for Healthcare Improvement (IHI). Triple Aim and Population Health [Internet]. Boston, MA (US): Institute for Healthcare Improvement; 2023 [cited 2023 Aug 22]. Available from: https://www.ihi.org/Engage/Initiatives/TripleAim/Pages/default.aspx.
23) Sikka R, Morath JM, Leape L. The Quadruple Aim: care, health, cost and meaning in work. BMJ Qual Saf. 2015;24(10):608-10. doi: 10.1136/bmjqs-2015-004160.
24) Institute of Medicine (IOM). Crossing the Quality Chasm: A New Health System for the 21st Century. Washington DC (US): National Academy Press; 2001.
25) Te Tāhū Hauora Health Quality & Safety Commission. Patient-reported measures [Internet]. Wellington (NZ): Te Tāhū Hauora; 2023 [cited 2023 Aug 22]. Available from: https://www.hqsc.govt.nz/our-data/patient-experience/.
26) Te Tāhū Hauora Health Quality & Safety Commission. New Zealand patient experience survey programme refresh 2019/20 [Internet]. Wellington (NZ): Te Tāhū Hauora; 2021 [cited 2023 Aug 22]. Available from: https://www.hqsc.govt.nz/resources/resource-library/new-zealand-patient-experience-survey-programme-refresh-201920/.
27) Te Tāhū Hauora Health Quality & Safety Commission. New Zealand COVID-19 patient experience survey questionnaire [Internet]. Wellington (NZ): Te Tāhū Hauora; 2020 [cited 2023 Aug 22]. Available from: https://www.hqsc.govt.nz/our-data/patient-reported-measures/patient-experience/survey-results/#COVID.
28) Booker QS, Austin JD, Balasubramanian BA. Survey strategies to increase participant response rates in primary care research studies. Fam Pract. 2021;38(5):699-702. doi: 10.1093/fampra/cmab070.
29) Creavin ST, Creavin AL, Mallen CD. Do GPs respond to postal questionnaire surveys? A comprehensive review of primary care literature. Fam Pract. 2011;28(4):461-7. doi: 10.1093/fampra/cmr001.
30) Ricci-Cabello I, Avery AJ, Reeves D, et al. Measuring patient safety in primary care: the development and validation of the “Patient Reported Experiences and Outcomes of Safety in Primary Care” (PREOS-PC). Ann Fam Med. 2016;14(3):253-61. doi: 10.1370/afm.1935.
31) Medina-Lara A, Grigore B, Lewis R, et al. Cancer diagnostic tools to aid decision-making in primary care: mixed-methods systematic reviews and cost-effectiveness analysis. Health Technol Assess. 2020;24(66):1-332. doi: 10.3310/hta24660.
32) Thomson M, Pledger M, Hamblin R, et al. Comparing initial and follow-up responders to a New Zealand patient experience survey. N Z Med J. 2018;131(1482):46-58.
33) Johnson TP, Wislar JS. Response rates and nonresponse errors in surveys. JAMA. 2012;307(17):1805-6. doi: 10.1001/jama.2012.3532.
34) Ahmed F, Burt J, Roland M. Measuring patient experience: concepts and methods. Patient. 2014;7(3):235-41. doi: 10.1007/s40271-014-0060-5.
35) Groves RM, Peytcheva E. The impact of nonresponse rates on nonresponse bias: a meta-analysis. Public Opin Q. 2008;72(2):167-89. doi: 10.1093/poq/nfn011.
36) Stats NZ Tatauranga Aotearoa. A guide to good survey design: Fifth edition [Internet]. Wellington (NZ): Stats NZ Tatauranga Aotearoa; 2019 [cited 2023 Nov 14]. Available from: https://www.stats.govt.nz/assets/Uploads/Methods/A-guide-to-good-survey-design-fifth-edition/a-guide-to-good-survey-design-fifth-edition.pdf.
37) Leeuw ED, Hox JJ, Dillman DA, editors. International handbook of survey methodology. New York (US): Lawrence Erlbaum Associates; 2009.
38) Te Tāhū Hauora Health Quality & Safety Commission. Survey information and methodology [Internet]. Wellington (NZ): Te Tāhū Hauora; 2023 [cited 2023 Nov 14]. Available from: https://www.hqsc.govt.nz/our-data/patient-reported-measures/patient-experience/survey-information-and-methodology/.
39) Rolstad S, Adler J, Rydén A. Response burden and questionnaire length: is shorter better? A review and meta-analysis. Value Health. 2011;14(8):1101-8. doi: 10.1016/j.jval.2011.06.003.
40) Te Whatu Ora – Health New Zealand. Zero Data [Internet]. Wellington (NZ): Te Whatu Ora – Health New Zealand; 2023 [cited 2023 Aug 22]. Available from: https://www.tewhatuora.govt.nz/our-health-system/digital-health/zero-data/.
41) Te Tāhū Hauora Health Quality & Safety Commission. From PES to PDSA – Workbook: Using adult primary care patient experience survey data for quality improvement [Internet]. Wellington (NZ): Te Tāhū Hauora; 2023 [cited 2023 Nov 7]. Available from: https://www.hqsc.govt.nz/resources/resource-library/from-pes-to-pdsa-workbook-using-adult-primary-care-patient-experience-survey-data-for-quality-improvement/.
42) Te Tāhū Hauora Health Quality & Safety Commission. A window on quality 2022 (Part 2): COVID-19 and impacts on our broader health system (Part 2) | He tirohanga kounga 2022: Me ngā pānga ki te pūnaha hauora whānui (Wāhanga 2) [Internet]. Wellington (NZ): Te Tāhū Hauora; 2023 [cited 2023 Aug 22]. Available from: https://www.hqsc.govt.nz/resources/resource-library/a-window-on-quality-2022-part-2-whakarapopototanga-matua-he-tirohanga-kounga-2021-wahanga-2/.
43) Manatū Hauora – Ministry of Health. Health and Disability System Review – Interim Report [Internet]. Wellington (NZ): Manatū Hauora – Ministry of Health; 2019 [cited 2023 Aug 22]. Available from: https://www.health.govt.nz/publication/health-and-disability-system-review-interim-report.
44) Jaine R, Giness D, Hohaia J, et al. Raising the bar on the National Patient Experience Survey [Internet]. Wellington (NZ): Health Quality & Safety Commission New Zealand; 2017 [cited 2023 Nov 7]. Available from: https://www.hqsc.govt.nz/assets/Consumer-hub/Partners-in-Care/Publications-resources/Raising-the-bar-on-the-National-Patient-Experience-Survey-Final-REVIEWED-Report-19-05-2017.pdf.
45) George V, Jaine R, Opferkuch C, et al. Raising the Bar on the National Adult Inpatient Experience Survey Phase 2: Co-Designing Nudges [Internet]. Wellington (NZ): Health Quality & Safety Commission New Zealand; 2018 [cited 2023 Nov 7]. Available from: https://www.hqsc.govt.nz/assets/Our-data/Publications-resources/Co-designing-nudges-raising-the-bar-on-the-national-adult-inpatient-experience-survey-Nov-2018.pdf.
46) Auckland, Waitemata & Counties Manukau Health Alliances. 2020/2021 System Level Measures Improvement Plan [Internet]. Auckland (NZ): Waitemata DHB; 2021 [cited 2023 Nov 7]. Available from: https://www.waitematadhb.govt.nz/assets/Documents/health-plans/Metro-Auckland-SLM-Improvement-Plan-2020-21-Final.pdf.
47) Auckland, Waitemata & Counties Manukau Health Alliances. 2021/2022 System Level Measures Improvement Plan [Internet]. Auckland (NZ): Auckland DHB; 2022 [cited 2023 Nov 7]. Available from: https://www.adhb.health.nz/assets/Documents/About-Us/Planning-documents/Metro-Auckland-SLM-Improvement-Plan-2021-2022.pdf.
48) Te Tāhū Hauora Health Quality & Safety Commission. Adult primary care patient experience explorer [Internet]. Wellington (NZ): Te Tāhū Hauora; 2023 [cited 2023 Aug 22]. Available from: https://reports.hqsc.govt.nz/APC-explorer/.
49) Te Tāhū Hauora Health Quality & Safety Commission. Adult hospital inpatient experience explorer [Internet]. Wellington (NZ): Te Tāhū Hauora; 2023 [cited 2023 Aug 22]. Available from: https://reports.hqsc.govt.nz/AHI-explorer/.
50) Locock L, Montgomery C, Parkin S, et al. How do frontline staff use patient experience data for service improvement? Findings from an ethnographic case study evaluation. J Health Serv Res Policy. 2020;25(3):151-161. doi: 10.1177/1355819619888675.
51) Coulter A, Locock L, Ziebland S, Calabrese J. Collecting data on patient experience is not enough: they must be used to improve care. BMJ. 2014;348:g2225. doi: 10.1136/bmj.g2225.
Sign in to view your account and access
the latest publications by the NZMJ.
Don't have an account?
Let's get started with creating an account.
Already have an account?
Become a member to enjoy unlimited digital access and support the ongoing publication of the New Zealand Medical Journal.
The New Zealand Medical Journal is fully available to individual subscribers and does not incur a subscription fee. This applies to both New Zealand and international subscribers. Institutions are encouraged to subscribe. The value of institutional subscriptions is essential to the NZMJ, as supporting a reputable medical journal demonstrates an institution’s commitment to academic excellence and professional development. By continuing to pay for a subscription, institutions signal their support for valuable medical research and contribute to the journal's continued success.
Please email us at nzmj@pmagroup.co.nz