How to make effective health recommendations—implications for coronial practice: a narrative review

This review was undertaken as part of a wider research interest investigating ways to reduce preventable mortality. This arm of research was conducted to allow evidence-based advice to be provided to Aotearoa New Zealand’s coroners to assist them in making recommendations that are more likely to be implemented, as part of their death-preventing role, with the overall goal of reducing preventable mortality in Aotearoa New Zealand. Making recommendations is a widely undertaken practice for those interacting with the health sector, either from a coronial or other perspective. Given the dearth of information on the art and science of making recommendations, we wished to provide much needed guidance to support this activity.

Introduction

Coroners have the ability to make recommendations to improve outcomes. As part of their investigation into deaths, coroners are able to “make recommendations or comments … [that] may, if drawn to public attention, reduce the chances further deaths occurring in similar circumstances.”¹ Other organisations—for example, those involved in mortality review and other health-related activities—also have the ability to make recommendations to improve outcomes. In many contexts, recipients of recommendations are not obliged to implement or even respond to them, underscoring the need for recommendations to be carefully crafted in order to maximise the likelihood of uptake.

Ideally, one would investigate the characteristics of effective recommendations. However, the literature does not adequately address this topic. In the literature, the more commonly asked questions are “What recommendations were implemented”, and “What were the characteristics of these?” Therefore, given the dearth of research examining effective recommendations, this narrative review collates evidence on the characteristics of recommendations that are associated with implementation. A narrative review was chosen as the body of literature is broad and required nuanced description and analysis. A narrative review allows a wide variety of studies to be included, with subsequent interpretation and critique that provides a practical synthesis on a particular topic.²

Methods

Databases were searched for articles that investigated the degree to which recommendations were implemented or provided critique or commentary of recommendations. All searches were limited to research about human subjects written in the English language. Given that recommendation making is an activity carried out in a broad range of settings, a range of databases were selected that included both health and policy journals. Ovid MEDLINE, APA PsycInfo and sociology databases (Australian Public Affairs, Humanities & Social Sciences Collection, Health Collection and Asia Collection) were searched. The Boolean operators AND/OR were used with the following search terms: “recommendation(s)” AND “effective”, AND “policy” (public policy/or policy or health policy/or policy/or environmental policy/ or organisational policy/or policy making) OR “legislation”. In addition, a keyword search of the Journal of Public Policy was undertaken, using the keyword “recommendation”. In order to examine articles that might be published in legal journals, the law database of Gale OneFile was searched for articles using the keyword “coroner”. There were 4,462 article returns using the above database searches. Article titles, and abstracts if needed, were read in order to determine if articles were in scope. No articles that examined the effectiveness of coronial recommendations were found. Consequently, articles were read and kept if the title and/or abstract indicated they examined implementation of recommendations or provided systematic critique of coronial recommendations. Given the paucity of papers on the topic in the context of coronial recommendations, papers were also downloaded if they related to other types of independent inquiry relating to significant events or issues where there is a mandate to make independent recommendations, including mortality review, royal commissions of inquiry and other public inquiries. To increase the likelihood that all relevant papers were included, a snowballing technique was used, and further articles were identified from the reference lists of the papers identified using the methods above. The same process was applied, with titles and abstracts, if needed, being read to determine possible relevance. Articles were considered in-scope for this review if they were directly about recommendations, or if they were about the process of making recommendations in other independent settings. Specifically, articles were included if they examined the characteristics of recommendations or provided critique as to why recommendations were or were not implemented. A total of 64 articles were downloaded for review. Four publications reported on the same study, so three of these publications were excluded. Of the remaining 60 articles, 11 fit the inclusion criteria in that they were original research that systematically reviewed a number of recommendations and either examined implementation of these recommendations (n=9) or provided guidance about making recommendations through analysis of the content of recommendations (n=2). The most recent search was completed in March 2024.

Ethics approval was granted by the University of Otago Human Ethics Committee (Ethics Committee reference number HD20/060).

Results

Recommendations have been analysed in a number of different ways. Two studies examined the content of coronial recommendations in detail.^3,4 In their review of recommendations made in response to heavy vehicle crashes in Victoria, Australia, using an accepted injury prevention framework (the Haddon matrix), Brodie et al.³ found that while there was some evidence that coroners had referred to relevant national safety strategies, there were a number of areas in the national strategy that were not addressed and a number of lost opportunities to make recommendations. In their examination of coronial recommendations made in response to injury deaths using a bespoke framework incorporating elements of injury prevention, Bugeja et al.⁴ found a low prevalence of recommendation making. Of the recommendations that were made, a shortcoming was that there was little evidence that coroners’ recommendations were routinely informed by public health prevention principles. The authors noted the opportunity cost of not maximising the preventive potential of the coronial system and advocate for better incorporation of a public health perspective when making recommendations.

Similarly, in their review of recommendations related to mortality review, Baker and Griffin⁵ also noted the critical importance of public health theory in recommendations, specifically advocating for environmental change for long-term success, stating that recommendations advocating for behaviour change alone are weak.

Using a more general classification and including all causes of death, Moore and Henaghan⁶ classified coronial recommendations in Aotearoa New Zealand into categories of: 1) review of organisational practices, policies or procedures; 2) change to organisational practices, policies or procedures; 3) immediate law reform; and 4) publicity related to an issue. Forty-nine percent of recommendations were classified into categories 1 or 4, indicating that nearly half could be considered “soft” recommendations, i.e., recommendations that do not require a tangible change to factors that could be contributing to deaths, instead asking entities to “review” or “consider” a particular course of action.⁷ Sutherland et al.⁷ note the lack of utility of soft recommendations, likening them to a “free pass” for organisations.

One measure of the merit of a recommendation is how well it is received by the organisation to which it is directed.⁶ Research in Aotearoa New Zealand found that organisations considered that they (the organisation) should not be expected to implement change based on a single death, but that coroners should refer to similar cases or put that single death in the context of the overall burden of mortality and serious morbidity from that cause, and also put deaths from that cause into the context of wider mortality, in order to focus efforts on areas of significant mortality. Organisations stated they were more likely to implement coroners’ recommendations where the findings referred to similar cases.⁶

While many organisations seemed to understand the value of good recommendations, Sutherland et al.⁸ found that only 60% of recipient organisations thought that coroners’ recommendations were useful to their organisation, and only half believed that the recommendations would prevent future death and injury. Their study did not determine the feasibility, effectiveness and cost of recommendations, which would have enabled the authors to determine which recommendations were not implemented because of poor quality, and which were not implemented because of “poor choices and obstinacy among recipient organisations”.⁸

There was a small number of studies examining the implementation of recommendations.^5,6,8–12 Authors examined circumstances around both recommendations that were and were not implemented. See Table 1 and Table 2 for a summary of characteristics of recommendations that were and were not implemented.

Many authors noted the importance of clear communication. Being able to precisely identify the issue and provide solutions that were clearly implementable by a single organisation,⁵ or at least had a clear path to action,¹¹ were associated with implementation. Situations in which there was an insufficient link between findings and recommendations were associated with non-implementation.¹⁰ Recommendations that were vague and lacked clear action were less likely to be implemented. While there might have been a vision for improvements, unless this was accompanied by clear identification of both the problem and the solution they were less likely to be implemented. Similarly, uncertainties around the agency to implement change were found to be unhelpful.⁵ While it has often been lamented by coroners that their recommendations are “falling on deaf ears”,¹³ there was some evidence of a “cumulative effect” of recommendations, where having similar recommendations, or restating previous recommendations, could be a prompt for action,⁹ especially where there has been significant media attention.⁵ Fixing what was conspicuously broken as a first priority also improved success.¹¹

A number of studies commented on aspects of timing.^5,10–12 A prolonged period between the event and recommendations being made to agencies was associated with non-implementation.^10,12 The reasons for this are likely to be multiple but may include that timely reviews support maintained public and media interest in the topic,¹¹ and enable governmental or organisational accountability to remain consistent.¹² Other aspects of timing relate to the socio-political context. This ranges from simple aspects such as the time of year (i.e., recommendations to prevent drowning should be released at the start of summer), as well as the political context.⁵

Situations in which there was a blurring of jurisdiction for both prevention of the issue and implementation of the recommendation provided complexity. For implementation, situations in which there was overlap between the coronial role and that of other investigating authorities were associated with lower implementation rates.¹⁰ Similarly, deaths where the contributory factors sat under the influence of multiple sectors were also associated with lower implementation rates. In that instance Baker and Griffin⁵ recommended investing in collaborative work. The authors noted this is easier when there are shared values, and where recommendations can be framed to appeal to common values.⁵ A situation in which there are multiple vested interests can also be used to gain traction on a particular issue; the use of allies to support each other’s work was found to be beneficial in having recommendations implemented.^5,14

From their perspective as representatives of a mortality review committee, Baker and Griffin⁵ consider much of the focus of making recommendations should be on the process itself, rather than purely the actual recommendations. This includes activities such as good background work, including consultation and pre-release of information, defining allies, identifying foes and developing relationships with key stakeholders.⁵ Clark and Trick¹¹ also emphasised process as a crucial factor, with respect and the ability to find common ground substantially improving success, even to the point of being able to overcome fiscal constraints.

There were factors external to recommendation makers that were also found to be influential in the implementation of recommendations, both within organisations to which recommendations were directed and in the wider socio-political environment. The internal policies and practices of organisations could have an impact on the degree to which coronial recommendations are considered.^8,14 Some entities did not have an effective system to ensure that coronial recommendations were properly considered and responded to, or for monitoring and following through with recommendations. For example, in their study Watterson et al.⁹ found some government authorities were unable to answer the question of whether a recommendation was implemented or not. Leadership in institutions and a strong professional commitment to improve the performance of an organisation were associated with implementation of recommendations.¹²

Factors related to the socio-political environment included how well the recommendations aligned with government programmes already in existence, or aligned with current ideology or the policy agenda,^5,14 whether the inquest and its recommendations attracted media attention and whether some form of public advocacy accompanied the recommendation.¹⁴ For example, extensive media coverage and persistent lobbying by interested parties was shown in a case study to be one significant difference between the same recommendations that called for legislative change being implemented (Tasmania) or not implemented (Victoria).⁹

Recommendations that were implemented tended to be of low cost and relatively easy to implement.¹² In general, and particularly in resource-constrained environments, recommendations unlikely to attract additional funding were less likely to be implemented.^5,10 However, in some situations resource constraints could be overcome when other factors were favourable.¹¹

There appeared to be fewer barriers to implementing recommendations that affect the majority of the population, or that are considered to be a human right in society (e.g., access to clean drinking water).¹² This is problematic for issues that predominantly affect minoritised and marginalised groups. However, it is possible to write recommendations about politically sensitive topics in a way that draws support from those directly affected as well as the public as a whole. This can be achieved by focussing on priorities that have a strong support base in public opinion, and by linking new priorities to those that were already well established as being valued by the general public.¹¹

There were numerous reasons for not implementing recommendations, including: where recommendations were perceived as being impractical; where there was an insufficient link between findings and recommendations; where there was a range of behaviours that are considered to be acceptable practice; where there was implicit apportionment of blame; where coroners lacked expertise; and where the focus on single cases failed to put individual cases into the broader perspective.¹⁰

Summary of the key characteristics of implemented recommendations

Many aspects that contribute to recommendations being implemented are outside the control of those who make recommendations. These include the legislative framework under which they operate, organisational policies and culture and the socio-political environment. The following section relates to aspects that recommendation makers have some control over, to a greater or lesser degree.

Based on the findings from all the literature that was identified in the above searches, the characteristics of recommendations that were more likely (Table 1) and less likely (Table 2) to be implemented were as follows.

View Table 1–3.

In summary, we recommend that entities making health recommendations undertake the steps in Table 3 when crafting recommendations.

Discussion

The literature around the process of making recommendations is not substantial. There is very little work on the characteristics of effective recommendations, either in the coronial context or in other related contexts, such as mortality review or commissions of inquiry. There is more literature around recommendations that are implemented, and what organisations would like to see in recommendations. Characteristics of implemented recommendations are summarised in Table 1 and Table 2 above. The following section discusses key areas of tension and debate relating to the characteristics of recommendations that are implemented, weaknesses of using implementation as a measure of success when making recommendations and gaps in the literature.

Our review of the available evidence aligns with advice such as that from the World Health Organization¹⁶ and the National Institute for Health and Care Excellence,¹⁷ which provide broad guidance for making recommendations in particular situations.

Impediments to the implementation of recommendations

There are frequently impediments to recommendations being implemented. There have been instances where useful recommendations have been ignored until significantly more deaths have occurred.¹⁸ There are multiple potential explanations for this: organisations can have significant impediments to change, including a tendency to keep doing what they have always done, which requires less effort than change.¹⁹ Implementing change often requires a specific strategy. Organisations may wish to implement a recommendation but fail to apply an effective implementation strategy.²⁰ Change is often also costly. Furthermore, recommending change could be considered to imply issues of organisational underperformance. While interventions to reduce deaths could be considered an “investment”, there may be a substantial outlay of resource required, and it might be years before any financial benefit is realised, or it is not realised by the organisation that implemented the change.⁸ There is often a failure to prioritise the future over the immediate. Furthermore, there can be discord between what those investigating issues believe organisations should and can do, and what organisations feel they can do—possibly due to a “misalignment of perspectives”.⁸

Having a range of recommendations, from the relatively technical to those that involve systemic change, is a sensible way to approach complex issues.^5,12 However, one must consider whether this runs the risk of straightforward changes being implemented and the more difficult recommendations being deferred, with the reason given that the organisation has implemented “most” of the recommendations.

Equity considerations

The literature suggests that it is easier to implement recommendations when a large group of people is affected, and less easy to implement changes that benefit small and/or minoritised groups.¹² However, in Aotearoa New Zealand deaths are not distributed evenly throughout the population. Those who have been marginalised by social and economic policies bear the brunt of this.²¹ This ethnic and social gradient is not widely recognised as an infringement of one’s “right to life”; rather, it is often incorrectly viewed as a failure of the individual to do what is in their best interests.²² Clark and Trick¹¹ note the importance of “fixing what is conspicuously broken”. For example, in the presence of significant inequity in access to the determinants of health, it is nonsensical to make recommendations that require individuals to overcome significant resource-related barriers in order to comply. Where recommendations need to be targeted to overcome some of that marginalisation, it is possible that clear justification as to why the action needs to be carried out (including reference to relevant legislation, e.g., human rights legislation or constitutional documents, including Te Tiriti o Waitangi) might alleviate some of the opposition to necessary change.

The role of media

The literature suggests that publicity around deaths can influence outcomes.⁹ This can be either in a positive or negative way: in Aotearoa New Zealand, the publicity around the death of a 2-year-old girl who died from complications of an infection in a cold and mouldy Kāinga Ora (government-supplied) house was likely to have been influential in a significant number of Kāinga Ora houses being insulated and improved.²³ Conversely, the recommendation to have warning labels on soft drinks, following the death of a 31-year-old female from cardiac arrest following large daily consumption of Coca-Cola, received much attention both nationally^24,25 and internationally.^26–29 However, this did not result in the recommendations being implemented and is used as an example of coroners “getting it wrong”. While publicity can be a positive factor in achieving change, those who make recommendations need to be careful about how their recommendations will be perceived by the public. The ability to craft a good recommendation, with a clear link between the problem and solution, should provide recommendation makers with some protection against negative publicity and the negative reputational effects this could cause.

External considerations

Recommendations often fall into a messy social and political environment. Many of the factors that appear to influence the likelihood of a recommendation being implemented are not only outside the control of those making recommendations but are also unlikely to be favourable all at the same time. This phenomenon is well recognised in the policymaking arena and is not unique to recommendation makers.³⁰ However, the findings of this review suggest it would be beneficial for those making recommendations to have some understanding of the types of environments that are conducive to change in order for them to capitalise on this when they can.

Possible value of non-implemented recommendations

Primarily, a useful recommendation is one that achieves its aim. However, there is a difference between recommendations that are likely to be implemented and those that simply need to be stated. While the former are more likely to have an immediate effect, there is still likely to be value in making recommendations that require a high level of co-operation across multiple agencies to be implemented, or that require significant central government co-operation. Even though they are less likely to be implemented at the time, they provide a means for advocacy groups and others to gain inroads in important issues,⁹ can set an agenda for transformational change in the future⁵ and also provide recommendations to refer back to for people investigating similar cases.⁶

Strengths and limitations of this research

This review collates and synthesises the research about characteristics of implemented recommendations and provides guidance for those making recommendations. Making recommendations is a common activity in a range of settings, and this review provides a much-needed summary of the evidence to contribute to this process. Due to the dearth of coronial literature, findings from non-coronial settings were included in the review. This broad scope might be seen as a limitation. However, there was a high degree of commonality in findings from the different settings, resulting in conclusions that can be applied to non-coronial settings. Other limitations of this study include the limiting of articles to the English language. It is possible not all the relevant literature was identified.

Areas for further research

There are a number of gaps in the literature. Given the high frequency of recommendation making across many areas in health and public policy and the paucity of research examining either efficacy or implementation of such recommendations, priorities for future research include examination of both of these factors. There is also very little research examining recommendations using frameworks that are based on prevention theory. Because recommendations are not made in a void, but rather in complex environments,³⁰ any such research would benefit from considering not only the recommendations themselves but the social, cultural and political environments into which recommendations are made, particularly the identification of any modifiable structural barriers to implementing them. Other considerations, such as the influence of vested interests, would also be worth examining. Until there is a broader body of evidence to call upon, we advise anyone making recommendations to consult our list of characteristics of implemented recommendations and follow the steps outlined in Table 3 when deciding on the content and recipients of any recommendations they make.

Conclusion

The quality and utility of recommendations is important from both health and public policy perspectives. There is a high opportunity cost to society when interventions that yield the highest health return on investment are not implemented.³¹ Given the high toll of preventable death and serious morbidity, there is a need to gain as much benefit as possible from interventions that are most likely to be effective, particularly in systems, such as our coronial system, that have a specific preventive function. While there are many factors outside the control of individuals that influence whether recommendations are implemented, there are a significant number of characteristics that are within the control of the recommendation maker that could be heeded.