Findings from Focus Groups and Consensus Meeting in Pristina, Kosovo

Download PDF version

Prepared by
Knowledge Translation Program, Li Ka Shing Knowledge Institute, St. Michael’s Hospital, Canada
Contact

Abbreviations

AGREE
Appraisal of Guidelines Research and Evaluation
BMJ
British Medical Journal
doi
digital object identifier
Dr.
doctor
e.g.
exempli gratia; for example
et al.
et alii; and others
ext
extension
GRADE
Grading of Recommendations Assessment, Development, and Evaluation
GREAT
Guideline-driven, Research priorities, Evidence synthesis, Application of evidence, and Transfer of knowledge
i.e.
id est; that is
KT
knowledge translation
LMICs
low or middle income countries
mcg
microgram
n
number
NCCMT
National Collaborating Centre for Methods and Tools
NGO
non-governmental organization
PDF
Portable Document Format
PMCID
PubMed Central identifier
PMID
PubMed identifier
PPH
post-partum haemmorrhage
RAND
Research ANd Development
SD
standard deviation
St.
Saint
TIES
Team for Implementation, Evaluation and Sustainability
UCLA
University of California, Los Angeles
WHO
World Health Organization

Executive summary

Background

Local stakeholders including clinicians, health care managers, researchers and policy-makers from Kosovo were invited to participate in a focus group followed by a one-day meeting as part of the program entitled, “GREAT” (Guideline-driven, Research priorities, Evidence synthesis, Application of evidence, and Transfer of knowledge). The meeting was held in Pristina, Kosovo in October 2012. It focused on the key messages of the World Health Organization’s (WHO) guidelines on maternal health, specifically on the prevention and management of postpartum hemorrhage. During the focus groups, participants discussed the local barriers and facilitators to the implementation of the WHO guidelines in diverse clinical settings across Kosovo. The meeting facilitators presented information on knowledge translation, and WHO guideline development, and reported key findings from the focus groups and previously conducted surveys and interviews. Subsequently, participants used the RAND Appropriateness Method (Fitch et al., 2001) to vote on the importance of elements in the WHOguidelines on postpartum hemorrhage and on barriers to their implementation.

Methods

Focus groups were conducted with participants on Day 1 of the GREAT meeting. Participants were assigned to one of two focus groups; 11 participants attended focus group #1, and 8 participants attended focus group #2. The focus groups were led by two facilitators: Dr. Sharon Straus (Director of Knowledge Translation (KT) Program) and Dr. Julia Moore (Leads the Team for Implementation, Evaluation and Sustainability [TIES] for theKT Program). A translator was present to provide assistance with English to Albanian translation. During Day 2 of the meeting, facilitators and participants reviewed and discussed results from surveys, interviews, and focus groups using a nominal group process (Jones & Hunter, 1995). Participants were asked to discuss and rate the importance of WHO guidelines and of previously identified barriers to WHO guideline implementation using a process consistent with the RAND appropriateness method (Fitch et al., 2001).

Findings

Key findings from focus groups were centred on 2 main themes which are described below.

1. Barriers to implementation

Several barriers to implementation were identified at both the individual health care provider level and the health care systems level. At the level of health care provider, participants reported the following barriers:

  • Lack of understanding of and engagement in WHO guideline development process.
  • Lack of capacity to document and monitor clinical practice.
  • Lack of communication and agreement among clinician groups (specifically, obstetricians and midwives).
  • Lack of training and continuing education opportunities available for clinicians and undergraduate students.

At the health care system level, participants identified the following barriers:

  • Fragmented system (post-conflict).
  • Lack of communication between Ministry of Health and clinician organizations.
  • Lack of clarity regarding roles and responsibilities of various clinicians, managers and policy makers.
  • Lack of a centralized system to document and access clinical data.
  • Lack of resources to implement the guideline such as relevant medications and equipment.
2. Suggestions for implementation

Suggestions to address some of the above-mentioned barriers to implementation were identified by participants during the focus groups. These suggestions included:

  • Create a centralized system for data collection across clinical settings as well as for formal and informal channels for practice sharing.
  • Incorporate standards into clinical practice including a monitoring system for guideline adherence.
  • Create motivational strategies such as, incentives for health care staff, (including managers and clinicians) to encourage guideline adherence.
  • Increase communication across stakeholder groups including clinicians, managers and policy makers through participation in activities such as guideline development committees.
  • Create a guideline implementation working group with representative stakeholders at the local level.

Recommendations and future directions

Implementation of the WHO maternal and perinatal guidelines in Kosovo requires consideration of unique barriers and facilitators as identified by local stakeholders. The impact of recent conflict including, a fragmented health care system and disjointed infrastructure for communication among health care stakeholders were identified as prominent barriers to guideline implementation. However, this meeting also offered a timely and valuable opportunity to build capacity at the level of individual health care providers and the health care system in preparation for development and execution of a successful implementation plan. In addition to improved local capacity and interprofessional communication, there are also important opportunities to share knowledge and implementation resources internationally. Based on the outcomes of the focus groups and consensus meeting, recommendations for next steps are outlined:

  • Develop a small working group with local representatives from clinician groups, the Ministry of Health guidelines committee and quality portfolio, clinical or health services researchers, and the WHO to move forward with implementation.
  • Consider offering workshops on guideline development methods (including use of GRADE (Guyatt et al., 2008), on appraisal of guidelines using AGREE, and on guideline adaptation (National Collaborating Centre for Methods and Tools [NCCMT], 2011), for representatives from the Ministry of Health and clinical groups.
  • Consider engaging some of the local clinicians on the WHO guidelines development group.
  • Engage those interested in guideline development and implementation from neighbouring countries in the workshop activities and create a ‘virtual’ community of practice to share experiences and avoid duplication of effort.

Purpose and approach

The purpose of this report is to provide GREAT Project stakeholders with the findings from the focus groups and consensus meeting held in Pristina, Kosovo in October 2012. Findings are organized according to:

  1. barriers identified at the health care provider and health care system level (to local implementation ofWHO maternal and perinatal guidelines); and
  2. suggestions for implementation. The analysis of the focus group data and consensus-building activities has yielded practical recommendations to inform future directions to implement WHO maternal and perinatal guidelines in Kosovo.

The logic model for the overall project is depicted in Figure 1 (refer to page 6) as an overview of the activities carried out by the project team in preparation for, during, and following the consensus meeting. The activities are linked to both short-term (by the end of consensus meeting) and longer-term objectives (through the efforts of the local working group) with the ultimate goal of reducing rates of maternal and perinatal morbidity and mortality.

Figure 1. GREAT project logic model
Activities
  • Assess knowledge and awareness about WHO maternal and perinatal guidelines through:
    • Surveys
    • Interviews
  • Provide education and resources on:
    • WHO guideline development process and maternal and perinatal guidelines
    • Knowledge to Action Cycle
  • Conduct focus groups to identify barriers and facilitators
  • Facilitate consensus-building activities
  • Provide opportunities for networking and mentorship
  • Identify members for, and support, local working group
Short-term outcomes (after 2-day consensus meeting)
  • Increase awareness and understanding of:
    • WHO guideline development
    • Maternal and perinatal guidelines
    • Knowledge to Action Cycle
  • Improve communication between meeting stakeholders
  • Identify local barriers and facilitators
  • Establish stakeholder consensus on importance of barriers
  • Understand resources and plan of action
Intermediate outcomes (achieved by working group)
  • Formation of local working group
  • Development of implementation plan
  • Implementation and evaluation of initiatives
  • Improved communication and collaboration between:
    • Working group members
    • Levels and sectors of government
    • International partners
  • Use of resources:
    • Accessed on portal
    • Application for additional funding
    • Internal/external requests
Near-final outcome
  • Improved adherence to WHO maternal and perinatal guidelines
  • Development of research agenda and capacity-building strategy in other LMICs
Final outcome
Reduced rates of maternal and perinatal mortality and morbidity
Target audience
  • Clinicians, health care managers, researchers, and policy-makers in Kosovo
  • Members of the World Health Organization

Methodology

Design and sampling

Health care managers, health care practitioners, and policy-makers were identified and purposively sampled for inclusion in the meeting through the WHO Liaison Office for Kosovo, led by Dr. Sami Uka. Invitations to attend the meeting were sent by email and email reminders were sent subsequently to confirm attendance as well as to optimize responses. Of the 30 email invitations that were sent, 19 invitees consented to participate. Participants were sent a package containing a summary of the WHO postpartum hemorrhage (PPH) guidelines and an overview of the goals for the meeting. To achieve rigorous qualitative methodology and ensure saturation of themes, participants were divided into two focus groups and the sample stratified to ensure representation from professions and organizations at both sessions. Participants from the two focus groups were then invited to attend an in-person consensus meeting on Day 2. The demographic information for focus group and consensus meeting participants is found in Appendix 1.

Focus groups

The focus group sessions were facilitated by two experienced researchers from Toronto, Canada. A translator was in attendance. Focus groups were recorded and facilitators took field notes to supplement the recordings. Each focus group session lasted approximately 90 minutes. The goal of these focus groups was to explore barriers and facilitators to implementation of the WHO postpartum hemorrhage guidelines.

Consensus meeting

The in-person consensus meeting was facilitated by Dr. Sharon Straus. Two translators were in attendance. Dr.Straus presented on research in knowledge translation, and Dr. Metin Gülmezoglu (coordinator of the GREATinitiative through the Department of Reproductive Health and Research at the WHO) discussed the WHOprocess for developing guidelines. Following these presentations, Dr. Straus used a nominal group process to have participants rate the importance of WHO guidelines using a 9-point Likert scale. Participants subsequently reviewed and discussed results obtained during surveys, interviews, and focus groups and then rated each identified barrier to implementing the guidelines. An electronic audience response system was used so that participant ratings could be shared in real time. Consistent with the RAND Appropriateness Method (Fitch et al., 2001) ratings were done using a Likert scale from 1 to 9 where 1 = extremely unimportant and 9 = extremely important. The purpose of this ranking exercise was to identify the extent of consensus among participants and to allow for prompt reflection and deliberation.

Data collection and analysis

Each focus group session lasted approximately 90 minutes. Both focus group sessions were digitally recorded and transcribed verbatim by a trained transcriptionist. Data analysis was conducted by an expert analyst in consultation with the meeting facilitators to discuss interpretations of the data for a shared understanding of key findings. After familiarization with the data from the transcripts, a coded framework was developed using the grounded theory approach and analysis was performed using NVivo 10, qualitative software. Two people independently developed the coding scheme. Codes were grouped to form categories of similar content and categories were further grouped into themes which describe the findings from the focus group.

Findings

Focus groups

Participants were provide with a summary of the WHO guidelines on the prevention and management of PPHand asked to identify and discuss local barriers and facilitators to their implementation.

Results from the focus groups were organized into 2 main themes:

  1. barriers to implementation (health care provider level and health care system level); and
  2. suggestions for implementation.

Each of these main themes is further examined through the delineation of sub-categories. Quotes from participants that support and illustrate the themes and sub-categories are provided in italics.

The framework of focus group findings is depicted in Figure 2.

Figure 2. Framework of focus group findings
  1. Barriers to implementation
    • Health care provider level
      • Lack of understanding of and engagement in WHO guideline development process
      • Lack of capacity to document and monitor clinical practice
      • Lack of communication and agreement among clinician groups
      • Lack of training and continuing education opportunities
    • Health care system level
      • Fragmented system
      • Disconnect between Ministry of Health and clinician organizations
      • Lack of clarity of roles and responsibilities
      • Lack of centralized system to collect and share data
      • Lack of resources
  2. Suggestions for implementation
1. Barriers to implementation
Health care provider level
Lack of understanding of and engagement in the guideline development process:
Participants reported a lack of understanding of the methodology used by the WHO in creating their guidelines:

Translator: “he’s asking questions in terms of how the [WHO] recommendations, how the guidelines were drafted”

Many of the participants stated that they had no knowledge of the Kosovan Guidelines Committee which was developed to create or appraise guidelines for implementation.

[Cross-conversation between participants]: “So, so you stated that there is a Kosovo council for, for drafting guidelines and protocols. Do you have any protocols approved or, or drafted in relation to gynecology?” [Participant responds ‘yes’]. “…Uh”

A number of participants felt that the guidelines that were being considered by the Guidelines Committee were being developed without formal engagement of clinicians from relevant settings. This lack of engagement left some participants feeling disinterested or weary of implementation efforts at the local practice level.

Participant: “What I see as a challenge is information, awareness raising for health care professionals at all levels, that would be one challenge, so that these recommendations from the centre to, to convey them further to lower levels.”

Lack of capacity to document and monitor clinical practice:
Participants expressed concern about their lack of capacity to document and monitor current clinical practice and recognized the importance of measurement on the ability to provide optimal quality of care:

Participant: “We come to the problem of how to collect this information that whether the protocol is being implemented and what to do with those not implementing the protocol.”

Participant: “…in limited resource countries, low income countries, and places which, uh, have, have difficulties in terms of data collection and health information systems, obviously lacking evidence, so we cannot, uh, feed the protocols with our own evidence or with our own data.”

Furthermore, this lack of documentation and monitoring has prohibited participants from attaining information on their current state, and they are therefore not able to measure the impact of guideline implementation efforts:

Participant: “Without measurement, improvement in care cannot be done.”

Lack of communication and agreement among clinician groups
Lack of communication between clinician groups, particularly between obstetricians and midwives, was identified as a prominent barrier:

Participant: “Communication between the professionals is, is, uh, lacking….”

Participants also discussed the lack of agreement amongst clinicians in terms of which recommendations to adhere to, and which guidelines to implement. One participant indicated that the lack of agreement could be attributed to a generational gap amongst the clinicians whereas another attributed this disassociation as a result of “professionals being educated in different universities”.

Participant: “I think this is a big fight, (laughter) it’s a strike, it’s a fight between the, how you say, new generation of, uh, physicians, gynecologists and the, and the old generation. They used to use, uh, more, more Ergometrine, but, uh, recently, if you can say, last ten years or maybe even fifteen years, we use, we use Oxytocin. I think that maybe compare results, the new age is getting better.”

Participant: “And when it came to collecting and then sharing experience information, they were not able to to agree upon what would be implemented, therefore, the eternal fight in terms of resistance or in implementation or non-implementation of protocols and guidelines continues further.”

However, there was keen interest expressed by participants to continue to engage with each other on education initiatives to improve interprofessional communication.

Lack of training and continuing education opportunities:
Participants felt that there was a lack of training opportunities at the undergraduate level, particularly for midwifery and medical programs. Additionally, participants identified lack of access to continuing education as a barrier experienced by clinicians, including focused education on guideline use and implementation. Participants noted opportunities for integration of such topics into the undergraduate training curriculum as well as for offering specific workshops:

Participant: “I think that there is enough space to, to, uh, to, to introduce this into our regular teaching program, or curricular, regular curricular, or, uh, inside some, uh, trainings and, uh, continual professional developing of the staff, medical workers and co-workers.”

Participant: “And of course, the WHO recommendations on them and then there are training, continuous training of, of different training workshops or perinatal health care, and things like that.”

Health care system level
Fragmented System:
Many participants agreed that the current health care system is fragmented and difficult to navigate following recent conflict experienced in Kosovo. Efforts to rebuild the system and recover losses are slow due to lack of funding. The system is comprised of various groups with different backgrounds, however, one participant suggested that there should only be one ‘team’ and not three separate branches.

Participant: “Immediately after the war, the failure or the decline of the, fall of the system and quite some investment in reactivating or rebuilding the system, a large mistake was done, made at that time. It was fragmented to be easily managed by, by different stakeholders on a, on a wider level….”

Participant: “So despite the best will of stakeholders, this has caused the managerial structures be divided and then the protocols are, are deriving from different backgrounds, different sources. This I put on the table only, only to illustrate that this is a new system being built after a failure of the past one. It is poorly funded and in terms of time, we have lost pace in terms of, uh, developments that have, that have gone through the last ten or fifteen year. So we are still in a recovery stage.”

Participant: “…in a fragmented system, a poorly communicated inside itself, poorly funded obviously and, uh, very much diverse in terms of knowledge and background, I don’t really expect any, any major improvement.”

Participant: “And, you know, which side, I think that, uh, uh, only one team without three, three, I mean, three, uh, how do you say…”

Translator: “Branches.”

Participant: “Three branches, teachers, clinicians, and Ministry of Health, I think, uh, the protocols would not be useful or would not be, uh, efficient.”

Lack of communication between Ministry of Health and clinician organizations:
Participants perceived there to be a lack of support from the Ministry of Health and felt that not enough priority was placed on implementation. As a result, participants perceive that funds are being directed elsewhere and therefore implementation of guidelines is falling behind.

Participant: “In terms of the content of the protocol, that’s very good and, and it, it, uh, it abides by the norms and the standards, but, uh, the Ministry of Health, is, is not even, is not even interested to deal with it. So, support from the state institutions has been missing because the Ministry policies have focused someplace else.”

Participant: “Ministry of Health funds only one protocol on gynecology and that is only on the cervix. And if we wait for, for funding, funding the, uh, different protocols once in three years, then we will never have a set of protocols.”

Participants also identified lack of communication between clinicians and policy-makers at the Ministry of Health as a barrier. Participants described lack of engagement on policy decisions and lack of access to formal and informal communication channels with government as a routine challenge to clinical practice. This also affected their level of awareness about changes at the healthcare systems level. For instance, a participant attending the focus group from the Ministry of Health shared that the Ministry has established a Kosovo Guidelines Advisory Committee to address some of the challenges being discussed, however, most participants were unaware of the Committee’s existence.

Participant: “For this reason there is a, a Kosovo council for, for compiling, drafting, uh, uh, uh, guidelines and protocols, clinical guidelines and protocols, uh, that was, that decision was made last year… Well this council is, uh, consists of 12 members. And, uh, this, this group, this council is mandated with the work of prioritizing, uh, the most urgent, uh, uh, protocols and guidelines and it sets itself an agenda, for example, for a year, what will be the highest priority protocols to be, to be drafted and approved, uh, uh. For example, last year, the council has, had in its agenda to draft and, and compile, uh, twenty protocols. And these protocols, after the, uh, compilation, the drafting, uh, they are subject to an evaluation committee, commission.”

Lack of clarity regarding roles and responsibilities of various stakeholders:
Participants relayed that there is a lack of understanding about the roles and responsibilities of the Ministry of Health versus the professional associations with respect to implementation of guidelines. The confusion around who should be responsible is an obstacle to implementation.

Participant: “Here’s a problem in terms of competency or, uh, division of labour between the Ministry and the association, associations. I believe that the Ministry of Health should manage health policies at the Kosovo level. But the professional associations should deal with the professional side, drafting, formulation of protocols and then monitoring of implementation for its own, its own field.”

Participant: “There were some weaknesses in the admin instruction, in terms of several details in the admin instruction. Of course, the drafting was clearly provided upon, but, uh, there were some details then who does what after the drafting, clear responsibility, division….”

Lack of centralized system to collect and share data:
When asked about their current systems for documenting and accessing health records and health care information, one participant stated that there currently is no such system in place. Specifically, there is no infrastructure to document and monitor clinical practice and quality of care including prescriptions, diagnoses and adverse events amongst others. Consequently, evaluating the impact of guideline implementation on health care and population health status is very challenging:

Participant: “Every information should be written.”

Facilitator: “And does that information then get collected at a central level that you could then access?”

Participant: “Not yet. Not yet.”

Participant: “That’s why there is a need to have a mechanism which measures, uh, the impact and then reports back on, on the outcomes, and then you get to the revision.”

Lack of Resources:
Participants cited limited access to resources such as medications, equipment, staff, and funds for implementation as a key challenge to guideline implementation.

Participant: “Uh, lack of, uh, health insurance means that the institutions are not always, uh, suited in terms of finance and technology and treatment as recommended by a guideline or a protocol. It might not be suited at any level possible, primary, secondary, tertiary, and we might not be able to implement.”

Participant: “The problem is that the associations have a lower number of, uh, of, uh, doctors, physicians. So because of that, there are no means to, to, to implement them.”

Participant: “The first, which is rather painful, I mean the lack of Oxytocin or sufficient medications which is directly related to missing funding or, or lack of funding in, in health care system.”

2. Suggestions for implementation

Participants were asked to provide feedback on how to improve implementation. All participants recommended the need to introduce mechanisms to document and monitor guideline implementation and care delivery. If such mechanisms were established, there would be an increase in their ability to evaluate implementation.

Participant: “The Ministry of Health obviously needs to, uh, to have a mechanism to monitor implementation of protocols since it has been binding.”

Participant: “That’s why there is a need to have a mechanism which measures, uh, the impact and then reports back on, on the outcomes, and then you get to the revision. Uh, we both know because we are at the front of the events.”

Participant: “In terms of assessing the efficiency or the effectiveness of a protocol, that depends largely or entirely on data reporting. If we have this protocol and we have Oxytocins as a first step, and then Ergometrine in a second stage. When you monitor the case, we cannot say that the protocol has not been implemented, because there might not be a first line, there might be, there might be a second line, but the evidence in terms of efficiency or effectiveness of Oxytocins or Ergometrines, that depends on your reporting and the data that you have, and then we can take measure to, to revise the protocol.”

Many participants also felt strongly that standards should be established for clinical care including a monitoring strategy to enhance guideline adherence:

Participant: “So there must be a binding mechanism, or even maybe punitive measures for those who do not implement.”

Alternatively, one participant stated the use of incentives to encourage staff adherence to the guidelines:

Participant: “Maybe it would be more effective if we choose, eh, the other, the other way around. We, uh, instead of using the stick, we use the carrot.”

Participants also suggested that there should be increased opportunities for clinicians and other relevant stakeholders to communicate and share developments in practice:

Participant: “Deviate from the recommendations to be reported by the health professionals and then to discuss them, to, to have groups for some discussion because only by, by identifying such problems the others could benefit from the, uh, from the experience.”

Furthermore, participants proposed that an implementation ‘working group’ consisting of various stakeholders be established. By engaging the appropriate stakeholders, the guideline recommendations would be more feasible and relevant to the target groups and ultimately increase buy in. Additionally, recommendations should be flexible and provide sufficient alternatives so that they can be readily adapted to local settings.

Participant: “The working groups should be composed of, uh, of, uh, the professionals. And then the group is then given, given the responsibility itself. Will it, will it take, uh, uh, some, some basics from the WHO for example, or will they draft its own newly, newly drafted or adapt to different conditions.”

Participant: “The ministry cannot, cannot know what is good and what is bad in my own management. Of course, the association can know that and should know that because we are professionals.”

Participant: “Obviously not every document can be used readily or directly. It has to be suited to our conditions. So it may be considered as adapted to our situation, but then it is formalized or officalized by the Ministry of Health and then it is binding or mandatory on the, on the health practitioners.”

Participant: “So if this has been my recommendation to, for meeting to recommend the working groups to, uh, absorb from the, the evidence based reliable sources, and then see about adapting them, the recommendations, and then follow up, follow them up in terms of revising it later based on the practice in implementation.”

Consensus meeting

The consensus meeting began with a presentation on research in knowledge translation given by Dr. Sharon Straus; additionally, Dr. Metin Gülmezoglu gave a description of the WHO process for developing guidelines. Following these presentations, participants were asked to rate the importance (using a 9-point Likert scale) of the WHO guidelines on prevention and management of PPH. The mean, standard deviation (SD) and range of participant ratings are presented in Appendix 2. Overall, ratings of the guidelines on the prevention of PPHrevealed that participants felt the following recommendations were most important:

  • Recommendation 3 (mean = 8.9, SD = 0.6): In the context of active management of the third stage of labour: Skilled attendants should offer oxytocin for prevention of PPH in preference to oral misoprostol (600 mcg);
  • Recommendation 2a (mean = 8.7, SD = 0.47): In the context of active management of the third stage of labour, if all injectable uterotonic drugs are available: Skilled attendants should offer oxytocin to all women for prevention of PPH in preference to ergometrine/methylergometrine; and
  • Recommendation 1 (mean = 8.6, SD = 0.60): Active management of the third stage of labour should be offered by skilled attendants to all women; the panel does not recommend active management by non-skilled attendants.

When rating the guidelines on management of PPH and retained placenta, participants identified the following recommendations as most important:

  • Recommendation 15b (mean = 8.85, SD = 0.37): Intravenous fluid replacement with isotonic crystalloids should be used in preference to colloids for resuscitation of women with PPH;
  • Recommendation 16 (mean = 8.81, SD = 0.40): Health care facilities should adopt a formal protocol for the management of PPH; and
  • Recommendation 17 (mean = 8.80, SD = 0.41): Health care facilities should adopt a formal protocol for patient referral to a higher level of care.

Participants also rated the importance of each of the barriers identified during the focus groups and these results are presented in Appendix 3. Overall, the barriers that were rated to be most important included:

  • Lack of access to medications (e.g., oxytocin) (mean = 9.00, SD = 0);
  • Lack of integration of health care resources (e.g., medications, equipment, health care personnel) across the system (mean = 8.95, SD = 0.05);
  • Lack of health care/clinical data (e.g., for medication use, hospital diagnoses, complications, etc) (mean = 8.81, SD = 0.15); and
  • Lack of ability to monitor implementation of guidelines (mean = 8.76, SD = 0.18).

Limitations

There are limitations to data collection and analysis. Key limitations were difficulties with language and technology during focus groups and the consensus meeting. A second limitation encountered was discrepancies between facilitator field notes and final transcripts. This issue is likely attributable to language barriers. However, the quality of the aggregate data across methodologies has provided rich insight into the perceived barriers to and suggestions for local implementation of WHO guidelines. It has also been used to inform recommendations to move forward on implementation.

Finally, a limitation specific to the consensus meeting was insufficient time to discuss and rate that importance of each WHO recommendation for the prevention and management of PPH. However, Dr. Metin Gülmezoglu, from the WHO, used his expertise to bring forward the most contentious and relevant recommendations for discussion and rating.

Recommendations and future directions

Based on the results of the focus groups and consensus meeting, four key recommendations for next steps are outlined:

  1. Develop a small working group with local representatives from clinician groups, the Ministry of Health guidelines committee and quality portfolio, clinical or health services researchers, and the WHO to move forward with implementation. The working group could start with the areas that are considered most changeable in the current context. For example, working on education round guidelines in general (i.e.their appraisal and methods used for development) and the PPH guidelines specifically.
  2. Consider offering workshops on guideline development methods including use of GRADE (Guyatt et al., 2008), on appraisal of guidelines using AGREE (NCCMT, 2011), and on guideline adaptation, for representatives from the Ministry of Health and clinical groups. This initiative could initially engage members of the above working group to participate in a ‘train the trainer’ workshop. These group members could then lead workshops for other participants to build national capacity.
  3. Consider engaging some of the local clinicians on the WHO guidelines development group.
  4. Engage those interested in guideline development and implementation from neighbouring countries in the workshop activities and create a ‘virtual’ community of practice to share experiences and avoid duplication of effort.

Appendix 1

Demographic information for focus group and consensus meeting participants
Characteristic n %
Gender
Male 8 42
Female 11 58
Age
< 30 3 16
30–40 2 11
41–50 8 42
51–60 5 26
> 61 1 5
Location of practice
Pristina 17 89
Other 2 11
Type of practice
Community setting 1 5
Hospital setting 8 42
Other 10 53
Profession
Gynecologist/obstetrician (practice and teaching) 7 37
Nursing/midwifery (practice and teaching) 5 26
Governmental organization 4 22
Non-governmental organization (NGO) 1 5
Other 2 11
Years in current profession
1 5
2–5 10 53
5–10 7 37
> 10 1 5
Total n = 19

Appendix 2

Ratings of WHO Guidelines: Prevention of Postpartum Haemorrhage and Management of Postpartum Haemorrhage and Retained Placenta (2007)
Guideline recommendations Mean Standard deviation Range
Prevention of Postpartum Haemorrhage (2007)
Recommendation 1: Active management of the third stage of labour should be offered by skilled attendants to all women; the panel does not recommend active management by non-skilled attendants.

(Quality of evidence: strong recommendation, moderate quality of evidence).

8.60 0.60 (7–9)
Recommendation 2a: In the context of active management of the third stage of labour, if all injectable uterotonic drugs are available: Skilled attendants should offer oxytocin to all women for prevention of PPH in preference to ergometrine/methylergometrine.

(Quality of evidence: Strong recommendation, low quality evidence).

8.70 0.47 (8–9)
Recommendation 2b: If oxytocin is not available: Skilled attendants should offer ergometrine/ methylergometrine or the fixed drug combination of oxytocin and ergometrine to women without hypertension or heart disease for prevention of PPH.

(Quality of evidence: Strong recommendation, low quality evidence).

7.95 1.13 (6–9)
Recommendation 3: In the context of active management of the third stage of labour: Skilled attendants should offer oxytocin for prevention of PPH in preference to oral misoprostol (600 mcg).

(Quality of evidence: Strong recommendation, high quality evidence).

8.90 0.31 (8–9)
Recommendation 4: In the context of active management of the third stage of labour: Skilled attendants should not offer sublingual misoprostol for prevention of PPH in preference to oxytocin; further research is needed to define the role of sublingual misoprostol administration for prevention of PPH.

(Quality of evidence: strong recommendation, very low quality evidence).

8.55 0.94 (6–9)
Recommendation 7: In the absence of active management of the third stage of labour, a uterotonic drug (oxytocin or misoprostol) should be offered by a health worker trained in its use for prevention of PPH.

(Quality of evidence: strong recommendation, moderate quality evidence).

8.40 0.50 (8–9)
Recommendation 9: Given the current evidence for active management includes cord traction, the panel does not recommend any change in the current practice. Further research is needed.

(Quality of evidence: strong recommendation, very low quality evidence).

8.45 0.83 (6–9)
Management of Postpartum Haemorrhage and Retained Placenta (2007)
Recommendation 8: External aortic compression for the treatment of PPH due to uterine atony after vaginal delivery may be offered as a temporizing measure until appropriate care is available.

(Quality of evidence: very low. Strength of recommendation: weak).

5.86 1.80 (3–9)
Recommendation 9: In women who have not responded to treatment with uterotonics, or if uterotonics are not available, intrauterine balloon or condom tamponade may be offered in the treatment of PPH due to uterine atony.

(Quality of evidence: low. Strength of recommendation: weak).

6.24 1.51 (4–9)
Recommendation 15b: Intravenous fluid replacement with isotonic crystalloids should be used in preference to colloids for resuscitation of women with PPH.

(Quality of evidence: low. Strength of recommendation: strong).

8.85 0.37 (8–9)
Recommendation 16: Health care facilities should adopt a formal protocol for the management of PPH.

(Quality of evidence: no formal evidence reviewed; consensus. Strength: strong).

8.81 0.40 (8–9)
Recommendation 17: Health care facilities should adopt a formal protocol for patient referral to a higher level of care.

(Quality of evidence: no formal evidence reviewed; consensus. Strength of recommendation: strong).

8.80 0.41 (8–9)
Recommendation 18: Simulations of PPH treatment may be included in pre-service and in-service training programmes.

(Quality of evidence: no formal evidence reviewed; consensus. Strength of recommendation: weak).

6.52 1.83 (5–9)
Response scale: 1 = extremely unimportant; 5 = neither unimportant or important; 9 = extremely important.

Appendix 3

Barriers identified in focus groups and participant ratings of importance of identified barriers
Barriers Mean Standard deviation Range
Lack of access to medications (e.g. oxytocin). 9.00 0 (9–9)
Lack of integration of health care resources (e.g. medications, equipment, health care personnel) across the system. 8.95 0.05 (8–9)
Lack of health care/clinical data (e.g. for medication use, hospital diagnoses, complications, etc…). 8.81 0.15 (8–9)
Lack of ability to monitor implementation of guidelines. 8.76 0.18 (8–9)
Lack of ability to document and monitor current practice. 8.66 0.32 (7–9)
Lack of communication between professional associations and the Ministry of Health around guideline development. 8.63 0.23 (7–9)
Lack of integration of clinical care across the system. 8.60 0.34 (7–9)
Lack of accountability for guideline implementation. 8.57 0.35 (7–9)
Lack of communication between professional associations and the Ministry of Health around guideline dissemination and implementation. 8.43 0.35 (7–9)
Lack of between professional associations and the Ministry of Health around data required for monitoring guideline implementation. 8.37 0.63 (6–9)
Lack of agreement with recommendations from guidelines. 8.32 2.42 (2–9)
Lack of communication across health care provider groups (e.g.physicians, nurses, midwives). 8.26 2.89 (2–9)
Lack of process for identifying local opinion leaders/champions. 8.19 0.63 (7–9)
Lack of continuing education/continuing professional development on guideline use and implementation (capacity building). 8.10 0.89 (5–9)
Lack of a process for prioritising guidelines and recommendations for implementation. 8.09 0.56 (7–9)
Lack of undergraduate training (for medical and midwifery students) on guideline use and implementation. 7.85 0.93 (6–9)
Lack of clarity on who is responsible for implementing guidelines (e.g. Ministry of Health and/or professional associations). 7.84 2.17 (5–9)
Lack of awareness of the protocols/recommendations from the guideline. 7.70 5.31 (1–9)
Lack of awareness of the significance of the clinical problem. 7.21 3.43 (2–9)
Response scale: 1 = extremely unimportant; 5 = neither unimportant or important; 9 = extremely important.

References

  1. Fitch K, Bernstein SJ, Aguilar MD, Burnand B, LaCalle JR, Lázaro P, van het Loo M, McDonnell J, Vader JP, Kahan JP. The RAND/UCLA Appropriateness Method User’s Manual. RAND Corporation; 2001.http://www.rand.org/pubs/monograph_reports/MR1269.html.
  2. Guyatt GH, Oxman AD, Vist GE, Kunz R, Falck-Ytter Y, Alonso-Coello P, Schünemann HJ; GRADEWorking Group. GRADE: an emerging consensus on rating quality of evidence and strength of recommendations. BMJ. 2008 Apr 26;336(7650):924-6. doi: 10.1136/bmj.39489.470347.AD; PubMedPMID: 18436948; PubMed Central PMCID: PMC2335261; http://www.bmj.com/content/336/7650/924.
  3. Jones J, Hunter D. Consensus methods for medical and health services research. BMJ. 1995 Aug 5;311(7001):376-80. Review. PubMed PMID: 7640549; PubMed Central PMCID: PMC2550437.
  4. National Collaborating Centre for Methods and Tools. Critically appraising practice guidelines: TheAGREE II instrument. Hamilton, Ontario: McMaster University. (Updated 28 November, 2011). Retrieved from http://www.nccmt.ca/registry/view/eng/100.html.