Introduction

A quality management system (QMS) requires meaningful and measurable parameters to be established and documented. Specific parameters such as key performance indicators (KPIs), may be measured, recorded and assessed as an invaluable component of total quality management. As such, KPIs may be used to monitor both clinic and individual performance.

However, it is necessary to have universally accepted externally referenced benchmarks against which to assess each clinic or individual’s competencies and performance. Hence, the main purpose of this article is to review benchmarking of staff and clinic performance.

Performance Indicators

The Vienna consensus also concluded that there are five performance indicators (PIs) that might reflect the quality of sperm preparation and insemination (Figure 2). It is important to appreciate that with sperm preparation, sperm motility is a more reliable PI than sperm recovery due to the variability in sperm preparation methods.

Furthermore, blastocyst quality on day 5 (D5) of development partly depends upon the quality of the paternal genome whereas early cleavage embryo development primarily reflects the maternal genome, embryonic genome activation not occurring until around day 3 (D3) of development in the human. The proportion of monopronucleate (1PN) oocytes following ICSI is expected to be lower than following IVF since MII oocytes are selected for ICSI whereas no such selection occurs prior to IVF.

In addition to the five PIs listed in Figure 2, it might also be desired to benchmark competency in semen analysis, 5-10% variation from the mean sperm concentration, motility and morphology being acceptable. Also, sperm cryopreservation could be another PI, >50% of the pre-freeze motility being deemed a generally accepted competency benchmark for slow freezing.

Key Performance Indicators

The Vienna consensus concluded that there are 12 KPIs that might be considered integral to a QMS since they encompass the most important steps within the IVF lab process (Figure 3).

Since there is a tendency among some practitioners to include triploid (three pronucleate; 3PN) and polyploid (four or more pronuclei; ≥ 4PN) zygotes in their calculation of fertilization rates, it is important to note that the recommended KPI for fertilization rate only includes those zygotes displaying two pronuclei (2PN) and two polar bodies (2PB). Since the number of mature oocytes is unknown at the time of insemination by IVF, it is the long held belief of this author that the only way to scientifically and accurately compare IVF and ICSI fertilization rates is for the denominator to be the same for both, in which case this should be the number of COCs retrieved, and that belief is reflected in the same competency and benchmark values provided in Figure 3, which is an amendment from the values recommended by the Vienna consensus.

In much the same way that reporting cycle outcomes per cycle stimulated more faithfully reflects the overall quality of cycle management, reporting normal fertilization rate (number of 2PN + 2PB zygotes) per COC retrieved more faithfully reflects the overall quality of ovarian stimulation, oocyte handling (eg oocytes may be compromised or lost during denudation prior to ICSI but would not be included in assessment of fertilization rate when expressed per MII oocyte injected) and insemination, whether that be via IVF or ICSI. Although insemination procedures may differ, the initial number of COCs and final number of normally fertilized oocytes are equally relevant and it is important to be able to compare this KPI properly using what should be the same benchmark for both IVF and ICSI.

Previously, various KPIs for cryopreservation, including competencies and benchmarks, have been recommended.3 In this respect, it is important to differentiate between cryosurvival and viability.

The cryosurvival and viability of cleavage stage embryos can be determined by the number of intact blastomeres and number of thawed/warmed embryos developing on to the blastocyst stage, respectively. With blastocysts, however, determination of cryosurvival is less straightforward and viability can only be indirectly assessed via the timing and extent of blastocoele reexpansion, and only in those blastocysts exhibiting a collapsed blastocoele on thawing/warming.

Summary

There are three different types of indicator, RI, PI and KPI that may be used to monitor and assess the competency and performance of staff and organizations. The recently published list of LPIs provides a useful checklist for the purposes of lab audits and troubleshooting.2

References

  • ESHRE Guideline Group on Good Practice in IVF Labs. Revised guidelines for good practice in IVF laboratories (2015). Hum Reprod, 2016, 31: 685-686 (full report in supplementary data).
  • ESHRE Special Interest Group of Embryology and Alpha Scientists in Reproductive Medicine, The Vienna consensus: report of an expert meeting on the development of ART laboratory performance indicators. Reprod BioMed Online, 2017 (in press).
  • Alpha Scientists in Reproductive Medicine. The Alpha consensus meeting on cryopreservation key performance indicators and benchmarks: proceedings of an expert meeting. Reprod BioMed Online, 2012, 25, 146-167.
  • Reprinted from Reprod BioMed Online, Vol 35, Issue 5, ESHRE Special Interest Group of Embryology and Alpha Scientists in Reproductive Medicine, The Vienna consensus: report of an expert meeting on the development of ART laboratory performance indicators, 494-510, Copyright (2017), with permission from Elsevier.

Addendum

Various terms are used in benchmarking, which can be confusing, so it helps to understand their meaning, which is as follows:

Benchmark: Externally referenced result expected under optimal conditions.
Benchmarking: Comparison of results against internationally recognized gold standards.
Clinical performance indicator (CPI): Parameters used to monitor and assess clinical processes.
Competency: Minimum standard required to achieve proficiency and ongoing performance.
Competitive benchmarking: Comparison of results against local or national competitors.
Functional benchmarking: Comparison of processes against internationally recognized gold standard processes.
Internal benchmarking: Comparison of results between units within a related group.
Key performance indicator (KPI): Parameter used to monitor and assess crucial steps in a process.
Laboratory performance indicator (LPI): Parameter used to monitor and assess lab processes.
Laboratory performance measure (LPM): Synonymous with LPI.

Performance indicator (PI): Parameter used to monitor and assess steps in a process.
Program performance indicator (PPI): Parameter used to monitor and assess the overall organization’s processes.
Quality assurance (QA): All the systematic activities implemented within the quality system that can be demonstrated to provide confidence that a product or service will fulfil requirements for quality.
Quality control (QC): The operational techniques and activities used to fulfil requirements for quality.
Quality improvement (QI): Actively progressive increased effectiveness and efficiency within a quality system.
Quality management: All activities that determine the quality policy and its implementation via quality planning, QC, QA and QI within a quality system.
Quality management system (QMS): Quality policies, objectives, instructions and procedures.
Reference indicator (RI): Parameter used to indirectly monitor and assess steps in a process.
Total quality management (TQM): Quality management that embraces all staff, their patients, their suppliers and society at large.

Stephen Fleming CooperSurgical Fertility Training labsSteven Fleming PhD is an Honorary Associate at the University of Sydney and Director of Embryology for ORIGIO. Previously, he was the founding Scientific Director of Assisted Conception Australia, and a Senior Fellow at the University of Queensland.

After completing his doctorate in 1987, he undertook postdoctoral research at the Royal North Shore Hospital in Sydney, and from 1993-1997 he was appointed Lecturer in Obstetrics and Gynaecology at the University of Nottingham in England, where he established the world’s first Master’s degree in Assisted Reproduction Technology with Simon Fishel PhD.

From 1998-2008 Steven was the Scientific Director of Westmead Fertility Centre in Sydney and appointed Senior Lecturer in Obstetrics and Gynecology at the University of Sydney. He is the recipient of numerous research grants, and an author and editor of several books, book chapters and peer reviewed journal articles. Steven’s research interests include cryopreservation, endometrial physiology, endometriosis and oocyte maturation, among others.