Managing Quality in the ART Laboratory

In this webinar, David Morroll, PhD introduces the concepts of Quality Assurance and Quality Control in the context of the ART laboratory, drawing on international guidelines. Factors contributing to an effective quality management system are highlighted, including staffing, competencies, traceability, and managing risks and adverse incidents.

The structure of the session is as follows:

• What is quality management?
• Total quality management in ART
• Summary

What do we mean by quality management in IVF laboratories?

Definitions vary a little but the following is taken from the website of the Association of Project Managers and it’s a good example, I believe. It states that:

“Quality management is a discipline for ensuring that outputs or services and the processes by which these are delivered meet stakeholders’ requirements and a fit for purpose.”

It also suggests that quality management has four components;

• Quality planning, which describes the processes and metrics that will be used and should be agreed with relevant stakeholders to ensure that their expectations for quality are correctly identified.
• Quality assurance, which is the overarching approach that provides confidence that processes and services are being well-managed, are consistent or standardized and are performed by personnel with the right skills, attitudes and resources.
• Quality control consists of inspection testing and measurement that confirms that outputs actually meet specification, are fit for purpose and meet stakeholder expectations. In other words, quality control tests weather acceptance criteria have – or, indeed, have not – been met.

The overall aim of quality management is to first of all ensure that specifications and standards are met but, just as importantly, it is to seek areas in which the laboratory can improve.

Shown here is the quality cycle whereby one measures the performance of the systems in use and then targets where improvements can potentially be made. Again, you can see variations on this theme but, in essence, the four actions are:

• Plan: Here, the possible opportunities for change are first identified and the changes planned and agreed
• Do: This is the part of the process in which the tested changes that have been agreed are then piloted and data collected to measure the effect of those changes
• Check: Here, data analysis and review is undertaken which reflects on the impact of the tested changes and then decides whether those changes were positive, neutral or, indeed, negative.
• Act: This is the implementation of changes if they’re deemed useful and positive.

Thereafter, the cycle would simply begin again, driving the process of gradual and continual improvement.

Of course, quality management can be assessed more formally by application of standards such as those published by the International Organization for Standardization (ISO) and I’ve listed some pertinent ones for IVF labs here:

Though it’s unlikely that an IVF lab would seek certification for ISO 17025, 31000 or 14644, there are some relevant sections that would pertain to an IVF setting.

It’s much more likely, however, that a laboratory would have certification to ISO 9001 (as a general standard for quality management systems that might be applied across the whole unit) or to ISO 15189, which is specific to medical laboratories.

ISO 15189 is currently under review, but the 2012 version is still applicable. As indicated, it deals with requirements for quality and competence across medical laboratories so it’s not specific to ART or, indeed, any particular lab discipline. And whilst, perhaps, not readily interpreted for medically assisted reproduction, the general aims of confirming and recognizing competence for lab users, regulators and accrediting bodies can be useful and, indeed, has been implemented across some IVF centres.

More commonly, perhaps, ISO 9001 can be applied to the IVF centres since it’s a universal standard and the requirements are generic and applicable to all types of organizations. This standard assesses the quality management system and shows that the organization meets the needs of users in a consistent way. User satisfaction is evaluated, as are the processes for continual improvement.

We know that this has been successfully introduced and, indeed, Alper and colleagues reported in 2013 how they used ISO quality control and the 9001 standards to contribute to developing their service.

Total quality management in a statistic conception

I’m going to use the ESHRE revised guidelines for good practice as a helpful starting point. I know these standards and these guidelines may not be globally applicable but they’re a useful starting point that will guide us through how quality management impacts on the IVF laboratory, in particular.

I’ve highlighted here some of the key elements of the guidelines that pertain to quality management and I’ve highlighted in blue some particular keywords that we’ll revisit during the progress through this presentation.

By way of background, it’s interesting that the ESHRE guidelines recommend that a clinical embryologist is responsible for quality management within the laboratory, though, of course, the overall quality manager would not necessarily be a laboratory person.

The guidelines cover a wide range of relevant topics including databases that allow KPI extraction and statistical analysis for risk assessments and preventive actions, for systems, for non-compliance, emergencies, errors, adverse events, near-misses and complaints as well as the choosing of objective and relevant KPIs and participation in internal quality control and external quality assurance. That’s backed up, of course, by the annual review of the quality management system and audits; both internal and external.

But the keywords highlighted here (responsibilities, competencies, SOPs and so on) we’ll now go through in order to highlight quality management in assisted conception laboratories.

Responsibilities for quality management in IVF laboratories

The first word refers to defining responsibilities. This is a key part of quality management; people must understand their role within the wider organization and know a.) to whom they report and b.) who reports to them.

This is an old example from my work previously. This organizational chart supports the structure needed for effective decision making and also dealing with adverse events, depending on the nature, scale and impact. In addition, each person should have an up-to-date job description which sets out what is expected of them, what they can do and maybe even what they are not able to do since this defines their scope of practice. No-one should be asked to perform any activity in which they’re not competent and have not been adequately trained. Of course, it’s also up to the individual to make clear that they cannot undertake activities when they have not received training.

Assessing competencies for IVF laboratories quality management

In terms of training or ensuring competence, as stated in the ESHRE guidelines, it’s essential training is fit-for-purpose and is fully recorded.

This slide, which takes information from the ESHRE website, shows the minimum number of procedures required to achieve certification with ESHRE as an embryologist. Now, this isn’t necessarily definitive; some people may well need more cases to demonstrate that they can work independently without full supervision, but it’s certainly a helpful guide. The key here is that all training needs to be recorded in a suitable logbook and then can be used as evidence that a degree of competence has been built up.

In their certification of clinical embryologists ESHRE also tackles the important issue of staffing levels and we should stress from the outset that the level of staffing within a laboratory is absolutely an issue of quality. It’s clear that embryologists should not work alone; so at least two qualified embryologists are recommended for labs undertaking up to 150 cases per year. The number of additional embryologists beyond that will depend on the types and numbers of treatments performed, though larger organizations may not depend solely on embryologists so they may use technical staff, andrologists, admin support and so on. There should also be sufficient personnel to cover leave and sickness with backup options. The underpinning principle here is that patient safety and quality of care are paramount, so I’ll reiterate again that staffing levels are an essential element of quality management.

The issue is not one solely tackled by ESHRE; there are references to this across a number of organizations. This is just another example taken from the ASRM laboratory guidelines and which again shows the need for a minimum of two embryologists in a laboratory with increasing numbers as the workload gets heavier.

It’s tricky to recommend absolutely exact numbers by number of cases, egg-retrieval cycles or events since some cycles are more complex and so more time-consuming than others. Just think of a standard ICSI compared with a non-obstructive azoospermia case with surgical sperm recovery on the day of egg collection. Many of us would have spent hours looking for sperm in such cases. Consider too, how many more man-hours might be needed with PGT cases. As the take-up of PGT and IVF centres continues on an upward trend, this adds disproportionately to the workload. This was addressed by Alikani, M. et al. back in 2014, where they looked at how extended culture, biopsy, tubing, vitrification, possible warming, re-biopsy, hatching and so on all contribute to the demands placed on staff and facilities. That added complexity means more man-hours and therefore should be reflected in the numbers of staff that are provided to deliver that caseload.

Standard Operating Procedures (SOPs) for quality management in IVF laboratories

The procedures we use in the lab must be supported with written Standard Operating Procedures and these must be version-controlled, appropriately authorized and restricted so that only current versions are allowed. Of course, the SOPs must be written so they’re compliant with local standards, directives, legislation and so on. It should be completely clear exactly what the SOP covers. Most importantly the lab must have procedures in place to demonstrate that everyone uses that SOP appropriately, otherwise any variations in outcomes from the lab can’t be traced back to a particular way of doing things or a particular procedure. It’s also essential the SOP procedure is validated. But what does that mean?

The validation of the SOP should record equipment and consumables used and indicate which are most critical, so in this example (right) of an IVF insemination the red text indicates those having a direct impact on gametes and embryos.

In our system that we also listed exactly what data or information could be used to support process validation so, in this example, IVF insemination could use scientific literature, fertilization rates, polyspermy rate, cleavage rate and any effects of timing.

And our validation form had our historical data demonstrating that an acceptable and consistent level of performance was being achieved using that exact Standard Operating Procedure.

Even when we have tightly control procedures it’s possible for things to go wrong and quality management systems incorporate a strong theme of learning from mistakes. The first thing to say is that all complaints, near-misses and adverse incidents must first be recorded. Secondly, they should be addressed, and any corrective actions taken – those corrective actions should also be recorded and disseminated. And, thirdly, an annual review should be undertaken to look for recurring themes and trends so that those can then be addressed. It’s important to clearly display a complaints policy and give information about how complaints can be made and how those complaints can be submitted to management for the clinic. Remember, if someone has a complaint, or wants to offer constructive feedback, it’s a better, more productive approach to facilitate this and engage in a way that addresses the issues, otherwise the complainant will just head off to the nearest competitor. Incidents or adverse events can occur across all functions of the ART Centre, whether administrative, communicative, processing, identification, technical or traceability. Some are avoidable, some are unavoidable but the ones we need to take most awareness of is the serious adverse events, which is any untoward occurrence associated with procurement, testing, processing, storage and distribution of tissues and cells that would lead potentially to transmission of a disease, to death or life-threatening disabling or incapacitating conditions for patients, or which might result in prolonged hospitalization or morbidity. I guess in an IVF setting we would also perhaps add loss of embryos or transfer to the wrong patient.

At this point it’s worth reminding ourselves that to err is human; we are fallible, and mistakes can and do occur, but we must be constantly mindful of that fact. You can work out, in your own system, just how many times gametes and embryos are handled, moved between dishes, assessed and so on in a typical cycle. One study, Thornhill A et al., reported 21,500 witnessing steps across 1,757 cycles; that’s an average of just over twelve witnessing steps in a normal cycle. They used RI Witness to calculate a potential error rate of 0.11%, which is equivalent to one possible mismatch every 900 cycles or so. Now, if you take the 2016 SART data as a typical example, the average number of cycles per year/per center  is around 570 so, with an error rate of 0.11%, there’s a potential mismatch (with the wrong gametes or embryos being used) every 18 months! Fortunately, most potential errors are corrected but we can’t be complacent; we need to think about how we control that sort of risk.

Looking at the Netherlands, which publishes excellent data on errors and adverse events, there are typically a couple of events annually in which the wrong gametes or embryos are transferred back into a patient. There is absolutely no reason to assume the systems here or any less robust than elsewhere so we must assume that these events are happening on a relatively uncommon but regular basis.

Indeed, if we look at the collated data for across the European Union there were 145 serious adverse events reported that involve reproductive cells or tissues in 2017. These occur primarily during processing, testing or storage and they’re mainly due to human error and occasionally tissue or cell defects. The downside of this report is that not all the data is reported in a consistent way, so let’s look as well at a single country – the UK.

The Human Fertilization and Embryology Authority requires all licensed centres to report adverse incidents and near misses within a stipulated and short time frame and this throws up some interesting details. First of all, there are more incidents that relate to clinical management or administrative errors than laboratory incidents. These may be, for example, breaches of confidentiality or cases of hyperstimulation. But, unfortunately, problems in the lab tend to be far more serious. Thankfully, the incidence of most serious events is very low; for the Grade A (the worst cases in the HFEA grading system) these were rare. 0.2% of the errors were Grade A in the period covering 2016-17 and none were reported in the period of 2017-18. This might, in part, be due to the stringent requirements for witnessing laid out as a condition of licensing of an IVF laboratory in the HFEA’s code of practice.

Risk management is a key facet of quality management; identification of all risks and establishing ways of reducing or eliminating those risks is essential. The most serious errors in IVF labs are mix-ups of gametes or embryos so a robust chain of custody from sample procurement at egg collection or the man’s sample production through to embryo transfer or storage is a minimum requirement. This can be achieved by implementing a witnessing system that ensures that each time samples are moved between pots, dishes or tubes it is witnessed and that witnessing is appropriately recorded. It’s possible to do this using manual witnessing. Manual witnessing is a better system than no witnessing at all, of course, but it’s still prone to errors and it’s been shown to be significantly less effective than electronic systems. Simply, machines don’t get bored, tired or distracted. Electronic systems can use either RFID tags or barcodes and can also introduce gateways and stopping functions that warn of a possible error. It may still be necessary to perform manual witnessing of key steps, particularly entry and exit point – the egg collection, sample collection and an embryo transfer already mentioned. However, the added reliability, and reduced disturbances to co-workers being called over to witness are both crucial advantages. It’s also worth highlighting that the proximity needed for manual witnessing is something we may be trying to avoid in new circumstances, post COVID-19.

Does electronic witnessing in IVF actually work?

This paper by Laura Rienzi and colleagues used a Failure Mode and Effects Analysis to Assess Risks Before and After the introduction of RI Witness. The authors highlighted workload, staffing levels, and disturbances such as telephones to be major causes of potential errors and, of course, I think we can say that many IVF labs are still prone to those issues. However, the risk of failures at vulnerable steps in the ART processes were mitigated once electronic witnessing was introduced. As the authors themselves indicated, the severity of the consequences of any mistake can’t be reduced but the likelihood of an error was significantly reduced.

Another element of quality management is the ability to know, confidently, whose gametes or embryos are being handled, which media, oil, plasticware, pipettes or catheters are being used, which incubator the embryos were kept in, and when – and by whom – each procedure was performed. This level of traceability (and I’ve used the UK HFEA definition as a good example here) is essential so the impact of each part of the system can be determined and, where appropriate, corrected.

We’ve mentioned witnessing systems, but I prefer to think of them as ART management systems because, whilst they’re a useful tool for error prevention, they also very useful, potentially, in traceability. By recording steps in the process it’s possible to keep an electronic record of not only who does what, when, and where but also keep information on batches of consumables, when and how they’re used, and the levels of stock and inventory. That’s made possible because the system relies on the lab’s process map. The process map stipulates the order in which steps can be done so the system records what step is being performed in the right order. It records in which workstation it’s being used and, as it has individual logins, it records who’s doing that procedure. By registering products into the system it’s possible to record what is received from suppliers, which batch is in use and which are due to be used next, and which will expire soon. All the batch numbers can then be assigned to each patient’s treatment records. Because the system knows who is doing what, for how long, and where, the analytics can help with traceability as well as managing performance.

Testing of consumables in IVF laboratories for quality management

ESHRE guidelines also refer to the quality testing of consumables. In the vast majority of cases this is addressed by the manufacturers, for example CooperSurgical makes its certifications and licenses public on its website and invests in state-of-the-art production facilities. This is intended to guarantee a high-quality product that ensures good performance and patient safety, which you’ll see summarized in your Certificate of Analysis.

On the flip-side, please be aware that, if you choose to manufacture (such as making your own media or vitrification devices) you should really be providing a similar level of testing. This may also apply to altering a product, unless specified in the manufacturer’s instructions for use.

Quality management of equipment in IVF labs

The guidelines cover layout and ergonomics. But in terms of quality management, the critical considerations are that it’s validated, calibrated and maintained and, importantly, that there is enough of it to facilitate the workload anticipated. Just like having enough staff, IVF labs must have enough incubators, for example. It’s also important that staff know how to use the equipment they’re provided with and that critical equipment is continuously monitored and alarmed. It’s also worth mentioning here that quality management would require the lab to have well-documented and communicated contingency plans for incidents of equipment failure.

As part of the risk management the alarms are a minimum requirement for incubators, storage dewars and I would suggest media fridges. Better still, a monitoring or surveillance system is a more robust system and performs the required regular checks of equipment performance. This can be done manually but it’s much more time-consuming and labour-intensive. The advantage of a monitoring system is that it will also allow you to assess performance of individual pieces of equipment and then perhaps think about how that piece of equipment is used – whether it’s overused or underused – and whether process workflow can be modified to use the facilities provided in the optimal way.

Looking at a little bit more detail, new equipment can be subject to a User Requirement Specification which sets out exactly what it should do and detail any specific issues such as size, capacity and voltage that relates specifically to where it’s being used. Every piece of equipment needs to be validated, demonstrating that it does what it should. You would do this, typically, at installation (“Installation Qualification” / IQ). You would repeat it once it’s set up and operating (“Operating Qualification”/OQ) and then for a period of routine-use Performance Qualification (PQ). This would then be reviewed periodically with an Equipment Qualification Review (EQR). It’s important to ensure that equipment is properly maintained, serviced and calibrated and, of course, this has to be recorded. Typically, this servicing, date etc would be recorded on the instrument itself so it’s clearly visible.

The EQR – the Equipment Qualification Review – will be updated periodically to show the instrument is still fit for purpose but further full validations would typically be performed if the instrument is repaired or modified in any way.

Verifying conformance in IVF laboratories for quality management

We now move on to checking that what we said we would do, we do, and we do it to a good standard. Verifying conformance ensures just this; it records compliance with guidelines, regulations and standards as well as checking that one meets in-house targets. After establishing targets it’s necessary to record sufficient data to be able to confidently analyse performance and show compliance. These data are then reviewed as a team.

Performance Reviews for quality management in IVF labs

My colleague Martine Nijs talks more about monitoring performance in her webinar ‘Evaluation of IVF laboratory outcomes with key performance indicators’ so I’ll simply say at this point that one needs to record data and analyse KPIs to judge how the lab’s performing. But we can extend this to checking satisfaction of service users, looking at incidents and complaints, checking training records and perhaps monitoring research output. Of course, this can be done at various levels; groups of clinics, individual labs or, indeed, individual operator (whether embryologist, doctor, nurse or administrator).

Quality Control in IVF labs

To understand lab performance, quality control forms a key part. This is the inspection, testing and measurement parameters that ensure performance meet expectations. We mentioned that at the outset, but internal quality control should be used in a way that answers particular questions about performance, so that it’s truly reflecting laboratory performance and not variations in the patient population, for example. Added to iQC (Internal Quality Control), EQA (External Quality Assurance) primarily allows for inter-laboratory comparisons. This is a scheme that allows you to submit data that shows how you would make decisions [and] how you would assess things, and then compare it with people around the world or within the country. there are a number of schemes available for andrology and embryology.

Risk Assessment Analysis and Quality Management in IVF laboratories

We’ve talked about risk assessment in a number of areas but, specifically, it’s mentioned in the ESHRE guidelines. It’s important to remember it should cover all processes and all areas and it’s intended to determine the appropriate response to any risk identified. The response will depend on the type of risk and it will depend whether it can be eliminated, reduced or managed, or just accepted (were no action is really possible that will reduce the risk).

I’ve shown here a typical cycle, together with a risk assessment matrix that assesses the severity of something happening against the likelihood of something happening to give an overall score of the level of risk involved. That’s a fairly typical approach to a risk assessment.

Summary

We should all be striving to optimize the skills, processes and facilities at our disposal to optimize outcomes for patients. To back this up, maintain quality and drive improvements, a robust system of audits and evaluation is required, and these audits can be at all levels whether of people and processes. We should foster a culture that encourages complaints and feedback as a learning tool, and we should use internal Quality Control and External Quality Assurance proactively to monitor and improve the way a lab performs. Finally, we should be evaluating performance at all levels and interrogating on a regular basis.

About the Presenter: David Morroll PhD

David has worked as a Clinical Embryologist since 1986, training in Manchester, where he also completed his doctorate studies. He has since managed laboratories in a number of UK IVF units including those in Nottingham, London and Leeds, and consulted to a number of clinics overseas. He joined ORIGIO as Director of Embryology in November 2011.

He has previously served as Chair of the UK Association of Clinical Embryologists (ACE) and Association of Biomedical Andrologists (ABA), and was involved with several working groups, notably the HFEA Expert Group on Multiple Births after IVF.

David has a keen interest in quality management, including ISO certification and External Quality Assurance. He has also consulted to IVF units in Malta, India, the Philippines and Zimbabwe.  In January 2019, David was appointed to the role of Director of Clinical Support, focusing on post-marketing clinical studies, QA/complaints and laboratory audits.

Find out more:

The following are mentioned throughout the text above and/or are particularly relevant to it. To find out more about each item, please click the link.

Reference:

• ESHRE revised guidelines for good practice
• International Organization for Standardization (ISO)
• 2016 SART data
• CooperSurgical’s certifications and licenses
• Alper et al. Experience with ISO quality control in assisted reproductive technology 2013
• Alikani, M et al. Comprehensive evaluation of contemporary assisted reproduction technology laboratory operations to determine staffing levels that promote patient safety and quality care 2014
• Thornhill et al. Measuring human error in the IVF laboratory using an electronic witnessing system 2013
• Rienzi et al. Failure Mode and Effects Analysis to Assess Risks Before and After the introduction of RI Witness 2015
• Webinar: ‘Evaluation of IVF laboratory outcomes with key performance indicators’

Products:

• RI Witness