4 Steps To Implementing a QC Program For Your Flow Cytometry Experiments

With the emphasis on reproducibility in science, it is important to look at the process of quality control (QC) and quality assurance and how these can help improve the reproducibility of data.

First, what is the difference between quality control (QC) and quality assurance (QA)? For the purposes of this article, we will define QC as the activities that are performed to ensure that the product, in this case, flow cytometry data, meets the standard. QA, on the other hand, is focused on how the processes used to develop the product and how these can be improved as well as establishing methods that prevent poor quality product.

A lot of time is spent talking about QC, especially in regards to how the instrumentation performs. Less time is focused on QA. Looking at these two processes in the context of a flow cytometry experiment, QA is focused on the development of the panel, while QC is focused on how well and consistently the panel performs once it is put into production. This blog will focus on one manner of implementing a flow cytometry panel and how to QA and QC should be integrated into the process.

1. Start at the end:

To effectively develop a flow cytometry panel, it is critical to know what the biological question is. This, in turn, will lead to an understanding of what the panel will be designed to prove or refute. The way that this is done in hypothesis-driven science is to perform a statistical test to determine if the data supports refuting the null hypothesis (H0). This requires the investigator to determine the type of data that will be extracted from the experiment and how it will be analyzed.The statistical analysis plan, which is developed before the panel is developed, drives the rest of the process. This should not be left to the end to prevent the temptation of HARKing or p-Hacking.

2. Build the panel:

This topic has been given lots of attention in other blogs, such as the one linked above. Briefly, there are three considerations to balance:

  1. Target antigenClearly, it is important to use the correct antibody targeting the antigen of interest. This can be tricky since there can be many different clones targeting the same antigen. Websites such as the Human CD Antigen Table, BenchSci.com, the Antibody Registry andIn this analysis of which CD3 clone to use, two sources were used – BenchSci.com and the OMIPs as published in Cytometry A.Figure 1: As this table shows, there is no specific consensus as to which CD3 clone to use. Thus checking the primary literature is critical. Furthermore, as was pointed out by Bradbury and Plückthun, not all antibodies work well. This can be due to a variety of issues, so navigating which vendors regent to purchase can also be a test. Antibodypedia is a great resources to use to get an idea of what clone should be used. Finally, the expression of the target needs to be considered. This can be divided into three levels – low/unknown, medium and high. Primary lineage markers are in the high category. The medium expression category includes those markers that are used to refine the subset of interest and the low/unknown are those markers with either low or unknown expression. These are the targets of interest.
  2. Fluorochrome brightness Next is to consider the brightness of a given fluorochrome. This can be measured by the Staining Index. Briefly, this compares the distance between the positive and negative signals, which is divided by two times that standard deviation of the negative.Figure 2: Measuring the Staining Index. This data can be generated on the investigator’s instrument, or it is possible to turn to one of the published tables like this one at BioLegend or this one from BD Biosciences. These can provide a guideline to relative brightness, but this could be impacted on the investigator’s instrument due to different laser power, laser lines or filters.
  3. Instrument sensitivity: The third part of the panel design triangle is to identify where the instrument is most sensitive. While there can be several ways to measure sensitivity, the use of the SSM or spectral spillover matrix. This allows for the assessment and ranking of both fluorochromes and detectors.

Figure 3: SSM for an instrument. Across the top are the fluorochromes and down the side are the detectors. Areas that are high (red) are areas of potential concern.

Figure 4: Ranking the detectors that receive the most error and fluorochromes that contribute the most error in a polychromatic panel.

With these three pieces of information, the researcher can go on trying to develop the panel. This requires a bit of searching the databases for available fluorochrome/antibody combinations. This can be automated with one of the many tools out there for this, which range from vendor specific such as: Biolegend, Bio-Rad, and Thermofisher. Alternatively, the use of Fluorofinder, or Chromocyte, both of which are vendor independent.

3. Optimize the panel:

With the panel developed, now comes the optimization and validation steps. This includes titration, voltration, and panel testing. In this optimization phase, the necessary controls for identifying the populations of interest are also identified. This is also where Standard Operating Procedures are developed and approved, critical tools to ensure the consistent processing of the samples and experiment.

  1. Titration:The goal of titration is to identify the best concentration for the assay being performed. Each antibody to be used needs to be optimized. Cells are labeled with serial dilutions of the antibody and the staining index is calculated. The concentration which yields the best signal to noise (as calculated by the staining index) is the one that should be used. Don’t forget that the antibody should be titrated as it will be used. So if your samples will be fixed, so should the titration samples.
  2. Voltration:Determining optimal voltage can be achieved in several ways. The peak 2 method has been discussed before. The optimal voltage can be further refined by performing a a voltage optimization (voltration) experiment. In this experiment, cells are labeled with the titrated amount of antibody and run on the instrument starting at the recommended voltage and increasing the voltage till the positive events are out of the linear dynamic range of the detector. The staining index is plotted and data looks something like this.Figure 5: Voltration experiment.As can be seen from this data, as the voltage is increased for the Cy7-APC detector, the signal quickly falls out of the linear range of the detector, and off-scale. Looking at the curve, the peak 2 recommended voltage is best for this fluorochrome. The BV650 detector, on the other hand, shows an increasing improvement in the SI at 700 volts, resulting in an increase in the SI of about 15%. Thus this becomes the recommended voltage.Once these voltages are established, it is important to monitor them over time. To do this, use a relatively bright bead, like the 6th peak of the 8-peak bead set (or your other favorite bead), and run at these voltages to establish the MFI value for each detector. From there you can set up a template, like the one shown in Figure 6. Figure 6: QC tracking template. When sitting down to run the instrument, open up the template, and check the voltages on the system so that the bead is hitting the target threshold. Once that is set up, run the experiment. This is a level of QC that the investigator can add to the experimental workflow that helps to improve the quality and consistency of the date.
  3. Optimization:The optimization process is the final step before the panel is put into use. This consists of a series of tests to identify the optimal conditions to run the experiment. After titration and voltration, the optimization should include the sample preparation process, running the experiment on the flow cytometer and the data analysis workflow.This is where standard operating procedures (SOPs) are developed and refined. These SOPs provide detailed directions on how to run the experiment, from beginning to end. The SOP helps ensure the experiment is run in the correct manner each time, and each person who will be performing the experiment should be trained on the SOP. Changes to SOPs should not be taken lightly, and it is strongly recommended there is a change process developed in case the SOP must be changed.Additionally, during this optimization process, various controls should be run to determine how useful they are in identifying the population of interest. This includes FMO controls, unstimulated and stimulated controls, unstained controls and more. This is also a place to identify and document internal negative controls. It is also a place to develop the reference control, a standard sample that is run each time the assay is performed to ensure that the process worked.With the reference control, it is important to use this to establish acceptance criteria so that when the experiment is being performed the process can be tracked and issues identified (more on that later).
  4. Validation (optional, but recommended)Validation is the process where the performance of the assay is assessed. The optimization step determines the best conditions to run the assay and validation uses those conditions to assess how well the assay performs. In regulated environments, this is an important step, but many researcher labs don’t do this extra step. It is a good place to make sure everything is working correctly. Performing such experiments as the same sample run by N different people on the same day, or the same sample run by the same person over X days are ways to see the day to day variation and researcher variation to provide additional limits for the assay.

4. Implement the Panel

This is where the fun begins. The experimental design is set, the metrics are established and the SOPs are written. The team is trained on the SOPs and the samples are getting lined up. This is where the QC information comes into play. Take the peak 6 bead for tracking voltage. Since the beads were used to set target values, tracking the voltage over time for each detector should be implemented.

Figure 7: Levy-Jennings plot for one PMT

The Levy-Jennings plot is one way to track changes over time. In this figure, the green line represents the average of the data, the yellow lines +/- 1 standard deviation, and the red lines are +/-2 standard deviations. Depending on how tight the standards are will determine when there are issues. As you can see from this data, the voltage ranges vary within approximately 1 standard deviation from the mean.

This analysis can also be applied to the reference control. In the data shown below, three different populations are tracked over time. In this case, the Levy-Jennings control lines are based on the established ranges that were calculated based on the optimization runs.

Figure 8: Tracking changes over time.

This QC tracking provides a level of quality that goes directly to the reproducibility of the data. When running the experiment and using the reference control, issues with the experimental process can be detected and provide grounds to exclude data where the controls are outside of the acceptable range.

Additionally, all the paperwork associated with each experiment should be reviewed by a supervisor and initialed to indicate they were reviewed.

Quality control is the hallmark of improving reproducibility. QC programs are designed to help determine when the process in question goes off the expected path. Depending on the deviation from the established acceptance criteria will dictact the level of intervention that needs to occur. This can be as easy as cleaning the instrument and rerunning the QC, or as extreme as removing the data from the final analysis. Since there is documentation as to the deviations, this provides the rationale for excluding data.

At the end of the day, improving reproducibility is a critical step to improve the confidence of data. With reports of 25% or less of data being reproducible, it is essential that researchers adopt a mindset that will reduce the concerns and improve the quality of the scientific product. Consider adding a few additional steps in the process of developing your flow cytometry experiments that will provide that piece of mind.

To learn more about the 4 Steps To Implementing a QC Program For Your Flow Cytometry Experiments, and to get access to all of our advanced materials including 20 training videos, presentations, workbooks, and private group membership, get on the Flow Cytometry Mastery Class wait list.

Join Expert Cytometry's Mastery Class
Tim Bushnell, PhD
Tim Bushnell, PhD

Tim Bushnell holds a PhD in Biology from the Rensselaer Polytechnic Institute. He is a co-founder of—and didactic mind behind—ExCyte, the world’s leading flow cytometry training company, which organization boasts a veritable library of in-the-lab resources on sequencing, microscopy, and related topics in the life sciences.

Similar Articles

The Power Of Spectral Viewers And Their Use In Full Spectrum Flow Cytometry

The Power Of Spectral Viewers And Their Use In Full Spectrum Flow Cytometry

By: Tim Bushnell, PhD

What photon from yonder fluorochrome breaks?  It is … umm… hmmm. Let me see. Excitation off a 561 nm laser, with an emission maximum of 692 nm. I’m sure if Shakespeare was a flow cytometrist, he might have written that very scene. But the play is lost in time. However, since the protagonist had difficulty determining what fluorochrome was emitting photons, let’s consider how this could be figured out. In my opinion, one of the handiest flow cytometry tools is the spectral viewer. This tool helps visualize the excitation and emission profile of different fluorochromes, as well as allowing you…

Fickle Markers: Solutions For Antibody Binding Specificity Challenges

Fickle Markers: Solutions For Antibody Binding Specificity Challenges

By: Tim Bushnell, PhD

Reproducibility has been an ongoing, and important, concept in the sciences for years.  In the area of biomedical research, the alarm was sounded by several papers published in the early 2010’s.  Authors like Begley and Ellis, Prinz and coworkers, and Vasilevsky and colleagues, among others reported an alarming trend in the reproducibility of pre-clinical data.  These reports indicated between 50% to almost 90% of published pre-clinical data were not reproducible.  This was further highlighted in the article by Freedman and coworkers, who tried to identify and quantify the different sources of error that could be causing this crisis.  Figure 1,…

5 Common Flow Cytometry Questions, Answered

5 Common Flow Cytometry Questions, Answered

By: Tim Bushnell, PhD

I want to thank all of you who send us your questions about flow cytometry, so I thought I would dip into the old email bag and answer a few of the common ones here.  If your question isn’t answered this time, look for it to be answered in a future blog post.  Of course, if you want us to cover a specific topic, drop us a line.  1. How Fast Can I Go? This is  a common question. The allure of the ‘hi’ button is hard to resist.  The faster you go, the sooner you are finished with data…

Combining Flow Cytometry With Plant Science, Microorganisms, And The Environment

Combining Flow Cytometry With Plant Science, Microorganisms, And The Environment

By: Tim Bushnell, PhD

My first introduction to flow cytometry was talking to a professor who’d brought one on a research cruise to study phytoplankton. It was only later that I was introduced to the marvelous world that’s been my career for over 20 years.   In that time, I’ve had the opportunity to work with researchers in many different areas, exposing me to a wide variety of cell types and more important assays. What continues to amaze me is the number of different parameters we can measure, not just the number of fluorochromes, but the information we can extract from samples – animal, vegetable…

Common Numbers-Based Questions I Get As A Flow Cytometry Core Manager And How To Answer Them

Common Numbers-Based Questions I Get As A Flow Cytometry Core Manager And How To Answer Them

By: Tim Bushnell, PhD

Numbers are all around us.  My personal favorite is ≅1.618 aka ɸ aka ‘the golden ratio’.  It’s found throughout history, where it has influenced architects and artists. We see it in nature, in plants, and it is used in movies to frame shots. It can be approximated by the Fibonacci sequence (another math favorite of mine). However, I have not worked out how to apply this to flow cytometry.  That doesn’t mean numbers aren’t important in flow cytometry. They are central to everything we do, and in this blog, I’m going to flit around numbers-based questions that I have received…

3 Must-Have High-Dimensional Flow Cytometry Controls

3 Must-Have High-Dimensional Flow Cytometry Controls

By: Tim Bushnell, PhD

Developments such as the recent upgrade to the Cytobank analysis platform and the creation of new packages such as Immunocluster are reducing the computational expertise needed to work with high-dimensional flow cytometry datasets. Whether you are a researcher in academia, industry, or government, you may want to take advantage of the reduced barrier to entry to apply high-dimensional flow cytometry in your work. However, you’ll need the right experimental design to access the new transformative insights available through these approaches and avoid wasting the considerable time and money required for performing them. As with all experiments, a good design begins…

The Fluorochrome Less Excited: How To Build A Flow Cytometry Antibody Panel

The Fluorochrome Less Excited: How To Build A Flow Cytometry Antibody Panel

By: Tim Bushnell, PhD

Fluorochrome, antibodies and detectors are important. The journey of a thousand cells starts with a good fluorescent panel. The polychromatic panel is the combination of antibodies and fluorochromes. These will be used during the experiment to answer the biological question of interest. When you only need a few targets, the creation of the panel is relatively straightforward. It’s only when you start to get into more complex panels with multiple fluorochromes that overlap in excitation and emission gets more interesting.  FLUOROCHROMES Both full spectrum and traditional fluorescent flow cytometry rely on measuring the emission of the fluorochromes that are attached…

Flow Cytometry Year in Review: Key Changes To Know

Flow Cytometry Year in Review: Key Changes To Know

By: Meerambika Mishra

Here we are, at the end of an eventful year 2021. But with the promise of a new year 2022 to come. It has been a long year, filled with ups and downs. It is always good to reflect on the past year as we move to the future.  In Memoriam Sir Isaac Newton wrote “If I have seen further, it is by standing upon the shoulders of giants.” In the past year, we have lost some giants of our field including Zbigniew Darzynkiwicz, who contributed much in the areas of cell cycle analysis and apoptosis. Howard Shapiro, known for…

What Star Trek Taught Me About Flow Cytometry

What Star Trek Taught Me About Flow Cytometry

By: Tim Bushnell, PhD

It is no secret that I am a very big fan of the Star Trek franchise. There are many good episodes and lessons explored in the 813+ episodes, 12 movies (and counting). Don’t worry, this blog is not going to review all 813, or even 5 of them. Instead, some of the lessons I have taken away from the show that have applicability to science and flow cytometry.  “Darmok and Jalad at Tanagra.”  (ST:TNG season 5, episode 2) This is probably one of my favorite episodes, which involves Picard and an alien trying to establish a common ground and learn…

Top Technical Training eBooks

Get the Advanced Microscopy eBook

Get the Advanced Microscopy eBook

Heather Brown-Harding, PhD

Learn the best practices and advanced techniques across the diverse fields of microscopy, including instrumentation, experimental setup, image analysis, figure preparation, and more.

Get The Free Modern Flow Cytometry eBook

Get The Free Modern Flow Cytometry eBook

Tim Bushnell, PhD

Learn the best practices of flow cytometry experimentation, data analysis, figure preparation, antibody panel design, instrumentation and more.

Get The Free 4-10 Compensation eBook

Get The Free 4-10 Compensation eBook

Tim Bushnell, PhD

Advanced 4-10 Color Compensation, Learn strategies for designing advanced antibody compensation panels and how to use your compensation matrix to analyze your experimental data.