Instrument Quality Control For Reproducible Flow Cytometry Experiments

The NIH has released a series of reproducibility guidelines that scientists must address. These guidelines have been introduced because there’s a lot of data showing that reproducibility in science is frustratingly lacking in certain experiments.

Non-reproducible experiments waste time, money, and resources. As Begley and Ioannidis cite in their 2015 article :

“The estimates for irreproducibility based on these empirical observations range from 75% to 90%. These estimates fit remarkably well with estimates of 85% for the proportion of biomedical research that is wasted at-large.”

Reproducibility is a mindset, and involves an overall analysis of the scientific process to identify the areas that can be improved. In this article by Bruce Booth, he reviews “Begley’s six rules”. Two of these rules focus on the controls and reagents used in the experiment.

Of equal importance are the instruments used to make measurements. For example, how often are the pipettes calibrated? Are all lab members adequately trained in technique? This chart from Gilson is a useful one to have in the lab as a reminder on proper pipetting form.

The flow cytometer is an integral component of any flow cytometry experiment, and special attention should be paid to ensuring that it is working correctly and consistently. As an end-user, the researcher should be able to sit down at a machine and know that it is performing the same way today as it was yesterday and last week.

Equally important is that if any changes in instrument performance have occured, the end-user knows how they have been addressed and corrected, rather than letting them fester and potentially affect the results.

For those using core facility equipment, it is important to talk to the people who maintain your instruments and look at the quality control data.

Ask them how are they are assessing and maintaining the quality of the instrument and ask what is the best way for you to make sure your data is consistent. The staff will be delighted to to advise! Quality control measurements can include a variety of targets, such as PMT sensitivity, laser alignment, fluidic stability, background issues, and more.

1. PMT Voltage Optimization.

With the analog flow cytometers, researchers were taught to run an unstained cell sample and adjust voltages so that the background was in the first log decade. This outdated practice continues to be taught, even on new digital instruments.

A better approach is to have optimized PMT voltage settings, and to use these voltages for experiments.

In 2006, Maecker and Trotter published a paper on how to determine the optimal voltage for a PMT. In this case, the second peak of the Spherotech 8-peak Rainbow Calibration Particles (RCP-3-5A-2) is run over a voltage range for each PMT. The robust coefficient of variance (rCV) is calculated for each PMT at each voltage, and a curve is plotted, as shown in the figure below.

Figure 1: Peak-2 characterization of PMTs.

The resulting curve shows a steep descent where the rCV decreases as voltage increases. Eventually, an inflection point is reached where the slope of the line changes. At this point, increasing voltage doesn’t improve the rCV.

The optimal voltage range is defined as the region just after the inflection point. In the case of Figure 1, this is between 400 to 450 volts.

When running a one-off experiment, set the voltage for the detectors to these voltages and begin collecting data. There are very few reasons to deviate from this starting voltage.

The most common situation is if the fluorescent signal on your target cells is out of the linear range of the detector, or off-scale. In this case, one might consider reducing the PMT voltage, but a better approach would be to reduce the fluorescence intensity by staining with a mix of labeled and unlabeled antibody.

When doing this, you keep the total antibody concentration constant, but now the cells will have a lower total fluorescence due to the presence of unlabeled antibody.

However, peak-2 beads are not cells, and don’t necessarily have the same spectral characteristics as the fluorochromes used in the experiment.

If the panel is going to be used consistently over a long period of time, the optimal voltage can be refined by performing a voltage titration (voltration) experiment using stained cells.

An example of this secondary optimization is shown in Figure 2. On the left, is the optimization for CY7-APC and on the right, brilliant violet 650. The peak-2 value for each of these fluorochromes is pointed out with the blue arrow. However, this peak-2 value is not always the optimal voltage for a specific fluorochrome.

Figure 2: Voltage optimization of 2 fluorochromes.

To determine if the peak-2 value is optimal, increase the voltage past this point and calculate the staining index. In the case of CY7-APC, as voltage is increased, the signal very rapidly moved out of the linear dynamic range, as shown by the red arrow, and eventually moved completely off scale. This indicates that the peak-2 value is best for this fluorochrome.

Brilliant Violet 650, on the other hand, shows an increase in the staining index above the peak-2 value. At the optimal voltage for this fluorochrome-detector pair, the SI is about 16% higher than at the peak-2 value, suggesting that this higher voltage is better for making a more sensitive measurement.

Once this voltration is performed, and the optimal voltage identified, it becomes important to maintain performance consistency over time. To do this, another bead is used. In this case, the sixth peak of the 8-peak Rainbow Calibration Particles, or any other bright bead of your choice.

Run the bead at the optimal voltages and determine the target values in each detector. This is typically measured as the median fluorescent intensity (MFI). Each time the experiment is to be run, the calibration bead is run first and detector voltages are adjusted to ensure that the MFI is maintained, within some tolerance range.

Once that is done, begin collecting the rest of the tubes. If the voltage changes are significant, it is good to pause and consult the daily QC, or your core staff, to determine if there has been a change to the system.

Defining “significant” is up the investigator, but a good rule of thumb would be that deviation of more than 10% in voltage should require some investigation. You should collect and record at least 10,000 single beads for your QC tracking.

It is important to remember that Quality Control like this is not valuable unless it is monitored. To do that, use the Levey-Jennings plot. This is a quality control plot that shows the running average of the data with control lines. In the case below, the dotted lines represent +/- 1 and 2 standard deviations around the mean.

This plot can be used to spot trends in the data. For example, if a data point falls outside of the control lines, it is a sign there may be an issue with the system, and intervention may be required.

This could just be cleaning, but may also indicate something else that the manager of the instrument needs to know. If, over time, there is a trend towards one direction or the other (continued increase and/or decrease), it is also a good time to intervene, if for no other reason than to see if other issues with the system have occured.

Figure 3: Levey-Jennings plot over time tracking PMT voltage changes using peak-6 bead method.

As shown in Figure 3, the data on this instrument over 100 days within 2 standard deviations of the mean, giving added confidence that the instrument is working consistently and changes in the data are more likely due to biology, and not instrument issues.

If these start to fall out of this range, then you would need to investigate and troubleshoot. You can be more restrictive and use only one standard deviation, but the industry standard is to use two.

While instrument QC is usually left to the manager of the instrument, adding experimental QC using an independent measurement will aid in improving the quality, and ultimately reproducibility, of the data being generated. Something may change on the instrument between when the manager checks in the morning and when you sit down to run, so it’s wise to check performance for yourself.

2. Parallel Arrangement and Fluidics Stability.

Most of the current instruments have a parallel laser arrangement. This means that the cell passes through lasers in sequence, so they are exposed to only a single excitation source at a time. To ensure that the correct signal pulses are paired together, a factor called the “time delay” is calculated during QC.

If the time delay is off, this will adversely impact the data as the cells arrive either too early or too late, thus the correct pluses are not paired together.

This can easily be identified retrospectively, but it is better to identify the issues during acquisition. If there is a loss of signal on a given excitation source, it can suggest a clog in the system that needs to be cleaned.

Consider a partial clog on the input side of the cytometer (before the flow cell). When the diameter of a pipe is reduced, that increases the flow (speed) of the liquid in the reduced diameter. This results in cells arriving at the interrogation point more quickly than the time delay is expecting them.

A clog after the flow cell has the same issue, but now the pressure builds up behind the clog, slowing down the stream, and cells arrive after the time delay window. This can be seen in Figure 4.

Figure 4: Impact of clogs on data.

When running your experiments, put up a plot of a bright fluorochrome versus time for each of the lasers that are being used on the machine. This will help determine if there are problems with the fluidics, as well as with the lasers.

In the example shown here, PE-Cy5.5 was being excited by a green laser which was the fourth in line, but there was a clog causing a back pressure issue. The clog caused a speedup of the number of events because it constricted the flow core and increased the flow rate in the number of events.

Eventually, the clog worked itself out and the signal came back. As with any quality control metric used, it’s important to not only run theses controls, but to look at them on a regular basis.

3. Compensation.

Compensation is one of most important steps in the flow cytometry process. For proper compensation it is critical to bring the right controls to the instrument every time. This is such an important topic, it has been discussed extensively here. It is here in discussing instrument reproducibility as compensation controls also give insight into how the system is working, and provide a secondary proxy for instrument QC.

These compensation controls must follow the 3 commandments:

1. Controls need to be at least as bright as any sample to which you’ll apply to your compensation. That means, the control signal needs to be at least as bright as anything in your experimental signal. If it’s not as bright, then your compensation will be incorrect. It also has to be within the linear range of the detector.

2. The background fluorescence should be the same for the positive and negative control populations for any given parameter.

Compensation is a property of the fluorochrome, not a property of the antibody or a property of the carrier that brings the fluorochrome to the intercept point. This is why you can use either beads or cells. What’s important is that the background fluorescence, the autofluorescence of the negative population and a positive are the same. Since compensation corrects to the background of the control, it is critical to have a positive and negative in each control. This means it is possible to mix beads and cells in a compensation matrix as long as each positive is linked to the appropriate negative particle type.

3. The compensation colors must be exactly matched to your experimental color.
Alexa 488 cannot be used to compensate for GFP or for FITC. Likewise, if there are two different lots of tandem dye, make sure that lots are matched in the experiment and control. One lot of PE-Cy5 cannot be used to compensate a different lot of PE-Cy5 and vice versa, because each of these dyes are manufactured, and likely have different spectral characteristics.

BONUS 4. You need to have enough events!

This is a minimum of 10,000 single events, when using beads. When using cells, you need a minimum of 30,000 single cell events. Collecting more is good, but don’t collect less than these minimums.

Since fluorescence is a property of the fluorochrome, not the carrier, the question arises, “Should you use beads or should you use cells?” Either carrier is fine, as long as the positive populations have the same background fluorescence when unstained — i.e. if you use beads, then make sure you have unstained beads and if you use cells, make sure you have unstained cells. If you have some on beads and some on cells, then make sure you pair each positive with an appropriate negative.

4. Quality Control.

Often, the QC of an instrument is left to the manager of the instrument. This can be done in a variety of ways, from an instrument-specific QC protocol or something from the manufacturer, like the BD CS&T or the Attune performance tracking beads.

This process should be run every time the instrument is turned on, to ensure that a library of instrument behaviour is developed and to allow the users to understand how the system is performing. QC of this nature is critical best practices for the instrument, and provides baseline confirmation the system is working adequately.

Don’t hesitate to ask the manager of the instrument to look at their QC metrics, and ask for an explanation when things don’t seem clear. After all, the success or failure of the experiment lies in the hands of how well the flow cytometer is working, and most cytometrists will be overjoyed to talk shop!

In conclusion, one key area to improve consistency and reproducibility is to monitor instrument quality control. Knowing how the system is QCed and what happens when something goes wrong is paramount to ensuring that the instrument is properly characterized.

To learn more about Instrument Quality Control For Reproducible Flow Cytometry Experiments, and to get access to all of our advanced materials including 20 training videos, presentations, workbooks, and private group membership, get on the Flow Cytometry Mastery Class wait list.

Join Expert Cytometry's Mastery Class
Tim Bushnell, PhD
Tim Bushnell, PhD

Tim Bushnell holds a PhD in Biology from the Rensselaer Polytechnic Institute. He is a co-founder of—and didactic mind behind—ExCyte, the world’s leading flow cytometry training company, which organization boasts a veritable library of in-the-lab resources on sequencing, microscopy, and related topics in the life sciences.

Similar Articles

Getting A New Flow Cytometer? Try Before You Buy (And 2 Other Tips)

Getting A New Flow Cytometer? Try Before You Buy (And 2 Other Tips)

By: Tim Bushnell, PhD

There are 4 major ways to sort cells. The first way can use magnetic beads coupled to antibodies and pass the cells through a magnetic field. The labeled cells will stick, and the unlabeled cells will remain in the supernatant. The second way is to use some sort of mechanical force like a flapper or air stream that separates the target cells from the bulk population. The third way is the recently introduced microfluidics sorter, which uses microfluidics channels to isolate the target cells. The last method, which is the most common––based on Fuwyler’s work––is the electrostatic cell sorter. This…

5-Point Guide To Buying A New Microscope For Your Lab

5-Point Guide To Buying A New Microscope For Your Lab

By: Heather Brown-Harding, PhD

Have you ever noticed how painful it can be to purchase a new microscope? It would be hard to miss – this can be a frustrating process. A lot of scientists and students consider the new microscope hunt quite scary for a variety of reasons. It might be that you’re worried you won’t get the right microscope and that you’ll regret it, or you may find that dealing with salespeople, in general, makes you kind of uncomfortable. But remember, salespeople are just human beings like you and me, and if we can treat them as such, the whole process of…

Ask These 7 Questions Before Purchasing A Flow Cytometer

Ask These 7 Questions Before Purchasing A Flow Cytometer

By: Tim Bushnell, PhD

I am still convinced that my first cell sorter was possessed. The number of issues that I had with the system remains hard for me to believe, even after all these years. It had been purchased, in part, from one vendor because the sales rep for a competitor was nowhere to be found. At that time, I admit I wasn’t overly diligent in my research process. Since then, I’ve pinpointed some critical questions that need to be answered before purchasing a new instrument. At the end of the process, a shiny new instrument will arrive at your facility. Make sure…

How to Optimize Flow Cytometry Hardware For Rare Event Analysis

How to Optimize Flow Cytometry Hardware For Rare Event Analysis

By: Tim Bushnell, PhD

Preparing for rare event analysis requires an understanding of the power and limitation of the instrument to be used. From how fast to run the fluidics, to how the signal is processed to the number of gates that can be used in the sorting experiment, each factor impacts the outcome of the experiment.

3 Ways The ZE5 Cell Analyzer Accelerates Flow Cytometry Research Opportunities

3 Ways The ZE5 Cell Analyzer Accelerates Flow Cytometry Research Opportunities

By: Tim Bushnell, PhD

Some technological advances are incremental, while others are significant game-changing tools that offer the researcher the ability to significantly improve current assays while allowing for new and novel avenues of research to be performed. With speed, sensitivity, and capacity to spare, the ZE5 fits into the game-changing category. Reduced carryover, increased speed of acquisition, and a large number of parameters all open up new and novel assays while improving the quality and reproducibility of ongoing ones.

3 Advantages Of Using The ZE5 Cell Analyzer

3 Advantages Of Using The ZE5 Cell Analyzer

By: Tim Bushnell, PhD

Since the first laser was mounted to create the first flow cytometer, there has been a push for more - more lasers, more detectors, more colors. As a result, today’s researchers require a large number of lasers and detectors to ensure current panels can be run and new, expanded panels can be developed. This can be problematic because, in general, making one decision to improve a cell analyzer can limit the analyzer in other ways. It may seem like an impossible task, but the team of Bio-Rad and Propel Laboratories, collaborated to bring the ZE5™ Cell Analyzer to the market…

3 Advantages FCS Express 6 Has Over Other Flow Cytometry Data Analysis Software Programs

3 Advantages FCS Express 6 Has Over Other Flow Cytometry Data Analysis Software Programs

By: Tim Bushnell, PhD

FCS Express is the ideal data analysis software program to use when analyzing your flow cytometry experiments because it is the most user-friendly program available that is both aligned with current data analysis best practices and maintains rigorous quality control standards.

How To Use A Threshold To Reduce Background Noise In Flow Cytometry

How To Use A Threshold To Reduce Background Noise In Flow Cytometry

By: Tim Bushnell, PhD

Getting a clear signal with reduced noise is an essential component to good data. Adding a threshold when acquiring flow cytometry data is one way to do that. It reduces the number of events by setting a bar that a signal pulse must clear before it is counted as an event. Depending on the importance of the data, the downstream applications for the data (or sorted cells) will dictate how critical the threshold is. In combination with proper sample preparation, appropriate thresholding will reduce debris and ensure best outcome.

How To Set And Monitor Optimal Voltages For A Flow Cytometry Experiment

How To Set And Monitor Optimal Voltages For A Flow Cytometry Experiment

By: Tim Bushnell, PhD

The best way to take out the fear and agony of setting voltages is to use some optimization methods. The peak 2 method is a useful and robust method of identifying optimal PMT voltage ranges. Refining that to the voltage walk with the actual cells and fluorochromes of interest will further improve sensitivity, which is especially critical for rare cell populations or emergent antigens. This article describes how to set up, monitor, and maintain optimal voltage settings for your flow cytometry experiment.

Top Technical Training eBooks

Get the Advanced Microscopy eBook

Get the Advanced Microscopy eBook

Heather Brown-Harding, PhD

Learn the best practices and advanced techniques across the diverse fields of microscopy, including instrumentation, experimental setup, image analysis, figure preparation, and more.

Get The Free Modern Flow Cytometry eBook

Get The Free Modern Flow Cytometry eBook

Tim Bushnell, PhD

Learn the best practices of flow cytometry experimentation, data analysis, figure preparation, antibody panel design, instrumentation and more.

Get The Free 4-10 Compensation eBook

Get The Free 4-10 Compensation eBook

Tim Bushnell, PhD

Advanced 4-10 Color Compensation, Learn strategies for designing advanced antibody compensation panels and how to use your compensation matrix to analyze your experimental data.