Instrument Quality Control For Reproducible Flow Cytometry Experiments

The NIH has released a series of reproducibility guidelines that scientists must address. These guidelines have been introduced because there’s a lot of data showing that reproducibility in science is frustratingly lacking in certain experiments.

Non-reproducible experiments waste time, money, and resources. As Begley and Ioannidis cite in their 2015 article :

“The estimates for irreproducibility based on these empirical observations range from 75% to 90%. These estimates fit remarkably well with estimates of 85% for the proportion of biomedical research that is wasted at-large.”

Reproducibility is a mindset, and involves an overall analysis of the scientific process to identify the areas that can be improved. In this article by Bruce Booth, he reviews “Begley’s six rules”. Two of these rules focus on the controls and reagents used in the experiment.

Of equal importance are the instruments used to make measurements. For example, how often are the pipettes calibrated? Are all lab members adequately trained in technique? This chart from Gilson is a useful one to have in the lab as a reminder on proper pipetting form.

The flow cytometer is an integral component of any flow cytometry experiment, and special attention should be paid to ensuring that it is working correctly and consistently. As an end-user, the researcher should be able to sit down at a machine and know that it is performing the same way today as it was yesterday and last week.

Equally important is that if any changes in instrument performance have occured, the end-user knows how they have been addressed and corrected, rather than letting them fester and potentially affect the results.

For those using core facility equipment, it is important to talk to the people who maintain your instruments and look at the quality control data.

Ask them how are they are assessing and maintaining the quality of the instrument and ask what is the best way for you to make sure your data is consistent. The staff will be delighted to to advise! Quality control measurements can include a variety of targets, such as PMT sensitivity, laser alignment, fluidic stability, background issues, and more.

1. PMT Voltage Optimization.

With the analog flow cytometers, researchers were taught to run an unstained cell sample and adjust voltages so that the background was in the first log decade. This outdated practice continues to be taught, even on new digital instruments.

A better approach is to have optimized PMT voltage settings, and to use these voltages for experiments.

In 2006, Maecker and Trotter published a paper on how to determine the optimal voltage for a PMT. In this case, the second peak of the Spherotech 8-peak Rainbow Calibration Particles (RCP-3-5A-2) is run over a voltage range for each PMT. The robust coefficient of variance (rCV) is calculated for each PMT at each voltage, and a curve is plotted, as shown in the figure below.

Figure 1: Peak-2 characterization of PMTs.

The resulting curve shows a steep descent where the rCV decreases as voltage increases. Eventually, an inflection point is reached where the slope of the line changes. At this point, increasing voltage doesn’t improve the rCV.

The optimal voltage range is defined as the region just after the inflection point. In the case of Figure 1, this is between 400 to 450 volts.

When running a one-off experiment, set the voltage for the detectors to these voltages and begin collecting data. There are very few reasons to deviate from this starting voltage.

The most common situation is if the fluorescent signal on your target cells is out of the linear range of the detector, or off-scale. In this case, one might consider reducing the PMT voltage, but a better approach would be to reduce the fluorescence intensity by staining with a mix of labeled and unlabeled antibody.

When doing this, you keep the total antibody concentration constant, but now the cells will have a lower total fluorescence due to the presence of unlabeled antibody.

However, peak-2 beads are not cells, and don’t necessarily have the same spectral characteristics as the fluorochromes used in the experiment.

If the panel is going to be used consistently over a long period of time, the optimal voltage can be refined by performing a voltage titration (voltration) experiment using stained cells.

An example of this secondary optimization is shown in Figure 2. On the left, is the optimization for CY7-APC and on the right, brilliant violet 650. The peak-2 value for each of these fluorochromes is pointed out with the blue arrow. However, this peak-2 value is not always the optimal voltage for a specific fluorochrome.

Figure 2: Voltage optimization of 2 fluorochromes.

To determine if the peak-2 value is optimal, increase the voltage past this point and calculate the staining index. In the case of CY7-APC, as voltage is increased, the signal very rapidly moved out of the linear dynamic range, as shown by the red arrow, and eventually moved completely off scale. This indicates that the peak-2 value is best for this fluorochrome.

Brilliant Violet 650, on the other hand, shows an increase in the staining index above the peak-2 value. At the optimal voltage for this fluorochrome-detector pair, the SI is about 16% higher than at the peak-2 value, suggesting that this higher voltage is better for making a more sensitive measurement.

Once this voltration is performed, and the optimal voltage identified, it becomes important to maintain performance consistency over time. To do this, another bead is used. In this case, the sixth peak of the 8-peak Rainbow Calibration Particles, or any other bright bead of your choice.

Run the bead at the optimal voltages and determine the target values in each detector. This is typically measured as the median fluorescent intensity (MFI). Each time the experiment is to be run, the calibration bead is run first and detector voltages are adjusted to ensure that the MFI is maintained, within some tolerance range.

Once that is done, begin collecting the rest of the tubes. If the voltage changes are significant, it is good to pause and consult the daily QC, or your core staff, to determine if there has been a change to the system.

Defining “significant” is up the investigator, but a good rule of thumb would be that deviation of more than 10% in voltage should require some investigation. You should collect and record at least 10,000 single beads for your QC tracking.

It is important to remember that Quality Control like this is not valuable unless it is monitored. To do that, use the Levey-Jennings plot. This is a quality control plot that shows the running average of the data with control lines. In the case below, the dotted lines represent +/- 1 and 2 standard deviations around the mean.

This plot can be used to spot trends in the data. For example, if a data point falls outside of the control lines, it is a sign there may be an issue with the system, and intervention may be required.

This could just be cleaning, but may also indicate something else that the manager of the instrument needs to know. If, over time, there is a trend towards one direction or the other (continued increase and/or decrease), it is also a good time to intervene, if for no other reason than to see if other issues with the system have occured.

Figure 3: Levey-Jennings plot over time tracking PMT voltage changes using peak-6 bead method.

As shown in Figure 3, the data on this instrument over 100 days within 2 standard deviations of the mean, giving added confidence that the instrument is working consistently and changes in the data are more likely due to biology, and not instrument issues.

If these start to fall out of this range, then you would need to investigate and troubleshoot. You can be more restrictive and use only one standard deviation, but the industry standard is to use two.

While instrument QC is usually left to the manager of the instrument, adding experimental QC using an independent measurement will aid in improving the quality, and ultimately reproducibility, of the data being generated. Something may change on the instrument between when the manager checks in the morning and when you sit down to run, so it’s wise to check performance for yourself.

2. Parallel Arrangement and Fluidics Stability.

Most of the current instruments have a parallel laser arrangement. This means that the cell passes through lasers in sequence, so they are exposed to only a single excitation source at a time. To ensure that the correct signal pulses are paired together, a factor called the “time delay” is calculated during QC.

If the time delay is off, this will adversely impact the data as the cells arrive either too early or too late, thus the correct pluses are not paired together.

This can easily be identified retrospectively, but it is better to identify the issues during acquisition. If there is a loss of signal on a given excitation source, it can suggest a clog in the system that needs to be cleaned.

Consider a partial clog on the input side of the cytometer (before the flow cell). When the diameter of a pipe is reduced, that increases the flow (speed) of the liquid in the reduced diameter. This results in cells arriving at the interrogation point more quickly than the time delay is expecting them.

A clog after the flow cell has the same issue, but now the pressure builds up behind the clog, slowing down the stream, and cells arrive after the time delay window. This can be seen in Figure 4.

Figure 4: Impact of clogs on data.

When running your experiments, put up a plot of a bright fluorochrome versus time for each of the lasers that are being used on the machine. This will help determine if there are problems with the fluidics, as well as with the lasers.

In the example shown here, PE-Cy5.5 was being excited by a green laser which was the fourth in line, but there was a clog causing a back pressure issue. The clog caused a speedup of the number of events because it constricted the flow core and increased the flow rate in the number of events.

Eventually, the clog worked itself out and the signal came back. As with any quality control metric used, it’s important to not only run theses controls, but to look at them on a regular basis.

3. Compensation.

Compensation is one of most important steps in the flow cytometry process. For proper compensation it is critical to bring the right controls to the instrument every time. This is such an important topic, it has been discussed extensively here. It is here in discussing instrument reproducibility as compensation controls also give insight into how the system is working, and provide a secondary proxy for instrument QC.

These compensation controls must follow the 3 commandments:

1. Controls need to be at least as bright as any sample to which you’ll apply to your compensation. That means, the control signal needs to be at least as bright as anything in your experimental signal. If it’s not as bright, then your compensation will be incorrect. It also has to be within the linear range of the detector.

2. The background fluorescence should be the same for the positive and negative control populations for any given parameter.

Compensation is a property of the fluorochrome, not a property of the antibody or a property of the carrier that brings the fluorochrome to the intercept point. This is why you can use either beads or cells. What’s important is that the background fluorescence, the autofluorescence of the negative population and a positive are the same. Since compensation corrects to the background of the control, it is critical to have a positive and negative in each control. This means it is possible to mix beads and cells in a compensation matrix as long as each positive is linked to the appropriate negative particle type.

3. The compensation colors must be exactly matched to your experimental color.
Alexa 488 cannot be used to compensate for GFP or for FITC. Likewise, if there are two different lots of tandem dye, make sure that lots are matched in the experiment and control. One lot of PE-Cy5 cannot be used to compensate a different lot of PE-Cy5 and vice versa, because each of these dyes are manufactured, and likely have different spectral characteristics.

BONUS 4. You need to have enough events!

This is a minimum of 10,000 single events, when using beads. When using cells, you need a minimum of 30,000 single cell events. Collecting more is good, but don’t collect less than these minimums.

Since fluorescence is a property of the fluorochrome, not the carrier, the question arises, “Should you use beads or should you use cells?” Either carrier is fine, as long as the positive populations have the same background fluorescence when unstained — i.e. if you use beads, then make sure you have unstained beads and if you use cells, make sure you have unstained cells. If you have some on beads and some on cells, then make sure you pair each positive with an appropriate negative.

4. Quality Control.

Often, the QC of an instrument is left to the manager of the instrument. This can be done in a variety of ways, from an instrument-specific QC protocol or something from the manufacturer, like the BD CS&T or the Attune performance tracking beads.

This process should be run every time the instrument is turned on, to ensure that a library of instrument behaviour is developed and to allow the users to understand how the system is performing. QC of this nature is critical best practices for the instrument, and provides baseline confirmation the system is working adequately.

Don’t hesitate to ask the manager of the instrument to look at their QC metrics, and ask for an explanation when things don’t seem clear. After all, the success or failure of the experiment lies in the hands of how well the flow cytometer is working, and most cytometrists will be overjoyed to talk shop!

In conclusion, one key area to improve consistency and reproducibility is to monitor instrument quality control. Knowing how the system is QCed and what happens when something goes wrong is paramount to ensuring that the instrument is properly characterized.

To learn more about Instrument Quality Control For Reproducible Flow Cytometry Experiments, and to get access to all of our advanced materials including 20 training videos, presentations, workbooks, and private group membership, get on the Flow Cytometry Mastery Class wait list.

Join Expert Cytometry's Mastery Class
Tim Bushnell, PhD
Tim Bushnell, PhD

Tim Bushnell holds a PhD in Biology from the Rensselaer Polytechnic Institute. He is a co-founder of—and didactic mind behind—ExCyte, the world’s leading flow cytometry training company, which organization boasts a veritable library of in-the-lab resources on sequencing, microscopy, and related topics in the life sciences.

Similar Articles

Which Fluorophores To Use For Your Microscopy Experiment

Which Fluorophores To Use For Your Microscopy Experiment

By: Heather Brown-Harding, PhD

Fluorophore selection is important. I have often been asked by my facility users which fluorophore is best suited for their experiments. The answer to this is mostly dependent on whether they are using a widefield microscope with set excitation/emission cubes or a laser based system that lets you select the laser and the emission window. Once you have narrowed down which fluorophores you can excite and collect the correct emission, you can further refine the specific fluorophore that is best for your experiment.  In this blog  we will discuss how to determine what can work with your microscope, and how…

4 No Cost Ways To Improve Your Microscopy Image Quality

4 No Cost Ways To Improve Your Microscopy Image Quality

By: Heather Brown-Harding, PhD

Image quality is critical for accurate and reproducible data. Many people get stuck on the magnification of the objective or on using a confocal instead of a widefield microscope. There are several other factors that affect the image quality such as the numerical aperture of the objective, the signal-to-noise ratio of the system, or the brightness of the sample.  Numerical aperture is the ability of an objective to collect light from a sample, but it contributes to two key formulas that will affect your image quality. The first is the theoretical resolution of the objective. It is expressed with the…

What Is Total Internal Reflection Fluorescence (TIRF) Microscopy & Is It Right For You?

What Is Total Internal Reflection Fluorescence (TIRF) Microscopy & Is It Right For You?

By: Heather Brown-Harding, PhD

TIRF is not as common as other microscopy based techniques due to certain restrictions. We will discuss these restrictions, then analyze why it might be perfect for your experiment.  TIRF relies on an evanescent wave, created through a critical angle of coherent light (i.e. laser) that reaches a refractive index mismatch.  What does it mean in practice?  A high angle laser reflects off the interface of the coverslip and the sample. Although the depth that this wave penetrates is dependent on the wavelength of the light, in practice it is approximately 50-300nm from the coverslip. Therefore, the cell membrane is…

5 Drool Worthy Imaging Advances Of 2020

5 Drool Worthy Imaging Advances Of 2020

By: Heather Brown-Harding, PhD

2020 was a difficult year for many, with their own research being interrupted- either by lab shutdowns or recruitment into the race against COVID-19. Despite the challenges, scientists have continued to be creative and have pushed the boundaries of what is possible. These are the techniques and technologies that every microscopist was envious of in 2020. Spatially Resolved Transcriptomics Nature Methods declared that spatially resolved transcriptomics was the 2020 method of the year. These are a  group of methods that combine gene expression with their physical location. Single-cell RNA sequencing (scRNAseq) was originally developed for cells that had been dissociated…

Getting A New Flow Cytometer? Try Before You Buy (And 2 Other Tips)

Getting A New Flow Cytometer? Try Before You Buy (And 2 Other Tips)

By: Tim Bushnell, PhD

There are 4 major ways to sort cells. The first way can use magnetic beads coupled to antibodies and pass the cells through a magnetic field. The labeled cells will stick, and the unlabeled cells will remain in the supernatant. The second way is to use some sort of mechanical force like a flapper or air stream that separates the target cells from the bulk population. The third way is the recently introduced microfluidics sorter, which uses microfluidics channels to isolate the target cells. The last method, which is the most common––based on Fuwyler’s work––is the electrostatic cell sorter. This…

5-Point Guide To Buying A New Microscope For Your Lab

5-Point Guide To Buying A New Microscope For Your Lab

By: Heather Brown-Harding, PhD

Have you ever noticed how painful it can be to purchase a new microscope? It would be hard to miss – this can be a frustrating process. A lot of scientists and students consider the new microscope hunt quite scary for a variety of reasons. It might be that you’re worried you won’t get the right microscope and that you’ll regret it, or you may find that dealing with salespeople, in general, makes you kind of uncomfortable. But remember, salespeople are just human beings like you and me, and if we can treat them as such, the whole process of…

Ask These 7 Questions Before Purchasing A Flow Cytometer

Ask These 7 Questions Before Purchasing A Flow Cytometer

By: Tim Bushnell, PhD

I am still convinced that my first cell sorter was possessed. The number of issues that I had with the system remains hard for me to believe, even after all these years. It had been purchased, in part, from one vendor because the sales rep for a competitor was nowhere to be found. At that time, I admit I wasn’t overly diligent in my research process. Since then, I’ve pinpointed some critical questions that need to be answered before purchasing a new instrument. At the end of the process, a shiny new instrument will arrive at your facility. Make sure…

How to Optimize Flow Cytometry Hardware For Rare Event Analysis

How to Optimize Flow Cytometry Hardware For Rare Event Analysis

By: Tim Bushnell, PhD

Preparing for rare event analysis requires an understanding of the power and limitation of the instrument to be used. From how fast to run the fluidics, to how the signal is processed to the number of gates that can be used in the sorting experiment, each factor impacts the outcome of the experiment.

3 Ways The ZE5 Cell Analyzer Accelerates Flow Cytometry Research Opportunities

3 Ways The ZE5 Cell Analyzer Accelerates Flow Cytometry Research Opportunities

By: Tim Bushnell, PhD

Some technological advances are incremental, while others are significant game-changing tools that offer the researcher the ability to significantly improve current assays while allowing for new and novel avenues of research to be performed. With speed, sensitivity, and capacity to spare, the ZE5 fits into the game-changing category. Reduced carryover, increased speed of acquisition, and a large number of parameters all open up new and novel assays while improving the quality and reproducibility of ongoing ones.

Top Technical Training eBooks

Get the Advanced Microscopy eBook

Get the Advanced Microscopy eBook

Heather Brown-Harding, PhD

Learn the best practices and advanced techniques across the diverse fields of microscopy, including instrumentation, experimental setup, image analysis, figure preparation, and more.

Get The Free Modern Flow Cytometry eBook

Get The Free Modern Flow Cytometry eBook

Tim Bushnell, PhD

Learn the best practices of flow cytometry experimentation, data analysis, figure preparation, antibody panel design, instrumentation and more.

Get The Free 4-10 Compensation eBook

Get The Free 4-10 Compensation eBook

Tim Bushnell, PhD

Advanced 4-10 Color Compensation, Learn strategies for designing advanced antibody compensation panels and how to use your compensation matrix to analyze your experimental data.