Instrument Quality Control For Reproducible Flow Cytometry Experiments

The NIH has released a series of reproducibility guidelines that scientists must address. These guidelines have been introduced because there’s a lot of data showing that reproducibility in science is frustratingly lacking in certain experiments.

Non-reproducible experiments waste time, money, and resources. As Begley and Ioannidis cite in their 2015 article :

“The estimates for irreproducibility based on these empirical observations range from 75% to 90%. These estimates fit remarkably well with estimates of 85% for the proportion of biomedical research that is wasted at-large.”

Reproducibility is a mindset, and involves an overall analysis of the scientific process to identify the areas that can be improved. In this article by Bruce Booth, he reviews “Begley’s six rules”. Two of these rules focus on the controls and reagents used in the experiment.

Of equal importance are the instruments used to make measurements. For example, how often are the pipettes calibrated? Are all lab members adequately trained in technique? This chart from Gilson is a useful one to have in the lab as a reminder on proper pipetting form.

The flow cytometer is an integral component of any flow cytometry experiment, and special attention should be paid to ensuring that it is working correctly and consistently. As an end-user, the researcher should be able to sit down at a machine and know that it is performing the same way today as it was yesterday and last week.

Equally important is that if any changes in instrument performance have occured, the end-user knows how they have been addressed and corrected, rather than letting them fester and potentially affect the results.

For those using core facility equipment, it is important to talk to the people who maintain your instruments and look at the quality control data.

Ask them how are they are assessing and maintaining the quality of the instrument and ask what is the best way for you to make sure your data is consistent. The staff will be delighted to to advise! Quality control measurements can include a variety of targets, such as PMT sensitivity, laser alignment, fluidic stability, background issues, and more.

1. PMT Voltage Optimization.

With the analog flow cytometers, researchers were taught to run an unstained cell sample and adjust voltages so that the background was in the first log decade. This outdated practice continues to be taught, even on new digital instruments.

A better approach is to have optimized PMT voltage settings, and to use these voltages for experiments.

In 2006, Maecker and Trotter published a paper on how to determine the optimal voltage for a PMT. In this case, the second peak of the Spherotech 8-peak Rainbow Calibration Particles (RCP-3-5A-2) is run over a voltage range for each PMT. The robust coefficient of variance (rCV) is calculated for each PMT at each voltage, and a curve is plotted, as shown in the figure below.

Figure 1: Peak-2 characterization of PMTs.

The resulting curve shows a steep descent where the rCV decreases as voltage increases. Eventually, an inflection point is reached where the slope of the line changes. At this point, increasing voltage doesn’t improve the rCV.

The optimal voltage range is defined as the region just after the inflection point. In the case of Figure 1, this is between 400 to 450 volts.

When running a one-off experiment, set the voltage for the detectors to these voltages and begin collecting data. There are very few reasons to deviate from this starting voltage.

The most common situation is if the fluorescent signal on your target cells is out of the linear range of the detector, or off-scale. In this case, one might consider reducing the PMT voltage, but a better approach would be to reduce the fluorescence intensity by staining with a mix of labeled and unlabeled antibody.

When doing this, you keep the total antibody concentration constant, but now the cells will have a lower total fluorescence due to the presence of unlabeled antibody.

However, peak-2 beads are not cells, and don’t necessarily have the same spectral characteristics as the fluorochromes used in the experiment.

If the panel is going to be used consistently over a long period of time, the optimal voltage can be refined by performing a voltage titration (voltration) experiment using stained cells.

An example of this secondary optimization is shown in Figure 2. On the left, is the optimization for CY7-APC and on the right, brilliant violet 650. The peak-2 value for each of these fluorochromes is pointed out with the blue arrow. However, this peak-2 value is not always the optimal voltage for a specific fluorochrome.

Figure 2: Voltage optimization of 2 fluorochromes.

To determine if the peak-2 value is optimal, increase the voltage past this point and calculate the staining index. In the case of CY7-APC, as voltage is increased, the signal very rapidly moved out of the linear dynamic range, as shown by the red arrow, and eventually moved completely off scale. This indicates that the peak-2 value is best for this fluorochrome.

Brilliant Violet 650, on the other hand, shows an increase in the staining index above the peak-2 value. At the optimal voltage for this fluorochrome-detector pair, the SI is about 16% higher than at the peak-2 value, suggesting that this higher voltage is better for making a more sensitive measurement.

Once this voltration is performed, and the optimal voltage identified, it becomes important to maintain performance consistency over time. To do this, another bead is used. In this case, the sixth peak of the 8-peak Rainbow Calibration Particles, or any other bright bead of your choice.

Run the bead at the optimal voltages and determine the target values in each detector. This is typically measured as the median fluorescent intensity (MFI). Each time the experiment is to be run, the calibration bead is run first and detector voltages are adjusted to ensure that the MFI is maintained, within some tolerance range.

Once that is done, begin collecting the rest of the tubes. If the voltage changes are significant, it is good to pause and consult the daily QC, or your core staff, to determine if there has been a change to the system.

Defining “significant” is up the investigator, but a good rule of thumb would be that deviation of more than 10% in voltage should require some investigation. You should collect and record at least 10,000 single beads for your QC tracking.

It is important to remember that Quality Control like this is not valuable unless it is monitored. To do that, use the Levey-Jennings plot. This is a quality control plot that shows the running average of the data with control lines. In the case below, the dotted lines represent +/- 1 and 2 standard deviations around the mean.

This plot can be used to spot trends in the data. For example, if a data point falls outside of the control lines, it is a sign there may be an issue with the system, and intervention may be required.

This could just be cleaning, but may also indicate something else that the manager of the instrument needs to know. If, over time, there is a trend towards one direction or the other (continued increase and/or decrease), it is also a good time to intervene, if for no other reason than to see if other issues with the system have occured.

Figure 3: Levey-Jennings plot over time tracking PMT voltage changes using peak-6 bead method.

As shown in Figure 3, the data on this instrument over 100 days within 2 standard deviations of the mean, giving added confidence that the instrument is working consistently and changes in the data are more likely due to biology, and not instrument issues.

If these start to fall out of this range, then you would need to investigate and troubleshoot. You can be more restrictive and use only one standard deviation, but the industry standard is to use two.

While instrument QC is usually left to the manager of the instrument, adding experimental QC using an independent measurement will aid in improving the quality, and ultimately reproducibility, of the data being generated. Something may change on the instrument between when the manager checks in the morning and when you sit down to run, so it’s wise to check performance for yourself.

2. Parallel Arrangement and Fluidics Stability.

Most of the current instruments have a parallel laser arrangement. This means that the cell passes through lasers in sequence, so they are exposed to only a single excitation source at a time. To ensure that the correct signal pulses are paired together, a factor called the “time delay” is calculated during QC.

If the time delay is off, this will adversely impact the data as the cells arrive either too early or too late, thus the correct pluses are not paired together.

This can easily be identified retrospectively, but it is better to identify the issues during acquisition. If there is a loss of signal on a given excitation source, it can suggest a clog in the system that needs to be cleaned.

Consider a partial clog on the input side of the cytometer (before the flow cell). When the diameter of a pipe is reduced, that increases the flow (speed) of the liquid in the reduced diameter. This results in cells arriving at the interrogation point more quickly than the time delay is expecting them.

A clog after the flow cell has the same issue, but now the pressure builds up behind the clog, slowing down the stream, and cells arrive after the time delay window. This can be seen in Figure 4.

Figure 4: Impact of clogs on data.

When running your experiments, put up a plot of a bright fluorochrome versus time for each of the lasers that are being used on the machine. This will help determine if there are problems with the fluidics, as well as with the lasers.

In the example shown here, PE-Cy5.5 was being excited by a green laser which was the fourth in line, but there was a clog causing a back pressure issue. The clog caused a speedup of the number of events because it constricted the flow core and increased the flow rate in the number of events.

Eventually, the clog worked itself out and the signal came back. As with any quality control metric used, it’s important to not only run theses controls, but to look at them on a regular basis.

3. Compensation.

Compensation is one of most important steps in the flow cytometry process. For proper compensation it is critical to bring the right controls to the instrument every time. This is such an important topic, it has been discussed extensively here. It is here in discussing instrument reproducibility as compensation controls also give insight into how the system is working, and provide a secondary proxy for instrument QC.

These compensation controls must follow the 3 commandments:

1. Controls need to be at least as bright as any sample to which you’ll apply to your compensation. That means, the control signal needs to be at least as bright as anything in your experimental signal. If it’s not as bright, then your compensation will be incorrect. It also has to be within the linear range of the detector.

2. The background fluorescence should be the same for the positive and negative control populations for any given parameter.

Compensation is a property of the fluorochrome, not a property of the antibody or a property of the carrier that brings the fluorochrome to the intercept point. This is why you can use either beads or cells. What’s important is that the background fluorescence, the autofluorescence of the negative population and a positive are the same. Since compensation corrects to the background of the control, it is critical to have a positive and negative in each control. This means it is possible to mix beads and cells in a compensation matrix as long as each positive is linked to the appropriate negative particle type.

3. The compensation colors must be exactly matched to your experimental color.
Alexa 488 cannot be used to compensate for GFP or for FITC. Likewise, if there are two different lots of tandem dye, make sure that lots are matched in the experiment and control. One lot of PE-Cy5 cannot be used to compensate a different lot of PE-Cy5 and vice versa, because each of these dyes are manufactured, and likely have different spectral characteristics.

BONUS 4. You need to have enough events!

This is a minimum of 10,000 single events, when using beads. When using cells, you need a minimum of 30,000 single cell events. Collecting more is good, but don’t collect less than these minimums.

Since fluorescence is a property of the fluorochrome, not the carrier, the question arises, “Should you use beads or should you use cells?” Either carrier is fine, as long as the positive populations have the same background fluorescence when unstained — i.e. if you use beads, then make sure you have unstained beads and if you use cells, make sure you have unstained cells. If you have some on beads and some on cells, then make sure you pair each positive with an appropriate negative.

4. Quality Control.

Often, the QC of an instrument is left to the manager of the instrument. This can be done in a variety of ways, from an instrument-specific QC protocol or something from the manufacturer, like the BD CS&T or the Attune performance tracking beads.

This process should be run every time the instrument is turned on, to ensure that a library of instrument behaviour is developed and to allow the users to understand how the system is performing. QC of this nature is critical best practices for the instrument, and provides baseline confirmation the system is working adequately.

Don’t hesitate to ask the manager of the instrument to look at their QC metrics, and ask for an explanation when things don’t seem clear. After all, the success or failure of the experiment lies in the hands of how well the flow cytometer is working, and most cytometrists will be overjoyed to talk shop!

In conclusion, one key area to improve consistency and reproducibility is to monitor instrument quality control. Knowing how the system is QCed and what happens when something goes wrong is paramount to ensuring that the instrument is properly characterized.

To learn more about Instrument Quality Control For Reproducible Flow Cytometry Experiments, and to get access to all of our advanced materials including 20 training videos, presentations, workbooks, and private group membership, get on the Flow Cytometry Mastery Class wait list.

Join Expert Cytometry's Mastery Class
Tim Bushnell, PhD
Tim Bushnell, PhD

Tim Bushnell holds a PhD in Biology from the Rensselaer Polytechnic Institute. He is a co-founder of—and didactic mind behind—ExCyte, the world’s leading flow cytometry training company, which organization boasts a veritable library of in-the-lab resources on sequencing, microscopy, and related topics in the life sciences.

Similar Articles

The Power Of Spectral Viewers And Their Use In Full Spectrum Flow Cytometry

The Power Of Spectral Viewers And Their Use In Full Spectrum Flow Cytometry

By: Tim Bushnell, PhD

What photon from yonder fluorochrome breaks?  It is … umm… hmmm. Let me see. Excitation off a 561 nm laser, with an emission maximum of 692 nm. I’m sure if Shakespeare was a flow cytometrist, he might have written that very scene. But the play is lost in time. However, since the protagonist had difficulty determining what fluorochrome was emitting photons, let’s consider how this could be figured out. In my opinion, one of the handiest flow cytometry tools is the spectral viewer. This tool helps visualize the excitation and emission profile of different fluorochromes, as well as allowing you…

3 Must-Have High-Dimensional Flow Cytometry Controls

3 Must-Have High-Dimensional Flow Cytometry Controls

By: Tim Bushnell, PhD

Developments such as the recent upgrade to the Cytobank analysis platform and the creation of new packages such as Immunocluster are reducing the computational expertise needed to work with high-dimensional flow cytometry datasets. Whether you are a researcher in academia, industry, or government, you may want to take advantage of the reduced barrier to entry to apply high-dimensional flow cytometry in your work. However, you’ll need the right experimental design to access the new transformative insights available through these approaches and avoid wasting the considerable time and money required for performing them. As with all experiments, a good design begins…

The Fluorochrome Less Excited: How To Build A Flow Cytometry Antibody Panel

The Fluorochrome Less Excited: How To Build A Flow Cytometry Antibody Panel

By: Tim Bushnell, PhD

Fluorochrome, antibodies and detectors are important. The journey of a thousand cells starts with a good fluorescent panel. The polychromatic panel is the combination of antibodies and fluorochromes. These will be used during the experiment to answer the biological question of interest. When you only need a few targets, the creation of the panel is relatively straightforward. It’s only when you start to get into more complex panels with multiple fluorochromes that overlap in excitation and emission gets more interesting.  FLUOROCHROMES Both full spectrum and traditional fluorescent flow cytometry rely on measuring the emission of the fluorochromes that are attached…

Flow Cytometry Year in Review: Key Changes To Know

Flow Cytometry Year in Review: Key Changes To Know

By: Meerambika Mishra

Here we are, at the end of an eventful year 2021. But with the promise of a new year 2022 to come. It has been a long year, filled with ups and downs. It is always good to reflect on the past year as we move to the future.  In Memoriam Sir Isaac Newton wrote “If I have seen further, it is by standing upon the shoulders of giants.” In the past year, we have lost some giants of our field including Zbigniew Darzynkiwicz, who contributed much in the areas of cell cycle analysis and apoptosis. Howard Shapiro, known for…

What Star Trek Taught Me About Flow Cytometry

What Star Trek Taught Me About Flow Cytometry

By: Tim Bushnell, PhD

It is no secret that I am a very big fan of the Star Trek franchise. There are many good episodes and lessons explored in the 813+ episodes, 12 movies (and counting). Don’t worry, this blog is not going to review all 813, or even 5 of them. Instead, some of the lessons I have taken away from the show that have applicability to science and flow cytometry.  “Darmok and Jalad at Tanagra.”  (ST:TNG season 5, episode 2) This is probably one of my favorite episodes, which involves Picard and an alien trying to establish a common ground and learn…

5 Flow Cytometry Strategies That Sun Tzu Taught Me

5 Flow Cytometry Strategies That Sun Tzu Taught Me

By: Tim Bushnell, PhD

Sun Tzu was a Chinese general and philosopher. His most famous writing is ‘The Art of War’, and has been studied by generals and CEOs, to glean ideas and strategies to help their missions. I was recently rereading this work and thought to myself if any of Sun Tzu’s lessons could apply to flow cytometry.  So I have identified 5 points that I think lend themselves to thinking about flow cytometry.  “Quickness is the essence of the war.” In flow cytometry, speed is of the essence. The longer the cells are out of their natural environment, the less happy they…

A Basic Guide To Flow Cytometry (3 Foundational Concepts)

A Basic Guide To Flow Cytometry (3 Foundational Concepts)

By: Meerambika Mishra

Mastering foundational concepts are imperative for successfully using any technique or system.  Robert Heinlein introduced the term ‘Grok’  in his novel Stranger in a Strange Land. Ever since then it has made its way into popular culture. To Grok something is to understand it intuitively, fully. As a cytometrist, there are several key concepts that you must grok to be successful in your career. These foundational concepts are the key tools that we use day in and day out to identify and characterize our cells of interest.  Cells Flow cytometry measures biological processes at the whole cell level. To do…

Which Fluorophores To Use For Your Microscopy Experiment

Which Fluorophores To Use For Your Microscopy Experiment

By: Heather Brown-Harding, PhD

Fluorophore selection is important. I have often been asked by my facility users which fluorophore is best suited for their experiments. The answer to this is mostly dependent on whether they are using a widefield microscope with set excitation/emission cubes or a laser based system that lets you select the laser and the emission window. Once you have narrowed down which fluorophores you can excite and collect the correct emission, you can further refine the specific fluorophore that is best for your experiment.  In this blog  we will discuss how to determine what can work with your microscope, and how…

4 No Cost Ways To Improve Your Microscopy Image Quality

4 No Cost Ways To Improve Your Microscopy Image Quality

By: Heather Brown-Harding, PhD

Image quality is critical for accurate and reproducible data. Many people get stuck on the magnification of the objective or on using a confocal instead of a widefield microscope. There are several other factors that affect the image quality such as the numerical aperture of the objective, the signal-to-noise ratio of the system, or the brightness of the sample.  Numerical aperture is the ability of an objective to collect light from a sample, but it contributes to two key formulas that will affect your image quality. The first is the theoretical resolution of the objective. It is expressed with the…

Top Technical Training eBooks

Get the Advanced Microscopy eBook

Get the Advanced Microscopy eBook

Heather Brown-Harding, PhD

Learn the best practices and advanced techniques across the diverse fields of microscopy, including instrumentation, experimental setup, image analysis, figure preparation, and more.

Get The Free Modern Flow Cytometry eBook

Get The Free Modern Flow Cytometry eBook

Tim Bushnell, PhD

Learn the best practices of flow cytometry experimentation, data analysis, figure preparation, antibody panel design, instrumentation and more.

Get The Free 4-10 Compensation eBook

Get The Free 4-10 Compensation eBook

Tim Bushnell, PhD

Advanced 4-10 Color Compensation, Learn strategies for designing advanced antibody compensation panels and how to use your compensation matrix to analyze your experimental data.