3 Action Steps You Can Take Right Now To Improve Your Flow Cytometry Reproducibility

Reproducibility is a key issue in science.

Massive amounts of time and money are wasted when the results of experiments are not reproducible.

For example, I was called into a lab to look at their data because they had spent thousands of dollars sorting precious human samples and were now doing genomics analysis with the isolated cells.

Unfortunately, the results of the genomics analysis made no sense based on the sorted populations. The lab was working backward through every step of the process to try to identify what might have happened and if the experiments were salvageable.

As I reviewed the sorting process, one of the striking factors was that the quality control of the cell sorting experiments was very, very poor. In fact, it was non-existent.

So whoever was running their sorter was not performing quality control on the instrument, so the sorting results were all over the place. Voltages were changed dramatically for each experiment, and the separation of the target cells ranged from barley differentiated to well separated. The compensation controls told another story about the problems with these sorting experiments.

In the end, this lab wasted tens of thousands of dollars, countless man-hours, and precious lost samples because there was not a focus on quality control and best practices.

Reproducibility is a state of mind.

It’s not one simple thing that you do that will make all your data more reproducible, it a shift in the way one thinks about and perform experiments.

With the emphasis on rigor and reproducibility in science, it’s very important that researchers start putting into place everything they can do to help improve the quality and reproducibility of their data.

Here are 3 action steps that can be taken to enhance experimental reproducibility…

1. Evaluate your quality control processes to improve reproducibility.

Quality control is an important component of reproducibility.

That includes monitoring the quality control of the instrument by making sure that quality control metrics are being run on a daily basis.

You, the end user, have a right to ask to look at that data.

Don’t be afraid to go up to whoever’s running the machine and say, “Hey, how’s the quality of the machine? “Can I look at the QC data, and see how it’s going?”

Let them talk to you about it so you understand what it means so that you can get a better feel for what’s going on with your instrument.

Figure 1: Quality control tracking using beads.

You don’t want to sit down at a machine that is not working properly and be unaware of the systems limitations.

Your data will end up looking poor or worse – you might make erroneous conclusions because the machine wasn’t performing properly.

Quality control is done to make sure the machine performs consistently on a day-to-day basis.

That means that you, as the end user, should also be thinking about quality control of your experiments, it is not just the job of the core facility.

At a minimum, researchers can include a bead tandard as a way of monitoring quality control before running the experiment.

2. Develop the assay completely before performing the assay.

When sitting down and to develop an assay, it’s important to work through the whole process.

The first part of that process is understanding what the biology is and what the experiment is trying to prove.

Next, sketch out the proposed primary analysis – what will the gating strategy look like, and what data will be extracted for secondary analysis. If the experiment is a cell sorting experiment, understand what the downstream application that cells will be used for is, and the limitations of that assay.

It is also important to decide what statistical analysis will be performed and calculate the power of the experiment to determine how many samples will be needed. These steps will go a long way to prevent p-hacking and prevent HARKing.

With those steps completed, the next step is experimental design. That includes what instrument will be used, what antigens will be needed and build an initial panel. Reviewing the diagrams for analysis that were drawn above can help identify what are the critical targets. Those should be paired with bright fluorochromes and in channels that have a low error that will allow for a more sensitive measurement. With this in place, the next steps include testing the reagents (titration and voltage optimization). From there, comes the optimization process, where the best conditions for the assay are determined.

Figure 2: Error contribution based on detector and fluorochromes.

After optimization comes validation of the assay. This includes characterization of the necessary controls that will ensure identification of the target populations, demonstrate the stability of the assay, the variation within the staining process and the like. Once the assay is validated, the process is locked down, using Standard Operating Procedures.

3. Ensure you have quality SOPs in place.

For those who are working in a regulated environment, the SOP is part of the daily routine. For others, the idea of a protocol is more common. The biggest difference between the two documents is the level of document control and the expectation of how close the document is followed. With SOPs, they must be followed exactly, and deviations have to be noted and signed. SOPs are not changed without significant discussion and demonstration of the need for change. All details about the reagents are noted, serial numbers of equipment, lot numbers of reagents and more. The exacting SOP ensures that everyone performs the experiment the same way.

A protocol, on the other hand, lists the general steps to follow. It is often changed on the fly, based on the needs of the experiment at the time. It lists recommended controls, but not all may be necessary for the specific assay. The level of detail about reagents and more are not expected.

For those performing a longitudinal study or for a long period of time, developing and implementing an SOP will improve the rigor of the experiment.

An added benefit of the SOP is that it can be used to train additional researchers to assist in the experiment, as they will know exactly what the steps are and how to perform them.

Reproducibility is the name of the game, take a few minutes to think about how you can change your activities and your research workflow to increase the quality and consistency of your data. A few action steps that you can take right now to improve your data’s reproducibility is to Evaluate your quality control processes, develop the assay completely before performing the assay, and ensure you have quality SOPs in place. Taking those few simple steps will protect you from the problems associated with non-reproducible data.

To learn more about the 3 Action Steps You Can Take Right Now To Improve Your Flow Cytometry Reproducibility, and to get access to all of our advanced materials including 20 training videos, presentations, workbooks, and private group membership, get on the Flow Cytometry Mastery Class wait list.

Join Expert Cytometry's Mastery Class
Tim Bushnell, PhD
Tim Bushnell, PhD

Tim Bushnell holds a PhD in Biology from the Rensselaer Polytechnic Institute. He is a co-founder of—and didactic mind behind—ExCyte, the world’s leading flow cytometry training company, which organization boasts a veritable library of in-the-lab resources on sequencing, microscopy, and related topics in the life sciences.

Similar Articles

The Power Of Spectral Viewers And Their Use In Full Spectrum Flow Cytometry

The Power Of Spectral Viewers And Their Use In Full Spectrum Flow Cytometry

By: Tim Bushnell, PhD

What photon from yonder fluorochrome breaks?  It is … umm… hmmm. Let me see. Excitation off a 561 nm laser, with an emission maximum of 692 nm. I’m sure if Shakespeare was a flow cytometrist, he might have written that very scene. But the play is lost in time. However, since the protagonist had difficulty determining what fluorochrome was emitting photons, let’s consider how this could be figured out. In my opinion, one of the handiest flow cytometry tools is the spectral viewer. This tool helps visualize the excitation and emission profile of different fluorochromes, as well as allowing you…

Fickle Markers: Solutions For Antibody Binding Specificity Challenges

Fickle Markers: Solutions For Antibody Binding Specificity Challenges

By: Tim Bushnell, PhD

Reproducibility has been an ongoing, and important, concept in the sciences for years.  In the area of biomedical research, the alarm was sounded by several papers published in the early 2010’s.  Authors like Begley and Ellis, Prinz and coworkers, and Vasilevsky and colleagues, among others reported an alarming trend in the reproducibility of pre-clinical data.  These reports indicated between 50% to almost 90% of published pre-clinical data were not reproducible.  This was further highlighted in the article by Freedman and coworkers, who tried to identify and quantify the different sources of error that could be causing this crisis.  Figure 1,…

5 Common Flow Cytometry Questions, Answered

5 Common Flow Cytometry Questions, Answered

By: Tim Bushnell, PhD

I want to thank all of you who send us your questions about flow cytometry, so I thought I would dip into the old email bag and answer a few of the common ones here.  If your question isn’t answered this time, look for it to be answered in a future blog post.  Of course, if you want us to cover a specific topic, drop us a line.  1. How Fast Can I Go? This is  a common question. The allure of the ‘hi’ button is hard to resist.  The faster you go, the sooner you are finished with data…

Combining Flow Cytometry With Plant Science, Microorganisms, And The Environment

Combining Flow Cytometry With Plant Science, Microorganisms, And The Environment

By: Tim Bushnell, PhD

My first introduction to flow cytometry was talking to a professor who’d brought one on a research cruise to study phytoplankton. It was only later that I was introduced to the marvelous world that’s been my career for over 20 years.   In that time, I’ve had the opportunity to work with researchers in many different areas, exposing me to a wide variety of cell types and more important assays. What continues to amaze me is the number of different parameters we can measure, not just the number of fluorochromes, but the information we can extract from samples – animal, vegetable…

Common Numbers-Based Questions I Get As A Flow Cytometry Core Manager And How To Answer Them

Common Numbers-Based Questions I Get As A Flow Cytometry Core Manager And How To Answer Them

By: Tim Bushnell, PhD

Numbers are all around us.  My personal favorite is ≅1.618 aka ɸ aka ‘the golden ratio’.  It’s found throughout history, where it has influenced architects and artists. We see it in nature, in plants, and it is used in movies to frame shots. It can be approximated by the Fibonacci sequence (another math favorite of mine). However, I have not worked out how to apply this to flow cytometry.  That doesn’t mean numbers aren’t important in flow cytometry. They are central to everything we do, and in this blog, I’m going to flit around numbers-based questions that I have received…

3 Must-Have High-Dimensional Flow Cytometry Controls

3 Must-Have High-Dimensional Flow Cytometry Controls

By: Tim Bushnell, PhD

Developments such as the recent upgrade to the Cytobank analysis platform and the creation of new packages such as Immunocluster are reducing the computational expertise needed to work with high-dimensional flow cytometry datasets. Whether you are a researcher in academia, industry, or government, you may want to take advantage of the reduced barrier to entry to apply high-dimensional flow cytometry in your work. However, you’ll need the right experimental design to access the new transformative insights available through these approaches and avoid wasting the considerable time and money required for performing them. As with all experiments, a good design begins…

The Fluorochrome Less Excited: How To Build A Flow Cytometry Antibody Panel

The Fluorochrome Less Excited: How To Build A Flow Cytometry Antibody Panel

By: Tim Bushnell, PhD

Fluorochrome, antibodies and detectors are important. The journey of a thousand cells starts with a good fluorescent panel. The polychromatic panel is the combination of antibodies and fluorochromes. These will be used during the experiment to answer the biological question of interest. When you only need a few targets, the creation of the panel is relatively straightforward. It’s only when you start to get into more complex panels with multiple fluorochromes that overlap in excitation and emission gets more interesting.  FLUOROCHROMES Both full spectrum and traditional fluorescent flow cytometry rely on measuring the emission of the fluorochromes that are attached…

Flow Cytometry Year in Review: Key Changes To Know

Flow Cytometry Year in Review: Key Changes To Know

By: Meerambika Mishra

Here we are, at the end of an eventful year 2021. But with the promise of a new year 2022 to come. It has been a long year, filled with ups and downs. It is always good to reflect on the past year as we move to the future.  In Memoriam Sir Isaac Newton wrote “If I have seen further, it is by standing upon the shoulders of giants.” In the past year, we have lost some giants of our field including Zbigniew Darzynkiwicz, who contributed much in the areas of cell cycle analysis and apoptosis. Howard Shapiro, known for…

What Star Trek Taught Me About Flow Cytometry

What Star Trek Taught Me About Flow Cytometry

By: Tim Bushnell, PhD

It is no secret that I am a very big fan of the Star Trek franchise. There are many good episodes and lessons explored in the 813+ episodes, 12 movies (and counting). Don’t worry, this blog is not going to review all 813, or even 5 of them. Instead, some of the lessons I have taken away from the show that have applicability to science and flow cytometry.  “Darmok and Jalad at Tanagra.”  (ST:TNG season 5, episode 2) This is probably one of my favorite episodes, which involves Picard and an alien trying to establish a common ground and learn…

Top Technical Training eBooks

Get the Advanced Microscopy eBook

Get the Advanced Microscopy eBook

Heather Brown-Harding, PhD

Learn the best practices and advanced techniques across the diverse fields of microscopy, including instrumentation, experimental setup, image analysis, figure preparation, and more.

Get The Free Modern Flow Cytometry eBook

Get The Free Modern Flow Cytometry eBook

Tim Bushnell, PhD

Learn the best practices of flow cytometry experimentation, data analysis, figure preparation, antibody panel design, instrumentation and more.

Get The Free 4-10 Compensation eBook

Get The Free 4-10 Compensation eBook

Tim Bushnell, PhD

Advanced 4-10 Color Compensation, Learn strategies for designing advanced antibody compensation panels and how to use your compensation matrix to analyze your experimental data.