How To Improve Reproducibility Through The Automated Analysis Of Flow Cytometry Data

Editor’s Note:  Reproducibility continues to be a critical area that all researchers need to be aware of. From the NIH’s focus on reproducibility in grant applications, to a renewed focus by reviewers on the way data has been analyzed and presented, it is imperative that researchers keep up on best practices to ensure they pass these hurdles. 

One area that flow cytometry researchers should be focusing on is the emerging changes in the area of automated data analysis. Over the last five years there have been dramatic changes and improvements in these programs and workflows. As Dr. Brinkman discusses below, the automated analysis of flow cytometry data is coming into its own. 

Flow cytometry (FCM) datasets that are currently being generated will be two orders of magnitude larger than any that exist today, and new instruments, both flow and mass cytometry, have increased the number of parameters measured for each single cell by 50% (to 30).

Even in 14 dimensional datasets there are 16,384 possible cell populations of interest pre-sample (1). The information contained within large and complex single cell datasets can only be realized with approaches to effectively curate, integrate, analyze, interpret, and share these datasets.

What Is Reproducibility And Automated Analysis?

While there are many steps in the analysis pipeline that can benefit from automated approaches for which approaches have been developed (Figure 1), a major bottleneck in the analysis of flow cytometry data is in the identification of cell populations.

how to improve reproducibility | Expert Cytometry | flow cytometry data

Manual analytical techniques lack the capacity and rigour to bring out the full potential of signals latent in the data (1, 2) and its subjectivity has been identified to be the primary source of variation between analytic results (3, 4).

Figure 1: Typical flow cytometry automated analysis workflows.

Analysis usually starts with several pre-processing steps (blue boxes) followed by identification of cell populations of interest (orange boxes) and visualization.

To address this problem, the computational cytometry community has developed a collection of widely used approaches for the high throughput analysis of FCM and Mass Cytometry (CyTOF) (5). Methods have been extensively evaluated against manual analysis through the Flow Cytometry: Critical Assessment of Population Identification Methods (FlowCAP) project (6,7,8) and have been found to meet and in many cases exceed the performance of manual analysis.

Only by taking advantage of cutting-edge computational abilities will we be able to realize the full potential of data sets now being generated and be able to keep up with the quick rate of progress and advancement in our fields.

Further Reading (References hyperlinked above)…

  1. Aghaeepour N, Chattopadhyay PK, Ganesan A, O’Neill K, Zare H, Jalali A, … Brinkman, RR. Early immunologic correlates of HIV protection can be identified from computational analysis of complex multivariate T-cell flow cytometry assays. Bioinformatics 2012, 28(7):1009-16.
  2. O’Neill K, Aghaeepour N, Špidlen J, Brinkman RR. Flow cytometry bioinformatics. PLoS Comput Biol 2013. 9(12):e1003365.
  3. Maecker H, Rinfret A, D’Souza P, Darden J, Roig E, Landry C, … Sekaly R. Standardization of cytokine flow cytometry assays. BMC Immunol 2005. 6:13.
  4. Qiu P, Simonds E, Bendall S, Gibbs KJ, Bruggner R, Linderman M, … Plevritis S. Extracting a cellular hierarchy from high-dimensional cytometry data with SPADE. Nat Biotechnol 2011. 29:886-91.
  5. Kvistborg P, Gouttefangeas C, Aghaeepour N, Cazaly A, Chattopadhyay PK, Chan C, … Maurer D. Thinking Outside the Gate: Single-Cell Assessments in Multiple Dimensions. Immunity 2015. 42(4):591-92.
  6. Aghaeepour N, Chattopadhyay P, Chikina M, Dhaene T, Van Gassen S, Kursa M, …, Brinkman RR. A benchmark for evaluation of algorithms for identification of cellular correlates of clinical outcomes. Cytometry 2016, 89(1):16-21.
  7. Aghaeepour N, Finak G, TheFlowCAPConsortium, TheDREAMConsortium, Hoos H, Mosmann T, … Scheuermann RH. Critical assessment of automated flow cytometry data analysis techniques. Nature Methods 2013. 10(3):228-238.
  8. Finak G, Langweiler M, Jaimes M, Malek M, Taghiyar J, Korin Y, … McCoy J. Standardizing Flow Cytometry Immunophenotyping Analysis from the Human ImmunoPhenotyping Consortium. Scientific Reports 2016. 6:20686.

As mentioned above, FCM datasets will soon be two orders of magnitude larger than those that exist today. As such, researchers must keep up on best practices for data reproducibility, especially in the area of automated data analysis. This will ensure that the field of flow cytometry and scientific research overall maintains its integrity while continuing to advance rapidly.

To learn more about how to improve reproducibility through automated analysis, and to get access to all of our advanced materials including 20 training videos, presentations, workbooks, and private group membership, get on the Flow Cytometry Mastery Class wait list.

Join Expert Cytometry's Mastery Class
Tim Bushnell, PhD
Tim Bushnell, PhD

Tim Bushnell holds a PhD in Biology from the Rensselaer Polytechnic Institute. He is a co-founder of—and didactic mind behind—ExCyte, the world’s leading flow cytometry training company, which organization boasts a veritable library of in-the-lab resources on sequencing, microscopy, and related topics in the life sciences.

Similar Articles

How To Do Variant Calling From RNASeq NGS Data

How To Do Variant Calling From RNASeq NGS Data

By: Deepak Kumar, PhD

Developing variant calling and analysis pipelines for NGS sequenced data have become a norm in clinical labs. These pipelines include a strategic integration of several tools and techniques to identify molecular and structural variants. That eventually helps in the apt variant annotation and interpretation. This blog will delve into the concepts and intricacies of developing a “variant calling” pipeline using GATK. “Variant calling” can also be performed using tools other than GATK, such as FREEBAYES and SAMTOOLS.  In this blog, I will walk you through variant calling methods on Illumina germline RNASeq data. In the steps, wherever required, I will…

Understanding Clinical Trials And Drug Development As A Research Scientist

Understanding Clinical Trials And Drug Development As A Research Scientist

By: Deepak Kumar, PhD

Clinical trials are studies designed to test the novel methods of diagnosing and treating health conditions – by observing the outcomes of human subjects under experimental conditions.  These are interventional studies that are performed under stringent clinical laboratory settings. Contrariwise, non-interventional studies are performed outside the clinical trial settings that provide researchers an opportunity to monitor the effect of drugs in real-life situations. Non-interventional trials are also termed observational studies as they include post-marketing surveillance studies (PMS) and post-authorization safety studies (PASS). Clinical trials are preferred for testing newly developed drugs since interventional studies are conducted in a highly monitored…

How To Profile DNA And RNA Expression Using Next Generation Sequencing (Part-2)

How To Profile DNA And RNA Expression Using Next Generation Sequencing (Part-2)

By: Deepak Kumar, PhD

In the first blog of this series, we explored the power of sequencing the genome at various levels. We also dealt with how the characterization of the RNA expression levels helps us to understand the changes at the genome level. These changes impact the downstream expression of the target genes. In this blog, we will explore how NGS sequencing can help us comprehend DNA modification that affect the expression pattern of the given genes (epigenetic profiling) as well as characterizing the DNA-protein interactions that allow for the identification of genes that may be regulated by a given protein.  DNA Methylation Profiling…

How To Profile DNA And RNA Expression Using Next Generation Sequencing

How To Profile DNA And RNA Expression Using Next Generation Sequencing

By: Deepak Kumar, PhD

Why is Next Generation Sequencing so powerful to explore and answer both clinical and research questions. With the ability to sequence whole genomes, identifying novel changes between individuals, to exploring what RNA sequences are being expressed, or to examine DNA modifications and protein-DNA interactions occurring that can help researchers better understand the complex regulation of transcription. This, in turn, allows them to characterize changes during different disease states, which can suggest a way to treat said disease.  Over the next two blogs, I will highlight these different methods along with illustrating how these can help clinical diagnostics as well as…

What Is Next Generation Sequencing (NGS) And How Is It Used In Drug Development

What Is Next Generation Sequencing (NGS) And How Is It Used In Drug Development

By: Deepak Kumar, PhD

NGS methodologies have been used to produce high-throughput sequence data. These data with appropriate computational analyses facilitate variant identification and prove to be extremely valuable in pharmaceutical industries and clinical practice for developing drug molecules inhibiting disease progression. Thus, by providing a comprehensive profile of an individual’s variome — particularly that of clinical relevance consisting of pathogenic variants — NGS helps in determining new disease genes. The information thus obtained on genetic variations and the target disease genes can be used by the Pharma companies to develop drugs impeding these variants and their disease-causing effect. However simple this may allude…

7 Key Image Analysis Terms For New Microscopist

7 Key Image Analysis Terms For New Microscopist

By: Heather Brown-Harding, PhD

As scientists, we need to perform image analysis after we’ve acquired images in the microscope, otherwise, we have just a pretty picture and not data. The vocabulary for image processing and analysis can be a little intimidating to those new to the field. Therefore, in this blog, I’m going to break down 7 terms that are key when post-processing of images. 1. RGB Image Images acquired during microscopy can be grouped into two main categories. Either monochrome (that can be multichannel) or “RGB.” RGB stands for red, green, blue – the primary colors of light. The cameras in our phones…

We Tested 5 Major Flow Cytometry SPADE Programs for Speed - Here Are The Results

We Tested 5 Major Flow Cytometry SPADE Programs for Speed - Here Are The Results

By: Tim Bushnell, PhD

In the flow cytometry community, SPADE (Spanning-tree Progression Analysis of Density-normalized Events) is a favored algorithm for dealing with highly multidimensional or otherwise complex datasets. Like tSNE, SPADE extracts information across events in your data unsupervised and presents the result in a unique visual format. Given the growing popularity of this kind of algorithm for dealing with complex datasets, we decided to test the SPADE algorithm in 5 software packages, including Cytobank, FCS Express, FlowJo, R, and the original, free software made available by the author of SPADE. Which was the fastest?

5 FlowJo Hacks To Boost The Quality Of Your Flow Cytometry Analysis

5 FlowJo Hacks To Boost The Quality Of Your Flow Cytometry Analysis

By: Tim Bushnell, PhD

FlowJo is a powerful tool for performing and analyzing flow cytometry experiments, if you know how to use it to the fullest. This includes understanding embedding and using keywords, the FlowJo compensation wizard, spillover spreading matrix, FlowJo and R, and creating tables in FlowJo. Extending your use of FJ using these hacks will help organize your data, improve analysis and make your exported data easier to understand and explain to others. Take a few moments and explore all you can do with FJ beyond just gating populations.

Statistical Challenges Of Rare Event Measurements In Flow Cytometry

Statistical Challenges Of Rare Event Measurements In Flow Cytometry

By: Tim Bushnell, PhD

It is necessary to sort through hundreds of thousands or millions of cells to find the few events of interest. With such low event numbers, we move away from the comfortable domain of the Gaussian distribution and move into the realm of Poisson statistics. There are 3 points to consider to build confidence in the data that the events being counted are truly events of interest and not random events that just happen to fall into the gates of interest.

Top Technical Training eBooks

Get the Advanced Microscopy eBook

Get the Advanced Microscopy eBook

Heather Brown-Harding, PhD

Learn the best practices and advanced techniques across the diverse fields of microscopy, including instrumentation, experimental setup, image analysis, figure preparation, and more.

Get The Free Modern Flow Cytometry eBook

Get The Free Modern Flow Cytometry eBook

Tim Bushnell, PhD

Learn the best practices of flow cytometry experimentation, data analysis, figure preparation, antibody panel design, instrumentation and more.

Get The Free 4-10 Compensation eBook

Get The Free 4-10 Compensation eBook

Tim Bushnell, PhD

Advanced 4-10 Color Compensation, Learn strategies for designing advanced antibody compensation panels and how to use your compensation matrix to analyze your experimental data.