6 Tips For Applying The Right Statistical Test To Your Flow Cytometry Data

Flow cytometry data are numbers rich.

Data from experiments can be population measurements (percent of CD4+ cells, for example), or it can be expression level (median fluorescent expression of CD69 on activated T cells).

Many times, researchers are content to show histograms to illustrate their point after a flow experiment. This approach misses the opportunity to take that content rich data and extend the analysis into a statistical analysis.

To properly perform statistical analysis, the first step is to understand the hypothesis. The hypothesis will guide the statistical analysis, identifying the correct test to be performed. There are several things that need to be considered when beginning the statistical analysis of the data.

1.  Design your experiment properly from the start.

Statistical power answers the question of what is the probability of correctly rejecting the null hypothesis when the null hypothesis falls. There are three factors that influence the power of an experiment: the sample size, the spread of the data and the number of replicates. The power of the experiment is related to the ability of the experiment to avoid statistical errors.

2.  Know the classes of statistical errors and how to avoid them.

False positives (Type I errors) are when a true null hypothesis is incorrectly rejected. False negatives (Type II errors) are when the test fails to reject a false null hypothesis.

In fact, the power of the experiment is defined as the b which is equal to the True positive/(true positive + false negative)

3.  Use the appropriate statistical test.

The biological hypothesis and experimental design will determine what is the appropriate test for the data. The distribution of the data is also important to consider. How best to determine the correct test? This table can help you determine which test is most appropriate.

4.  Set the appropriate threshold.

The a value is the threshold that will be used to determine in the data is statistically significant or not. For historical reasons, this value is usually set at 0.05. This can be interpreted as the chance of finding significance where there is none (i.e. The chance of committing a Type I error).

5.  Avoid the more significant trap.

Once the a value is set, if the P-value is below that value, the data is statistically significant. The data is not more significant if the P-value is 0.01 and the threshold is 0.05 than if the P-value is 0.04. If there is an expectation, and a desire to decrease the Type I error, the threshold should be set to a more stringent level (0.01 or more).

6.   Avoid multiple pairwise comparisons.

In the case where the experimental design has Drug X, Drug Y and the combination of Drug X and Y, to be compared to an untreated sample, what is the best test? Pairwise comparisons should not be performed in this case for the following reason. With the a set to 0.05, there is a 5% change of committing a Type one error. With each comparison, the change of committing a Type I error increases, as showing in the chart below.

Number of pairwise comparisonsChanges of a Type I error
210%
315%
419%
523%

At the end of the day, the statistical analysis of your flow cytometry data is a critical step for proving the validity of the hypothesis that was being tested. With careful and considered approach to performing the correct testing, the published data will stand up to the rigors of peer review and help lead to another discovery.

Join Expert Cytometry's Mastery Class
Tim Bushnell, PhD
Tim Bushnell, PhD

Tim Bushnell holds a PhD in Biology from the Rensselaer Polytechnic Institute. He is a co-founder of—and didactic mind behind—ExCyte, the world’s leading flow cytometry training company, which organization boasts a veritable library of in-the-lab resources on sequencing, microscopy, and related topics in the life sciences.

Similar Articles

We Tested 5 Major Flow Cytometry SPADE Programs for Speed - Here Are The Results

We Tested 5 Major Flow Cytometry SPADE Programs for Speed - Here Are The Results

By: Tim Bushnell, PhD

In the flow cytometry community, SPADE (Spanning-tree Progression Analysis of Density-normalized Events) is a favored algorithm for dealing with highly multidimensional or otherwise complex datasets. Like tSNE, SPADE extracts information across events in your data unsupervised and presents the result in a unique visual format. Given the growing popularity of this kind of algorithm for dealing with complex datasets, we decided to test the SPADE algorithm in 5 software packages, including Cytobank, FCS Express, FlowJo, R, and the original, free software made available by the author of SPADE. Which was the fastest?

5 FlowJo Hacks To Boost The Quality Of Your Flow Cytometry Analysis

5 FlowJo Hacks To Boost The Quality Of Your Flow Cytometry Analysis

By: Tim Bushnell, PhD

FlowJo is a powerful tool for performing and analyzing flow cytometry experiments, if you know how to use it to the fullest. This includes understanding embedding and using keywords, the FlowJo compensation wizard, spillover spreading matrix, FlowJo and R, and creating tables in FlowJo. Extending your use of FJ using these hacks will help organize your data, improve analysis and make your exported data easier to understand and explain to others. Take a few moments and explore all you can do with FJ beyond just gating populations.

Statistical Challenges Of Rare Event Measurements In Flow Cytometry

Statistical Challenges Of Rare Event Measurements In Flow Cytometry

By: Tim Bushnell, PhD

It is necessary to sort through hundreds of thousands or millions of cells to find the few events of interest. With such low event numbers, we move away from the comfortable domain of the Gaussian distribution and move into the realm of Poisson statistics. There are 3 points to consider to build confidence in the data that the events being counted are truly events of interest and not random events that just happen to fall into the gates of interest.

How to Optimize Flow Cytometry Hardware For Rare Event Analysis

How to Optimize Flow Cytometry Hardware For Rare Event Analysis

By: Tim Bushnell, PhD

Preparing for rare event analysis requires an understanding of the power and limitation of the instrument to be used. From how fast to run the fluidics, to how the signal is processed to the number of gates that can be used in the sorting experiment, each factor impacts the outcome of the experiment.

How To Choose The Correct Antibody For Accurate Flow Cytometry Results

How To Choose The Correct Antibody For Accurate Flow Cytometry Results

By: Tim Bushnell, PhD

With the added emphasis on reproducibility, it is critical to look at every step where experiments can be improved. No single step makes an experiment more reproducible, rather it is a process, making changes at each stage that leads to reproducibility. Antibodies comprise a critical component that needs to be reviewed. As Bradbury et al. in a commentary in Nature pointed out, the global spending on antibodies is about $1.6 billion a year, and it is estimated about half of that money is spent on “bad” antibodies. This does not include the additional costs of wasted time and effort by…

How To Achieve Accurate Flow Cytometry Calcium Flux Measurements

How To Achieve Accurate Flow Cytometry Calcium Flux Measurements

By: Tim Bushnell, PhD

Dyes exist for the detection of everything from large nucleic acids to reactive oxygen species, and from lipid aggregates to small ions. Concentrations of physiologically important ions such as sodium, potassium, and calcium can be important indicators of health and disease. Calcium ions play an especially critical role in cellular signaling. As a signaling messenger, calcium is involved in everything from muscle contractions, to cell motility, to enzyme activity. Calcium experiments can be very informative, and with the advent of cheaper UV lasers, more and more researchers can use ratiometric measurements to evaluate the signaling processes in phenotypically defined populations.

How to Perform Doublet Discrimination In Flow Cytometry

How to Perform Doublet Discrimination In Flow Cytometry

By: Tim Bushnell, PhD

You are probably familiar with the term, “doublet discrimination” or “doublet exclusion”, and have likely included this flow cytometry measurement into at least some (if not all) of your gating strategies. Even though you may utilize this important gating strategy, you may not have had the chance to delve deeper to explore exactly what doublets are and why it’s critical to exclude them. This article aims to give you insight on the what, why, and how of doublet discrimination.

4 Considerations For Assessing Protein Phosphorylation Using Flow Cytometry

4 Considerations For Assessing Protein Phosphorylation Using Flow Cytometry

By: Tim Bushnell, PhD

For those working in the signaling field, having the ability to take a sample and phenotypically identify it, while knowing what is happening inside the cell to the target molecules of choice opens up a host of new opportunities. These assays are amenable to high throughput setup, meaning that biologically relevant outcomes in pre-clinical drug discovery can be measured directly. All told, with a little forethought, some careful planning and validation, and our helpful tips, phosphoflow assays are within your reach.

5 Essential Calculations For Accurate Flow Cytometry Results

5 Essential Calculations For Accurate Flow Cytometry Results

By: Tim Bushnell, PhD

Flow cytometry is a numbers game. There are percentages of a population, fluorescence intensity measurements, sample averages, data normalization, and more. Many of these common calculations are useful, but surrounded by misconceptions. This primer will help you decide which calculation to use, when to use it, and how to interpret the results.

Top Technical Training eBooks

Get the Advanced Microscopy eBook

Get the Advanced Microscopy eBook

Heather Brown-Harding, PhD

Learn the best practices and advanced techniques across the diverse fields of microscopy, including instrumentation, experimental setup, image analysis, figure preparation, and more.

Get The Free Modern Flow Cytometry eBook

Get The Free Modern Flow Cytometry eBook

Tim Bushnell, PhD

Learn the best practices of flow cytometry experimentation, data analysis, figure preparation, antibody panel design, instrumentation and more.

Get The Free 4-10 Compensation eBook

Get The Free 4-10 Compensation eBook

Tim Bushnell, PhD

Advanced 4-10 Color Compensation, Learn strategies for designing advanced antibody compensation panels and how to use your compensation matrix to analyze your experimental data.