Skip to content

5 Steps For Accurate Flow Cytometry Statistical Analysis Results

Written by Tim Bushnell, PhD

At the end of many experiments, the question of statistical analysis rears its ugly head. When it comes up, many researchers freeze not knowing how to proceed, and they muddle through as best they can. With some proper planning and forethought, this doesn’t have to be the case.

To resolve this analysis dilemma, it is important to begin thinking about the statistical analysis during the initial designing of the experiments.

This is where some critical decisions need to be made to ensure that, if there are statistically significant findings to be uncovered, the data will be sufficient to support them.

During initial experiment design, consider the following…

1. Power the flow cytometry experiment properly.

Simply put, the statistical power of an experiment is the likelihood that the experiment will detect an effect if there is one to be measured.

The higher the experiment is powered, the lower the chance of making a Type II (false negative) statistical error.

There are a variety of calculators out there and one of the most useful is Statmate, from GraphPad software. Although those using OSX are out of luck with the package, it is a great program to add to your toolkit.

To use this tool, one needs to determine the statistical test to be performed, the threshold (α), and the standard deviation. With this information, a table such as the one shown below is generated.

Output from statmate

Figure 1: Output from Statmate.

Across the top are columns for different power, and down the side are the number of samples per group.

The numbers in blue represent the difference between the means between the control and experimental sample.

To use the chart, calculate the difference between the experimental and control means, and consult the blue numbers under the column of the power for the experiment.

Cross-referencing the blue number under the power to the number of samples per group will give you the number of samples needed to be run for the appropriate power.

Setting the power at the beginning of the experiment is the best practice and will provide the researcher with confidence on the possibility of finding a statistically significant result if there is one to be discovered.

2. Establish the threshold (significance level).

The threshold, denoted as the α, is the probability level below which the null hypothesis will be rejected.

This value is historically set at 0.05%.

This value is also related to the possibility of making a Type I (false positive) statistical error and is based on work by Roland Fisher from the 1920s, which he suggested was a “convenient cut-off to reject the null hypothesis.”

Consider the normal distribution, as shown below. If a two-tailed T-test is performed on the data, with a threshold of 0.05%, this is distributed evenly above and below the mean. The white areas represent that 5%.

Flow Cytometry chart showing normal distribution

Figure 2: From used under the GNU Free Documentation License.

The P value will be compared to the α to determine if the the null hypothesis can be rejected or not.

Since the α is a measure of committing a Type I error, the consequences of a false positive must be considered when establishing the threshold.

A higher threshold makes it easier to find significance, but increases the possibility of the Type I error.

Lower the threshold and it decreases the possibility of a Type I error. Thus, setting the threshold should be considered based on the specific conditions of the test.

Setting the threshold at the beginning of the experiment is a best practice, as it helps establish the probability of committing a Type I error.

3. Clearly state the hypothesis.

At the beginning of the experimental planning, it is critical to understand what the hypothesis being tested is.

If the hypothesis is poorly stated, the rest of the statistical analysis will be inaccurate, or as it is said ‘Garbage In, Garbage Out.’

Since the hypothesis will be used to establish the null hypothesis (H0), this becomes the most important step in the process, as it forms the basis of why the experiments are being performed.

For example, when asked to determine if a new drug, Pescaline D, increases the number of CD4+ T-cells in patients with Bowden’s malady, one can design an appropriate experiment.

The null hypothesis for this experiment could be stated as: Pescaline D causes no change to the percent of CD4+ cells in patients suffering from Bowden’s Malady.

Setting the null hypothesis at the beginning of the experiment will assist in the design of the experiment, help evaluate the best controls to use, and guide the direction of the statistical test.

4. Choose the correct test.

The statistical test should be identified at the beginning of the experiment.

Based on the null hypothesis, the correct testing method should be clear.

Some common statistical tests, and when they should be used, are listed here:

Common Flow Cytometry statistical tests

Figure 3: Suggested statistical testing. A more complete list can be found here:

This choice will influence the data that is extracted from the primary analysis.

In the case of the example above, the data would be the percentage of CD4+ cells between the control (untreated) and experimental (treated).

This would be tested using an unpaired T-Test. However, if the experimental design was to take a sample before treatment (control) and treat the patient (experimental), one would perform a paired T-Test.

For those performing T-tests, another consideration is whether to do the test as a ‘one-tailed’ or ‘two-tailed’ T-test.

This is another consideration that has to be made before the experiments are performed.

If the expected change is in one direction — that is, there will be an increase or a decrease — then a one-tailed T-test is appropriate.

On the other hand, if it is not known which direction the change will occur, a two-tailed T-test is the best test to choose. This is defined at the beginning of the experiment to avoid the desire to look at the data and choose a one- or two- tailed T-test at the end.

Choosing the appropriate statistical test at the beginning of the experimental design process is the best practice to prevent bias.

This will ensure that there is no experimenter bias introduced after the data is collected and will also ensure the correct data is extracted from the primary analysis.

5. Know how to plot your data and do it first.

Although it may sound strange, it is very valuable to plot your data before you move forward with your statistical analysis.

Back in 1973, the statistician Francis Anscombe published the now famous Anscombe’s quartet:

The Anscombe Quartet

Figure 4: Anscombe’s quartet. From: used under GNU General Public License.

These four datasets are statistically identical: including the mean, the sample variance, the regression line, and the correlation coefficient.

Anscombe published this dataset when many researchers were starting to use computers for their statistical analysis, and entering data without graphing it. This dataset was designed to point out the fact that graphing data is a critical first step and an important check on the researcher as well. If something looks odd, it may be odd.

When plotting data, it is good practice to use a plot that shows all the data points.

Take a look at these two graphs.

Percentage of CD4+ cells before (U) and after (T) treatment

Figure 5: Percentage of CD4+ cells before (U) and after (T) treatment.

The bar graph on the left shows the same data as the scatter graph on the right.

The difference is that with the scatter graph, it is possible to see that there are two different levels of response in the data after treatment, which is lost in the bar graph. Thus, some data is hidden or not fully evaluated with the bar graph that is visualized with the scatter graph.

Knowing the best graph to use is an important way to convey the important information and supports the statistical analysis that is performed on the data.

It is critical to prepare for your statistical analysis at the beginning of the experimental design process. This will ensure the correct data is extracted, the proper test applied, and that sufficient replicates are obtained so that if an effect is to be found, it will be found. Don’t rely on some magic number of events or samples to determine your experimental design. Rather, rely on the best statistical methods and comparisons to appropriate controls to ensure your data stands up to review.

To learn more about 5 Steps For Accurate Flow Cytometry Statistical Analysis Results, and to get access to all of our advanced materials including 20 training videos, presentations, workbooks, and private group membership, get on the Flow Cytometry Mastery Class wait list.

Tim Bushnell, PhD


Advanced Microscopy

Learn the best practices and advanced techniques across the diverse fields of microscopy, including instrumentation, experimental setup, image analysis, figure preparation, and more.
flow cytometry tablet eBook cover

Modern Flow Cytometry

Learn the best practices of flow cytometry experimentation, data analysis, figure preparation, antibody panel design, instrumentation and more. 

Advanced 4-10 Color Compensation

Advanced 4-10 Color Compensation, Learn strategies for designing advanced antibody compensation panels and how to use your compensation matrix to analyze your experimental data.

Top 40 Networking Scripts For PhDs

If you want to get replies from top employers and recruiters, this ebook is for you. These networking scripts will show you the exact words ...

Informational Interviews For PhDs

If you want to learn how to set up and execute informational interviews with PhDs working in industry, this ebook is for you. Here, you ...

Industry Resume Guide For PhDs

If you have a PhD and want to create the perfect industry resume to attract employers, this ebook is for you. Here, you will get ...

Top 20 Industry Jobs For PhDs

If you want to learn about the top 20 industry careers for PhDs regardless of your PhD background, this ebook is for you. Here, you ...

Salary Negotiation For PhDs

If you have a PhD and want to learn advanced salary negotiation strategies, this ebook is for you. Here, you will learn how to set ...

Top 20 Transferable Skills For PhDs

If you want to learn the top 20 transferable skills the industry employers ranked as most important for PhDs to include on their resumes and ...

Related Posts You Might Like

We Tested 5 Major Flow Cytometry SPADE Programs for Speed – Here Are The Results

Written By: Tim Bushnell, PhD As a follow-up to our post on tSNE where we compared the speed of calculation in leading software packages, let’s ...
Read More

5 FlowJo Hacks To Boost The Quality Of Your Flow Cytometry Analysis

Written By: Tim Bushnell, PhD Primary data analysis, that is the analysis at the sample or tube level, is where the populations of interest are ...
Read More

Statistical Challenges Of Rare Event Measurements In Flow Cytometry

Written by Tim Bushnell, PhD To conclude our series on rare event analysis, it is time to discuss the statistics behind rare event analysis. The ...
Read More

How to Optimize Flow Cytometry Hardware For Rare Event Analysis

Written by Tim Bushnell, PhD “Not everything that can be counted counts and not everything that counts can be counted.” — William Bruce Cameron (but ...
Read More

How To Choose The Correct Antibody For Accurate Flow Cytometry Results

Written by Tim Bushnell, PhD Next to the flow cytometer itself, the most important component of a flow cytometry experiment comes down to the antibodies. ...
Read More

How To Achieve Accurate Flow Cytometry Calcium Flux Measurements

Written by Tim Bushnell, PhD Most flow cytometry experiments work with antibodies conjugated to a fluorochrome for some variation on immunophenotyping. However, any fluorochrome that ...
Read More