Statistical Challenges Of Rare Event Measurements In Flow Cytometry

To conclude our series on rare event analysis, it is time to discuss the statistics behind rare event analysis. The first 2 parts of this series covered the hardware aspects of measuring rare events and some specific recommendations for gating/analysis of rare events.

It is necessary to sort through hundreds of thousands or millions of cells to find the few events of interest.

With such low event numbers, we move away from the comfortable domain of the Gaussian distribution and move into the realm of Poisson statistics.

There are 3 points to consider to build confidence in the data that the events being counted are truly events of interest and not random events that just happen to fall into the gates of interest.

1. How do you know if an event is real?

How do you know that your rare event is real? When subsetting the population, you might have an occurrence rate of 0.1% or lower. This means that for every 100,000 cells, 100 cells or fewer will be in the final gate of interest.

How can you confirm and be comfortable they are real?

In Poisson statistics, the number of positive events is the important factor, not the total number of events.

In Poisson statistics, the mean and variance of the distribution are equal to the number of positive events. The standard deviation is the square root of the variance.

So, if you have 2 events in a region the CV of that data is roughly 71%, whereas with 100 events, the CV drops to about 10%.

But, what does this really mean?

In this paper by Maecker and coworkers, the authors looked at inter-lab CV in flow cytometry experiments, and estimated it was high as 40%, some of which could be reduced by centralizing analysis. More importantly, the inter-lab CVs were highest (57-82%) on samples where the average percentage of cells was below 0.1%.

And, with as few as 12 events, that leads a standard deviation of 28%.

The frequency precision of our measurement is now dominated by assay errors, not the rare events that are analyzed.

With rare event analysis, you demonstrate significance through assay reproducibility. The number of samples that should be measured can be determined using the power calculation, which is discussed in more detail here.

2. How many total events do you need?

Statistically, assay variation can be a major source of error for this analysis. That leads to the question, “How many events do you need?”

The short answer is that whenever possible, collect as many events as possible.

As discussed previously, there may be limitations imposed by the hardware and software which limit how much data can be collected in a single file. This means you may have to collect multiple files from the same tube.

In third party software, it is possible to do some preliminary gating to reduce file size, and concatenate multiple files after this preliminary analysis to make the final gating more complete.

Turning back to how many events is enough, there is more than one n. To show the significance of the data, the analysis must be repeated multiple times (i.e. power the experiment appropriately) and have the correct complement of appropriate negative controls.

The data above shows both of these concepts. On the left, is the gating strategy and the control (normal patient control), while on the right are the results of several analysis runs on 2 patients to show the differences between the 2 populations.

The statistical analysis between these 2 show that there is a significant difference, as denoted by the asterisk.

Returning to the question of how many events is enough, the question to ask is, “What is the CV required for analysis — what spread of the data is acceptable?”

Is 10,000 events enough?

The chart below shows the coefficient of variation (CV) value for a given frequency of cells. The CV is related to the number of positive events, and is defined as the SD/mean.

In general, a lower CV is better. The CV is another way to express the precision and repeatability of an experiment.

Using this table, if a broad CV is acceptable, with a cell frequency of 0.1% 10,000 events is enough. However, a 10% CV requires 100 positive events and you can see now that 10,000 events is only good if at the 1% range.

For very rare events, a 10% CV for very rare populations requires collection of a million, or even 10 million, total events. Collecting at a rate of 10,000 events per second, it would take 1,000 seconds to collect 10 million events, or 16 minutes.

The CV is going to relate to the ability to identify a difference between 2 populations. This, in turn, will be related to the power of the experiment. Since we have the standard deviation of the population, it makes the calculations easier. However, the difference between the control and experimental will drive this.

In this paper by Mario Roederer, he discusses this issue of how many events you need to know if something is real. According to this paper, one of the important things to do is compare your positive sample to a set of controls so that you can interpret the data correctly.

There’s no arbitrary number of events that is the “right” number.

Even 12-14 positive events may be accurate, based upon your knowledge and the data generated by your controls.

3. How do you sort rare events?

The ability to sort cells for downstream applications is one of the most powerful applications of flow cytometry. Poisson statistics again play a role in determining an appropriate event rate.

If the drop drive frequency is 80 kHZ, or 80,000 droplets being generated per second, how many events per second should you run? Remember that a cell sorter sorts droplets, not cells, per se, but the cells are contained within the drops.

Depending on the sort envelope, the sort decision can include 1 or 2 droplets. So, what is a reasonable event rate?

When the event rate is equal to the drop drive frequency, Poisson statistics predict that a little under 40% of the drops will have no cells, about 40% will have 1, about almost 20% will have 2, and about 5% will have 3 or more cells.

If the event rate is ½ the drop drive frequency, 7.5% of the droplets will have 2 cells. When the event rate is ¼ the drop drive frequency, about 80% of the droplets are empty and about 2% of the drops will have 2 events. Going to ⅙ the drop drive frequency, the improvements are minimal.

So, if the drop drive frequency is 40 kilohertz or 40,000 droplets per second, the event rate should be no more than 10,000 events per second.

What does this mean practically?

This chart can help you determine how long it will take to sort, based on the frequency of the drop delay and the frequency of population, assuming 100,000 cells are needed for a downstream application.

Sort operators are often asked if there is a way to reduce the time it takes to sort, especially with a rare event population. Since there is a ceiling on event rates, our only option is to enrich the sample to increase the proportion of desired events.

This can be done using a depletion assay with magnetic beads from Miltenyi Biotec, the IMAG system, Dynabeads, and others.

In these systems, cells are tagged with an antibody label conjugated to a magnetic bead then exposed to a magnet. The cells that are labeled are held by the magnet, the cells that are not labeled stay in suspension or pass through the column for collection and downstream sorting.

Let’s look at when it might be helpful to incorporate this pre-sort enrichment step.

Starting with 100 million cells and a desired population at 0.01%, if you took those 100 million cells and sorted them at 20,000 events per second, it would take about 83 minutes to do the whole sort.

If we take those same 100 million cells and perform a magnetic bead enrichment, which will take about 45 minutes using one of the various magnetic isolation kits, the untouched cells will be about 10 million cells and the population of rare events is enriched to 0.1%.

Sorting those 10 million cells at 20,000 events per second will only take about 10 minutes, compared to 83 minutes pre-magnetic bead enrichment. The faster sort means that your cells are going to be healthier because you can get them back into culture, or into whatever buffer they need, more quickly.

After all the tweaking of the hardware and optimizing the data analysis, the statistics must be considered. Poisson statistics dominate rare event analysis. From determining how many cells to collect, to how fast to sort cells, the number of positive events is critical for determining the statistics involved. The charts and data in this blog can help design your next rare event analysis experiment, and help provide the basis for improving reproducibility and consistency of the experiments.

To learn more about Statistical Challenges Of Rare Event Measurements In Flow Cytometry, and to get access to all of our advanced materials including 20 training videos, presentations, workbooks, and private group membership, get on the Flow Cytometry Mastery Class wait list.

Join Expert Cytometry's Mastery Class
Tim Bushnell, PhD
Tim Bushnell, PhD

Tim Bushnell holds a PhD in Biology from the Rensselaer Polytechnic Institute. He is a co-founder of—and didactic mind behind—ExCyte, the world’s leading flow cytometry training company, which organization boasts a veritable library of in-the-lab resources on sequencing, microscopy, and related topics in the life sciences.

Similar Articles

We Tested 5 Major Flow Cytometry SPADE Programs for Speed - Here Are The Results

We Tested 5 Major Flow Cytometry SPADE Programs for Speed - Here Are The Results

By: Tim Bushnell, PhD

In the flow cytometry community, SPADE (Spanning-tree Progression Analysis of Density-normalized Events) is a favored algorithm for dealing with highly multidimensional or otherwise complex datasets. Like tSNE, SPADE extracts information across events in your data unsupervised and presents the result in a unique visual format. Given the growing popularity of this kind of algorithm for dealing with complex datasets, we decided to test the SPADE algorithm in 5 software packages, including Cytobank, FCS Express, FlowJo, R, and the original, free software made available by the author of SPADE. Which was the fastest?

5 FlowJo Hacks To Boost The Quality Of Your Flow Cytometry Analysis

5 FlowJo Hacks To Boost The Quality Of Your Flow Cytometry Analysis

By: Tim Bushnell, PhD

FlowJo is a powerful tool for performing and analyzing flow cytometry experiments, if you know how to use it to the fullest. This includes understanding embedding and using keywords, the FlowJo compensation wizard, spillover spreading matrix, FlowJo and R, and creating tables in FlowJo. Extending your use of FJ using these hacks will help organize your data, improve analysis and make your exported data easier to understand and explain to others. Take a few moments and explore all you can do with FJ beyond just gating populations.

How to Optimize Flow Cytometry Hardware For Rare Event Analysis

How to Optimize Flow Cytometry Hardware For Rare Event Analysis

By: Tim Bushnell, PhD

Preparing for rare event analysis requires an understanding of the power and limitation of the instrument to be used. From how fast to run the fluidics, to how the signal is processed to the number of gates that can be used in the sorting experiment, each factor impacts the outcome of the experiment.

How To Choose The Correct Antibody For Accurate Flow Cytometry Results

How To Choose The Correct Antibody For Accurate Flow Cytometry Results

By: Tim Bushnell, PhD

With the added emphasis on reproducibility, it is critical to look at every step where experiments can be improved. No single step makes an experiment more reproducible, rather it is a process, making changes at each stage that leads to reproducibility. Antibodies comprise a critical component that needs to be reviewed. As Bradbury et al. in a commentary in Nature pointed out, the global spending on antibodies is about $1.6 billion a year, and it is estimated about half of that money is spent on “bad” antibodies. This does not include the additional costs of wasted time and effort by…

How To Achieve Accurate Flow Cytometry Calcium Flux Measurements

How To Achieve Accurate Flow Cytometry Calcium Flux Measurements

By: Tim Bushnell, PhD

Dyes exist for the detection of everything from large nucleic acids to reactive oxygen species, and from lipid aggregates to small ions. Concentrations of physiologically important ions such as sodium, potassium, and calcium can be important indicators of health and disease. Calcium ions play an especially critical role in cellular signaling. As a signaling messenger, calcium is involved in everything from muscle contractions, to cell motility, to enzyme activity. Calcium experiments can be very informative, and with the advent of cheaper UV lasers, more and more researchers can use ratiometric measurements to evaluate the signaling processes in phenotypically defined populations.

How to Perform Doublet Discrimination In Flow Cytometry

How to Perform Doublet Discrimination In Flow Cytometry

By: Tim Bushnell, PhD

You are probably familiar with the term, “doublet discrimination” or “doublet exclusion”, and have likely included this flow cytometry measurement into at least some (if not all) of your gating strategies. Even though you may utilize this important gating strategy, you may not have had the chance to delve deeper to explore exactly what doublets are and why it’s critical to exclude them. This article aims to give you insight on the what, why, and how of doublet discrimination.

4 Considerations For Assessing Protein Phosphorylation Using Flow Cytometry

4 Considerations For Assessing Protein Phosphorylation Using Flow Cytometry

By: Tim Bushnell, PhD

For those working in the signaling field, having the ability to take a sample and phenotypically identify it, while knowing what is happening inside the cell to the target molecules of choice opens up a host of new opportunities. These assays are amenable to high throughput setup, meaning that biologically relevant outcomes in pre-clinical drug discovery can be measured directly. All told, with a little forethought, some careful planning and validation, and our helpful tips, phosphoflow assays are within your reach.

5 Essential Calculations For Accurate Flow Cytometry Results

5 Essential Calculations For Accurate Flow Cytometry Results

By: Tim Bushnell, PhD

Flow cytometry is a numbers game. There are percentages of a population, fluorescence intensity measurements, sample averages, data normalization, and more. Many of these common calculations are useful, but surrounded by misconceptions. This primer will help you decide which calculation to use, when to use it, and how to interpret the results.

Measuring Receptor Occupancy With Flow Cytometry

Measuring Receptor Occupancy With Flow Cytometry

By: Tim Bushnell, PhD

Measuring the receptor occupancy of a given target showcases the power of flow cytometry. With the right reagents, best practices, and attention to detail, this assay can become a mainstay in your research toolkit. It extends quantitative flow cytometry to the next level, to determine a complete biological picture of how efficiently a given target is being bound. This also serves as the basis for even more fine-analysis when combined with assessment of downstream targets that the engagement of the receptor by the target antibody may affect. Phosphorylation, cell cycle arrest, and protein expression are all within reach, resulting in…

Top Technical Training eBooks

Get the Advanced Microscopy eBook

Get the Advanced Microscopy eBook

Heather Brown-Harding, PhD

Learn the best practices and advanced techniques across the diverse fields of microscopy, including instrumentation, experimental setup, image analysis, figure preparation, and more.

Get The Free Modern Flow Cytometry eBook

Get The Free Modern Flow Cytometry eBook

Tim Bushnell, PhD

Learn the best practices of flow cytometry experimentation, data analysis, figure preparation, antibody panel design, instrumentation and more.

Get The Free 4-10 Compensation eBook

Get The Free 4-10 Compensation eBook

Tim Bushnell, PhD

Advanced 4-10 Color Compensation, Learn strategies for designing advanced antibody compensation panels and how to use your compensation matrix to analyze your experimental data.