When is listwise deletion appropriate

As you can see based on Graphic 1, there are only slight differences between the densities of observed and missing values. A Kolmogorov-Smirnov Test confirms what we already saw graphically: The differences of our observed and unobserved values are not significant. Kolmogorov—Smirnov test between missing and non-missing values ks.

Be aware that we were able to inspect the missing mechanisms with these statistical tests, since we created the data ourselves. In reality we would not be able to perform such tests and therefore we would need to rely on theoretical assumptions about the randomness of our incomplete data. However, it is in general very rare in statistics that missing data is MCAR! In most cases you should try to use more sophisticated methods such as missing data imputation to take the missing data and the structure of the missingness into account.

Note: Not all imputation methods reduce bias. In fact, some methods e. Learn here , which method is appropriate for your specific database. For illustration, consider the following example data:.

As shown in Graphic 2 in the left pane, the densities of observed and missing values differ. Complete cases seem to be higher on average. A correlation plot of X and Y is illustrated in the right pane. As modeled before, when we created the synthetic data, there is a positive correlation between X and Y. However, we can also see that the mean of Y differs between the observed and the missing values. The mean of the missing data, illustrated by the red straight line, indicate a slightly lower value for observations with missingness in Y.

If we would perform a complete case analysis, we would therefore overestimate the mean of Y. Not really the result we would hope to see! Mean of observed and missing values round mean y[! Kolmogorov—Smirnov test of missing and non-missing values — systematic missingness ks. The p-value of the Kolmogorov-Smirnov Test is significant — our missing values are significantly different to our observed data.

Are you not familiar with the programming language R? There are many other software programs available that can find incomplete rows in your data and perform casewise deletion. Please accept YouTube cookies to play this video. By accepting you will be accessing content from YouTube, a service provided by an external third party.

YouTube privacy policy. Take Me to The Video! Comments Hi Karen Just a small note to add regarding the missingness assumption required for listwise deletion sometimes also called complete case analysis. Leave a Reply Cancel reply Your email address will not be published. The Analysis Factor uses cookies to ensure that we give you the best experience of our website.

If you continue we assume that you consent to receive cookies on all websites from The Analysis Factor. Close Privacy Overview This website uses cookies to improve your experience while you navigate through the website. Out of these, the cookies that are categorized as necessary are stored on your browser as they are essential for the working of basic functionalities of the website.

We also use third-party cookies that help us analyze and understand how you use this website. These cookies will be stored in your browser only with your consent. You also have the option to opt-out of these cookies. But opting out of some of these cookies may affect your browsing experience. Necessary Necessary. Necessary cookies are absolutely essential for the website to function properly. This category only includes cookies that ensures basic functionalities and security features of the website.

This approach can take a long time to converge, especially when there is a large fraction of missing data, and it is too complex to be acceptable by some exceptional statisticians. This approach can lead to the biased parameter estimates and can underestimate the standard error. For the expectation-maximization imputation method, a predicted value based on the variables that are available for each case is substituted for the missing data. Because a single imputation omits the possible differences among the multiple imputations, a single imputation will tend to underestimate the standard errors and thus overestimate the level of precision.

Thus, a single imputation gives the researcher more apparent power than the data in reality. Multiple imputation is another useful strategy for handling the missing data. In a multiple imputation, instead of substituting a single value for each missing data, the missing values are replaced with a set of plausible values which contain the natural variability and uncertainty of the right values. This approach begin with a prediction of the missing data using the existing data from other variables [ 15 ].

The missing values are then replaced with the predicted values, and a full data set called the imputed data set is created. This process iterates the repeatability and makes multiple imputed data sets hence the term "multiple imputation". Each multiple imputed data set produced is then analyzed using the standard statistical analysis procedures for complete data, and gives multiple analysis results. Subsequently, by combining these analysis results, a single overall analysis result is produced.

The benefit of the multiple imputation is that in addition to restoring the natural variability of the missing values, it incorporates the uncertainty due to the missing data, which results in a valid statistical inference. Restoring the natural variability of the missing data can be achieved by replacing the missing data with the imputed values which are predicted using the variables correlated with the missing data.

Incorporating uncertainty is made by producing different versions of the missing data and observing the variability between the imputed data sets. Multiple imputation has been shown to produce valid statistical inference that reflects the uncertainty associated with the estimation of the missing data.

Furthermore, multiple imputation turns out to be robust to the violation of the normality assumptions and produces appropriate results even in the presence of a small sample size or a high number of missing data. With the development of novel statistical software, although the statistical principles of multiple imputation may be difficult to understand, the approach may be utilized easily.

Sensitivity analysis is defined as the study which defines how the uncertainty in the output of a model can be allocated to the different sources of uncertainty in its inputs. When analyzing the missing data, additional assumptions on the reasons for the missing data are made, and these assumptions are often applicable to the primary analysis.

However, the assumptions cannot be definitively validated for the correctness. Therefore, the National Research Council has proposed that the sensitivity analysis be conducted to evaluate the robustness of the results to the deviations from the MAR assumption [ 13 ].

Missing data reduces the power of a trial. Some amount of missing data is expected, and the target sample size is increased to allow for it. However, such cannot eliminate the potential bias. More attention should be paid to the missing data in the design and performance of the studies and in the analysis of the resulting data. The best solution to the missing data is to maximize the data collection when the study protocol is designed and the data collected. Application of the sophisticated statistical analysis techniques should only be performed after the maximal efforts have been employed to reduce missing data in the design and prevention techniques.

A statistically valid analysis which has appropriate mechanisms and assumptions for the missing data should be conducted.

Single imputation and LOCF are not optimal approaches for the final analysis, as they can cause bias and lead to invalid conclusions.

All variables which present the potential mechanisms to explain the missing data must be included, even when these variables are not included in the analysis [ 16 ]. Researchers should seek to understand the reasons for the missing data. Distinguishing what should and should not be imputed is usually not possible using a single code for every type of the missing value [ 17 ].

It is difficult to know whether the multiple imputation or full maximum likelihood estimation is best, but both are superior to the traditional approaches. Both techniques are best used with large samples. In general, multiple imputation is a good approach when analyzing data sets with missing data.

National Center for Biotechnology Information , U. Journal List Korean J Anesthesiol v. Korean J Anesthesiol. Published online May Hyun Kang. Find articles by Hyun Kang. Author information Article notes Copyright and License information Disclaimer. Corresponding author. Corresponding author: Hyun Kang, M.

Tel: , Fax: , moc. Received Feb 13; Accepted Feb This article has been cited by other articles in PMC. Abstract Even in a well-designed and controlled study, missing data occurs in almost all research. Types of Missing Data Rubin first described and divided the types of missing data according to the assumptions based on the reasons for the missing data [ 4 ]. Missing completely at random Missing completely at random MCAR is defined as when the probability that the data are missing is not related to either the specific value which is supposed to be obtained or the set of observed responses.

Missing at random Missing at random MAR is a more realistic assumption for the studies performed in the anesthetic field. Techniques for Handling the Missing Data The best possible method of handling the missing data is to prevent the problem by well-planning the study and collecting the data carefully [ 5 , 6 ].

Listwise or case deletion By far the most common approach to the missing data is to simply omit those cases with the missing data and analyze the remaining data. Pairwise deletion Pairwise deletion eliminates information only when the particular data-point needed to test a particular assumption is missing. Mean substitution In a mean substitution, the mean value of a variable is used in place of the missing data value for that same variable.

Regression imputation Imputation is the process of replacing the missing data with estimated values. Last observation carried forward In the field of anesthesiology research, many studies are performed with the longitudinal or time-series approach, in which the subjects are repeatedly measured over a series of time-points. Accordingly, the National Academy of Sciences has recommended against the uncritical use of the simple imputation, including LOCF and the baseline observation carried forward, stating that: Single imputation methods like last observation carried forward and baseline observation carried forward should not be used as the primary approach to the treatment of missing data unless the assumptions that underlie them are scientifically justified [ 13 ].

Maximum likelihood There are a number of strategies using the maximum likelihood method to handle the missing data. Expectation-Maximization Expectation-Maximization EM is a type of the maximum likelihood method that can be used to create a new data set, in which all missing values are imputed with values estimated by the maximum likelihood methods [ 14 ].

Multiple imputation Multiple imputation is another useful strategy for handling the missing data. Sensitivity analysis Sensitivity analysis is defined as the study which defines how the uncertainty in the output of a model can be allocated to the different sources of uncertainty in its inputs. Recommendations Missing data reduces the power of a trial. References 1. Graham JW.