Selection Bias and Mortality data

On the website of the journal Epidemiology and Prevention (https://repo.epiprev.it/) Corrado Magnani and Dario Gregori have inserted a brilliant note on Istat data released on April 1st. It is worth reading it, with a brief introduction. A selection bias (systematic bias) occurs when a subset of the population is over or under-represented in a study.

For example, if I want to study the effectiveness of a therapy, the results of the treatment could be more pronounced in the group of younger people and generally in better health, creating a false response if they are overrepresented compared to the typical patient. Having said that, we report the note on mortality data, observing that these data are however very useful for studies at the municipal level for the municipalities included. Here we propose a rough translation of their contribution:

<< On 1 April ISTAT made available mortality data at the municipal level for the period 1 - 21 March for 1,084 municipalities, without however providing details on the selection criterion. We immediately used this data, confident that there were no other selection criteria other than the operational ones required for the timely availability of information.

The information at the municipal level has been widely used, not only by us, to answer some questions that we consider extremely important, in particular as regards the variation in excess mortality compared to what is officially attributed to the epidemic, also with projections to regional and national level. In the 1084 municipalities of the sample from 1 to 21 March 2020, 16,126 deaths were observed while the expected based on the 2015-2019 average was 7,843.4 deaths. This highlights the importance of being able to use these values ​​to estimate excess mortality at the regional and national levels.

None of this is feasible properly since the sample has been distorted, with a clear selection bias.

The technical document accompanying the data made available on April 9, 2020, relating to the period up to March 28 (www.istat.it/it/files//2020/03/Il_punto_sui_decessi_9_aprile_2020.pdf), which had not been previously disclosed, reports on page 1 in a footnote the following selection criteria: “Municipalities with a number of deaths which, in the period 1 January - 28 March 2020, was greater than or equal to 10 units and which in March 2020 presented, compared to the corresponding average of the 2015-2019 five-year period, an increase in mortality of at least 20% ".

Any epidemiology student knows that selecting only the municipalities that "... have presented ... an increase in mortality of at least 20%" is a serious mistake, which introduces a selection bias. The consequence is to overestimate excess mortality, in an unquantifiable way. Correct data analysis and correct conclusions, therefore, become impossible, particularly if you want to evaluate the overall impact of this period on mortality.

It would be advisable for ISTAT to release a new and complete version of the data without inappropriate selection criteria. >>

Authors of the study:

Corrado Magnani - Unità di Statistica Medica ed Epidemiologia dei Tumori , Dipartimento di Medicina Traslazionale, Università del Piemonte Orientale, Novara

Dario Gregori -  Unità di Biostatistica, Epidemiologia e Sanità Pubblica, Dipartimento di Scienze Cardio-Toraco-Vascolari e Sanità Pubblica, Università di Padova,