Quality assessment in gastroenterological research: publication rates of meta-analyses

Meta-analysis is a formal quantitative study design used to systematically assess and pool the results of smaller research studies [1]. Meta-analyses allow researchers to derive more concrete conclusions in the absence of larger clinical studies. Smaller clinical studies may contain limited or conflicting results, which can complicate clinical decision-making based on their conclusions. As a result, meta-analysis plays a central role in evidence-based medicine by integrating multiple studies and overcoming external validity limitations [2,3].

A review of the meta-analyses presented at large gastroenterology meetings suggests that their number is increasing. We hypothesize that, while the number of meta-analyses is high, only a minority are published as full text. We aimed to compare the publication rates of meta-analysis abstracts presented at Digestive Disease Week (DDW) from 2014-2017.

We identified abstracts of meta-analyses presented at DDW from 2014-2017 published in supplements of the American Gastroenterological Association’s (AGA) journal Gastroenterology and the American Society for Gastrointestinal Endoscopy’s (ASGE) journal Gastrointestinal Endoscopy. Abstracts were further assessed by cross-referencing the PubMed database and Google Scholar to determine full-text manuscript publication status. Information on country of study, number of authors, male or female lead or senior authorship, university or community program, number of studies, and sample size was extracted. Descriptive analysis of abstract and full-length paper publication characteristics was performed using SPSS v25 IBM (USA).

During the study period, 523 meta-analyses were presented at DDW (330 AGA abstracts and 193 ASGE abstracts). Of those, 96 AGA and 44 ASGE abstracts were published as full-length manuscripts. The US contributed 374 (73.9%) meta-analyses, compared to 53 (10.5%) from Europe, 55 (10.9%) from Asia, and 24 (4.7%) from Canada. Ninety-five percent of abstracts were from university programs, compared to 5% from community programs (P<0.001). Male lead and senior authors accounted for 74% and 84% of authors, respectively. During the study period, only 26.6% of abstracts were eventually published as full-length manuscripts (523 abstracts vs. 139 manuscripts, P<0.001). Approximately 41% of abstracts in 2014, 6% of abstracts in 2015, 24% of abstracts in 2016 and 37% of abstracts in 2017 were published as full-length papers (Fig. 1). Abstracts and full-text manuscripts had a similar median number of studies included for analysis (11 vs. 12, P>0.05, respectively).

Figure 1 (A) Gastroenterology and Gastrointestinal Endoscopy (GIE) abstracts presented from 2014-2017. (B) Comparison of abstracts and full-length manuscript publications

Our report demonstrated that while the number of meta-analyses presented at DDW is growing, less than 30% of abstracts are translated into full-length manuscripts. Furthermore, the numbers of studies that met the inclusion criteria and were analyzed in either abstract or manuscript format were similar. This raises the question: “of what quality are these meta-analyses?”

Lui et al (2017) assessed the quality of 127 gastroenterology-based full-length manuscripts and reported that the methodological reporting quality of meta-analyses has improved with compliances to the PRISMA statement and AMSTAR checklist [4]. However, very little information is known about the quality of systematic reviews and meta-analyses submitted to conferences.

With the advent of new and user-friendly software, pooled analyses are relatively easier to perform. What is more, these analyses can be performed by operators with little to no statistical training. Anecdotally, some meta-analyses are simply a repeat of studies that have been previously performed with the addition of 1-2 recent original research studies, while other studies are underpowered, which brings into question the need for performing a meta-analysis.

While abstracts do play a role in distilling information for the reader, full-length manuscripts are essential in helping readers evaluate the integrity and quality of research. Hence, without an assessment of research methodology, conclusions drawn from meta-analyses abstracts are limited, at best. Additionally, authors and reviewers should evaluate the significance and limitations of systematic reviews and meta-analyses. Despite presenting the best available evidence, a meta-analysis based on low-quality studies might be misleading and must be interpreted with caution [5]. To this end, assessing the quality of studies is also limited within an abstract format; thus, a full-length manuscript is needed.

Conference abstracts are not a substitute for full-length manuscripts. A consensus statement from the PRISMA for Abstracts Group noted that abstracts should be robust enough in presenting a clear and truthful account of the intended research [6]. More attention should be paid by authors and society abstract reviewers to the quality of systematic reviews and meta-analyses, given their central role in providing robust evidence-based medicine in the field of gastroenterology.