From Submission to Publication: A Retrospective Review of the Tables and Figures in a Cohort of Randomized Controlled Trials Submitted to the British Medical Journal

Background 

Tables and figures are a powerful means of conveying investigational results.1, 2, 3, 4, 5, 6 When properly constructed, they succinctly convey complex relationships prose cannot describe. Tables and figures bring readers closer to the actual data than prose, and their use can reduce the potential for biases that can occur when the dimensionality of data is reduced.7 Dimensionality is reduced when the investigator presents summaries of the data rather than the data points themselves. For example, an investigator who collected the values of 2 continuous variables on 100 patients could present them as a scatterplot (200 numbers), 2 box plots (10 numbers+any outlying values), or a bar graph of each variable’s mean (2 numbers [+4 more if confidence intervals (CIs) are presented]). Only the scatterplot preserves the dimensionality of the data and conveys the relationship between the 2 variables within each patient that is inherent to this data set.⁎

With each decrease in dimensionality, there is the potential that truth and nuance will be masked.

Importance 

Given the potential importance of tables and figures, one might expect that the medical journal peer-review process would pay particular attention to them. Preliminary results suggest otherwise.8 Day et al found that only 6% of general reviewer comments concerned tables or figures (D. Schriger, written communication, August 2002). Cooper et al9, 10 found that many figures in large- and medium-circulation journals were suboptimal. Although there has been little empirical research on the qualities that make for a good table or figure, there has been a fair amount of theoretical work. This work suggests that tables and figures should be self-explanatory, should show overall patterns and detailed information, and should have a data density—the amount of information presented per cm2—that justifies their existence (ie, are denser than text).1, 2, 3, 4, 5, 6, 7

Goals of This Investigation 

We executed this study to assess formally the quality of tables and figures in reports of randomized controlled trials (RCTs) submitted to a general medical journal and to examine how often and in what ways they were modified between submission and publication. We chose to study the figures and tables in reports on a single study design, the RCT, for 2 reasons. First, the kinds of tables and figures required to present data from studies of similar design should be few and would be expected to be somewhat consistent among articles. Second, much of the content of reports of RCTs has been standardized in recent years because of CONSORT, and we wanted to see whether the CONSORT statement, though it makes no recommendations about data tables and data figures, might nevertheless have influenced the content of tables and figures.11, 12 In short, we wanted to understand how tables and figures were being used in reports of RCTs, whether tables and figures met their potential as a powerful means of conveying study results, and whether the peer-review process improved the quality of published tables and figures.

Figures 

There were 41 figures in the 62 submitted articles, 36 of which were available for evaluation (5 were referred to in the article, but no figures were found either because they were lost in the BMJ filing process or were never submitted). Between submission and publication, 8 of the 41 figures were dropped and 8 figures were added, resulting in 41 published figures. Interrater agreement was 96% on the 20 items; no item had less than 92% agreement.

Thirty-five published articles had no figures, 18 had 1, 4 had 2, and 5 had 3 (Figure 1). Eighty-eight percent (36/41) of published figures were univariate, and 79% (28/36) of these graphed a statistic about the data (eg, mean, median, proportion) rather than the data itself (Table 1). The number of detailed figures (those showing univariate or bivariate distributions) increased from submission to publication but only by 5% (7/36 to 10/41). Two scatterplots and 5 survival curves (17% of all figures) showed data for individual participants.

View Large Image Download to PowerPoint

Figure 1. Histogram of the number of tables and figures per article. Vertical thin lines are the means for each. Note that the y axis and gridlines are strategically placed to help readers glean the exact values of the bars, a nontraditional enhancement to the standard graphic layout.

Ten percent (2/19) of published figures of paired data depicted the pairing. Thirty-two percent (10/31) of figures that concerned data that had an underlying distribution (ie, data that were not binary or proportions) portrayed that distribution, and 22% of figures were deemed wholly redundant with text or tables. We judged that 49% of published figures met their potential.

The internal characteristics of the figures improved slightly from submission to publication, though a substantial number of published figures remained suboptimal (Table 2). The mode of comparison and the efficiency of data presentation were essentially unchanged. Of the 29 figures whose submitted and published versions were similar, 15 were unchanged and 14 had minor modifications: type of figure (4), labels (7), statistics (2), legends (2), gridlines (5), and shading (2). The mean data density index—the number of data points per square centimeter—was essentially identical (0.50 data elements/cm2) in the submitted and published versions of these 29 figures (Figure 2).

View Large Image Download to PowerPoint

Figure 2. Histogram of the data density index (DDI) for tables and figures. Vertical thin lines are the mean DDI for each.

Peer reviewers at BMJ made comments about the figures of 5 articles. One reviewer commented on the title and axis labels for one figure. This figure was omitted in the published article. The other 4 figures were published unchanged despite the reviewers’ suggestions. The reviewers had requested that authors rename the axis in one figure, add error bars in one figure, and completely redraw the other 2. For one of these, the authors provided the journal with the requested revision but successfully (and appropriately) argued that the original figure was better.

Tables 

There were 207 tables (205 available for evaluation) in 58 submissions. Thirty-four were dropped and 24 added between submission and publication, resulting in 197 published tables in 59 articles. Three publications had no tables; the majority had 3 or 4 (Figure 1). Eighty-four percent (166/197) of published tables were comparative; the others were informational (Table 3). Our 2 raters achieved at least 86% agreement on 17 categorical items (mean agreement 93%). On continuous items, raters agreed on 89% and were within 5% on 99% of items (see Table 3, Table 4 and the text below for items).

Table 3. Purpose of the 197 tables in published versions of 62 RCTs submitted to BMJ in 2001.

	Purpose of Table	Published Tables, %, n=197
	Information	16
	Model results	50
	About participation	22
	Results	16
	Model inputs	3
	Adverse effects	3
	Other	6
	Comparison	84
	Main results	36
	Baseline characteristics	35
	Secondary results	20
	Adverse effects	4
	Subgroups	2
	Participants	1

Table 4. Characteristics of 100 randomly sampled published tables.

	Characteristic	Published Tables, n=100
	Number of strata
	None (ie, a list)	10
	One-way (eg, mean height in men vs women)	78
	Two-way (eg, mean height by sex and ethnicity)	12
	Table characteristics
	Table is not self-explanatory	4
	Table has undefined symbols or abbreviations	23
	Table has internal contradictions	5
	Table has inappropriately precise entries (eg, 4.6654%)	21
	Table content (out of 8,888 elements), %
	Data elements	66
	Inferential statistical elements	8
	Label elements (box, stub)	26
	Undefined elements (155/6,609 numeric elements)	2
	Elements with undefined units (eg, mm Hg) (32/872)	4
	Elements with unstated denominator (307/6,328)	5
	Data density (data elements/cm2), mean	1.2

In our quality analysis of 100 randomly selected tables, only 12 were 2-way; time (before-after) was the second variable in 8 of these. Only 1 table stratified on a variable of interest to examine confounding. Three tables failed to show the core data, presenting only marginal totals. Two tables were deemed to contradict the text, 3 were redundant with text or figures, 4 could have been folded into other tables, 4 would have been better presented as figures, and 2 should have been presented as text.

There were 6,609 numeric and 2,279 explanatory elements in these tables, an average of 89 elements per table (median 76; interquartile range [IQR] 49, 115; range 11 to 352). One hundred fifty-five (2.3%) numeric elements in 9 tables were undefined, meaning that the table provided no indication of what the number represented. Twenty-one tables presented numbers that were too precise (did not heed the concept of significant figures). Forty tables had data elements that had units (eg, mm Hg or cm/second) associated with them. The units were not specified for 3.7% of the 872 numbers that needed them. Ninety-six percent of data elements had an associated N, the number of subjects the element was based on. This N was unavailable for 4.9% of these elements. Forty tables contained elements that presented test statistics or P values. In these tables, these inferential statistics comprised 30% of all numeric elements (median 18%; IQR 12%, 32%). Overall, 11% of all numeric cells were inferential statistics. The mean data density of these 100 tables was 1.2 elements/cm2 (median 1.1; IQR 0.8, 2.9) (Figure 2).

We judged 92 of the 100 table titles to be adequate, though 51% failed to define the predominant cell content. Five stubs (row headers) were not clear, and 8 were not logically organized. For boxes (column headers), these numbers were 3 and 5, respectively. Nineteen tables used shading for decorative purpose and 10 for navigational purposes (helping readers track across rows or down columns); none used shading to direct readers to the most important cells in the table.15

None of the 171 tables that appeared in both submitted and published articles had a major revision (eg, strata added); however, all but 8 had minor changes (title 56%, box 48%, stub 53%, cells 53%, footnotes 51%, shading 22%, gridlines 47%). We judged 59% of the 664 changes as improvements, 37% as neutral, and 4% as degradations.

One hundred sixteen tables were in articles that were formally peer reviewed at the BMJ; peer reviewers made 47 comments about 37 (32%) of them. Authors heeded 18 (57%) of the 26 comments about articles published in BMJ in contrast to 9 (31%) of the 21 comments about articles subsequently published elsewhere (difference 26%; 95% CI –1% to 54%).

Nine of the 34 dropped tables were replaced with text or figures, 14 had content folded into an existing or new table, and 11 were simply omitted. Thirty of these changes were deemed improvements. Seven of the 24 new tables were made by combining existing tables, 1 was converted from a figure, and 7 from text. The remaining 9 were completely new. Twenty changes were considered improvements.