BrainMap taxonomy of experimental design: description and evaluation.

Abstract

Coordinate-based, voxel-wise meta-analysis is an exciting recent addition to the human functional brain mapping literature. In view of the critical importance of selection criteria for any valid meta-analysis, a taxonomy of experimental design should be an important tool for aiding in the design of rigorous meta-analyses. The coding scheme of experimental designs developed for and implemented within the BrainMap database provides a candidate taxonomy. In this study, the BrainMap experimental-design taxonomy is described and evaluated by comparing taxonomy fields to data-filtering choices made by subject-matter experts carrying out meta-analyses of the functional imaging literature. Fifteen publications reporting a total of 46 voxel-wise meta-analyses were included in this assessment. Collectively these 46 meta-analyses pooled data from 351 publications, selected for experimental similarity within each meta-analysis. Filter implementations within BrainMap were graded by ease-of-use (A-C) and by stage-of-use (1-3). Quality filters and content filters were tabulated separately. Quality filters required for data entry into BrainMap were classed as mandatory (five filters), being above the use grading system. All authors spontaneously adopted the five mandatory filters in constructing their meta-analysis, indicating excellent agreement on data quality among authors and between authors and the BrainMap development team. Two non-mandatory quality filters (group size and imaging modality) were applied by all authors; both were Stage 1, Grade A filters. Field-of-view filters were the least-accessible quality filters (Stage 3, Grade C); two field-of-view filters were applied by six and four authors, respectively. Authors made a total of 115 content-filter choices. Of these, 78 (68%) were Stage 1, Grade A filters; 16 (14%) were Stage 2, Grade A; and 21 (18%) were Stage 2, Grade C. No author-applied filter was absent from the taxonomy.