Research ArticleSOCIAL SCIENCES

Voter information campaigns and political accountability: Cumulative findings from a preregistered meta-analysis of coordinated trials

See allHide authors and affiliations

Science Advances  03 Jul 2019:
Vol. 5, no. 7, eaaw2612
DOI: 10.1126/sciadv.aaw2612
  • Fig. 1 Prior beliefs and politician performance.

    The figure plots performance information (Q) against prior beliefs (P) in each of the studies (left) and across all studies (right). Voters are in the good news group (gray) if information exceeds priors (Q > P) or if it confirms positive priors (P = Q, and Q is greater than median); otherwise, they are in the bad news group (black). On the right side, P and Q are standardized with a mean of 0 and an SD of 1 in each study. The density of the dotted areas is proportionate to the number of voters at each value of P and Q; for the pooled analysis, the rugs along the horizontal and vertical axes indicate the distribution of values. The Mexico study lacked a preintervention survey; thus, we determine the good news and bad news groups according to whether Q is greater than the median. The red lines indicate the linear fit between priors and information. For the pooled analysis, the slope of the fit is 0.071; the correlation is 0.053.

  • Fig. 2 Meta-analysis: Country-specific effects on vote choice.

    Estimated change in the proportion of voters who support an incumbent after receiving good news (left) or bad news (right) about the politician, compared to receiving no information. Unadjusted estimates. For estimating the average of the study-specific effects (top row), each study is weighted by the inverse of its size. Horizontal lines show 95% CIs for the estimated change. Entries under each estimate show p-values calculated by randomization inference. In all cases, the differences are close to zero and statistically insignificant.

  • Fig. 3 Meta-analysis: Country-specific effects on turnout.

    See notes to Fig. 2. In all but one test, the differences are close to zero and are statistically insignificant, using p-values from randomization inference.

  • Fig. 4 Robustness of findings across specifications: Vote for incumbent.

    Estimates across all specifications of the overall treatment effect of the common informational intervention on vote for incumbent. The vertical axis lists all considered specification choices. The top row shows the collection of estimates across all specifications. Each subsequent row holds fixed a given specification choice and shows the distribution of treatment effect estimates, varying all other choices. Darkened vertical lines show estimates for which p < 0.05. The dashed vertical line indicates the estimated average treatment effect reported in table S5.

  • Fig. 5 Robustness of findings across specifications: Turnout.

    See notes to Fig. 4.

  • Table 1 Pillars of the Metaketa Initiative.

    Challenges for
    cumulative learning
    The Metaketa approach
    1. Confounding in observational
    research
    1. Randomized controlled trials
    (RCTs)
    2. Limited external validity
    of single RCTs
    2. Multiple studies in diverse
    contexts
    3. Heterogeneous, scattered findings3. Meta-analysis with overall finding
    4. Diversity of interventions4. Coordination on common arm
    intervention
    5. Noncomparable measurement
    that impedes aggregation
    5. Harmonized measurement of
    inputs, outcomes, and controls
    6. Researcher incentives for
    innovation over replication
    6. Study-specific interventions
    preserve innovation and allow
    analysis of comparative
    effectiveness
    7. Private data7. Open data and replication code
    8. Errors in data or code8. Third-party data analysis
    9. Fishing (data mining,
    specification searching, and
    failure to account for multiple
    hypothesis tests)
    9. Preanalysis plans with limited
    number of specified
    hypotheses
    10. Publication bias10. Publication of all registered
    analyses

Supplementary Materials

  • Supplementary material for this article is available at http://advances.sciencemag.org/cgi/content/full/5/7/eaaw2612/DC1

    Section S1. Study design materials and methods

    Section S2. Primary analysis: Robustness and reliability of results

    Section S3. Secondary analysis: A Bayesian approach

    Section S4. Possible explanations for the null findings

    Section S5. Effects of publicly disseminated information

    Table S1. Individual study designs.

    Table S2. Descriptive statistics for sample of good news.

    Table S3. Descriptive statistics for sample of bad news.

    Table S4. Balance of covariates.

    Table S5. Effect of information, conditional on distance between information and priors, on vote choice, and turnout.

    Table S6. Deviations from MPAP and study PAPs in the meta-analysis.

    Table S7. Differential attrition.

    Table S8. Manipulation check: Effect of treatment on correct recollection, pooling good and bad news (unregistered analysis).

    Table S9. Manipulation check: Absolute difference between posterior and prior beliefs for pooled good and bad news (unregistered analysis).

    Table S10. Effect of information on perception of importance of politician effort and honesty.

    Table S11. Effect of information and source credibility on evaluation of politician effort and honesty (unregistered analysis).

    Table S12. Relationship between evaluation of politician effort and honesty with vote choice (unregistered analysis).

    Table S13. Effect of bad news on politician backlash.

    Table S14. Additional hypotheses and results.

    Table S15. Effect of moderators on incumbent vote choice.

    Table S16. Effect of information and context heterogeneity on incumbent vote choice.

    Table S17. Effect of information and electoral competition on vote choice.

    Table S18. Effect of information and intervention-specific heterogeneity on vote choice.

    Table S19. Interaction analysis: Effect of good news on incumbent vote choice.

    Table S20. Interaction analysis: Effect of bad news on incumbent vote choice.

    Table S21. Private versus public information: Effect of good news on incumbent vote choice.

    Table S22. Private versus public information: Effect of bad news on incumbent vote choice.

    Fig. S1. Benin—Graphical representation of provided information.

    Fig. S2. Brazil—Flyers distributed to voters.

    Fig. S3. Burkina Faso—Flashcard illustrations of municipal performance indicators.

    Fig. S4. Mexico—Example of benchmarked leaflet in Ecatepec de Morelos, México.

    Fig. S5. Uganda 1—Candidate answering questions during a recording session and candidate as seen in video.

    Fig. S6. Power analysis of minimal detectable effects, computed using Monte Carlo simulation.

    Fig. S7. Bayesian meta-analysis: Vote choice.

    Fig. S8. Bayesian meta-analysis: Turnout.

  • Supplementary Materials

    This PDF file includes:

    • Section S1. Study design materials and methods
    • Section S2. Primary analysis: Robustness and reliability of results
    • Section S3. Secondary analysis: A Bayesian approach
    • Section S4. Possible explanations for the null findings
    • Section S5. Effects of publicly disseminated information
    • Table S1. Individual study designs.
    • Table S2. Descriptive statistics for sample of good news.
    • Table S3. Descriptive statistics for sample of bad news.
    • Table S4. Balance of covariates.
    • Table S5. Effect of information, conditional on distance between information and priors, on vote choice, and turnout.
    • Table S6. Deviations from MPAP and study PAPs in the meta-analysis.
    • Table S7. Differential attrition.
    • Table S8. Manipulation check: Effect of treatment on correct recollection, pooling good and bad news (unregistered analysis).
    • Table S9. Manipulation check: Absolute difference between posterior and prior beliefs for pooled good and bad news (unregistered analysis).
    • Table S10. Effect of information on perception of importance of politician effort and honesty.
    • Table S11. Effect of information and source credibility on evaluation of politician effort and honesty (unregistered analysis).
    • Table S12. Relationship between evaluation of politician effort and honesty with vote choice (unregistered analysis).
    • Table S13. Effect of bad news on politician backlash.
    • Table S14. Additional hypotheses and results.
    • Table S15. Effect of moderators on incumbent vote choice.
    • Table S16. Effect of information and context heterogeneity on incumbent vote choice.
    • Table S17. Effect of information and electoral competition on vote choice.
    • Table S18. Effect of information and intervention-specific heterogeneity on vote choice.
    • Table S19. Interaction analysis: Effect of good news on incumbent vote choice.
    • Table S20. Interaction analysis: Effect of bad news on incumbent vote choice.
    • Table S21. Private versus public information: Effect of good news on incumbent vote choice.
    • Table S22. Private versus public information: Effect of bad news on incumbent vote choice.
    • Fig. S1. Benin—Graphical representation of provided information.
    • Fig. S2. Brazil—Flyers distributed to voters.
    • Fig. S3. Burkina Faso—Flashcard illustrations of municipal performance indicators.
    • Fig. S4. Mexico—Example of benchmarked leaflet in Ecatepec de Morelos, México.
    • Fig. S5. Uganda 1—Candidate answering questions during a recording session and candidate as seen in video.
    • Fig. S6. Power analysis of minimal detectable effects, computed using Monte Carlo simulation.
    • Fig. S7. Bayesian meta-analysis: Vote choice.
    • Fig. S8. Bayesian meta-analysis: Turnout.

    Download PDF

    Files in this Data Supplement:

Navigate This Article