English
 
User Manual Privacy Policy Disclaimer Contact us
  Advanced SearchBrowse

Item

ITEM ACTIONSEXPORT
  Addressing Validity and Generalizability Concerns in Field Experiments

Riener, G., Schneider, S., & Wagner, V. (2020). Addressing Validity and Generalizability Concerns in Field Experiments.

Item is

Basic

show hide
Item Permalink: http://hdl.handle.net/21.11116/0000-0006-9D8A-E Version Permalink: http://hdl.handle.net/21.11116/0000-0006-9D8D-B
Genre: Paper

Files

show Files

Locators

show
hide
Description:
-

Creators

show
hide
 Creators:
Riener, Gerhard, Author
Schneider, Sebastian1, Author              
Wagner, Valentin, Author
Affiliations:
1Max Planck Institute for Research on Collective Goods, Max Planck Society, ou_2173688              

Content

show
hide
Free keywords: External validity, field experiments, generalizability, treatment effect, overlap, balance, precision, treatment assignment, unconfoundedness, self-selection bias, site-selection bias
 JEL: C9 - Design of Experiments
 JEL: C90 - General
 JEL: C93 - Field Experiments
 JEL: D04 - Microeconomic Policy: Formulation, Implementation, and Evaluation
 Abstract: In this paper, we systematically analyze the empirical importance of standard conditions for the validity and generalizability of field experiments: the internal and external overlap and unconfoundedness conditions. We experimentally varied the degree of overlap in disjoint sub-samples from a recruitment experiment with more than 3,000 public schools, mimicking small scale field experiments. This was achieved by using different techniques for treatment assignment. We applied standard methods, such as pure randomization, and the novel minMSE treatment assignment method. This new technique should achieve improved overlap by balancing covariate dependencies and variances instead of focusing on individual mean values. We assess the relevance of the overlap condition by linking the estimation precision in the disjoint sub-samples to measures of overlap and balance in general. Unconfoundedness is addressed by using a rich set of administrative data on institution and municipality characteristics to study potential self-selection. We find no evidence for the violation of unconfoundedness and establish that improved overlap, and balancedness, as achieved by the minMSE method, reduce the bias of the treatment effect estimation by more than 35% compared to pure randomization, illustrating the importance of, and suggesting a solution to, addressing overlap also in (field) experiments.

Details

show
hide
Language(s):
 Dates: 2020-06-29
 Publication Status: Published online
 Pages: -
 Publishing info: Bonn : Max Planck Institute for Research on Collective Goods, Discussion Paper 2020/16
 Table of Contents: -
 Rev. Type: -
 Identifiers: -
 Degree: -

Event

show

Legal Case

show

Project information

show

Source

show