A regression discontinuity method is close to an experiment under ideal conditions, in reducing selection bias (high internal validity), and in presenting challenges to broader generalization (low external validity). In the simplest regression discontinuity model, we observe mean outcomes conditional on an assignment variable Z that individual units have no direct control over, and we see a treatment turn on (go from 0 to 1, off to on) at one "cutoff" value of Z. In this case, the difference in conditional means just above and below the cutoff measures the change in mean outcomes arising from treatment.

If the units above and below the cutoff are comparable in every respect except treatment, this comparison is comparable to an experimental sample, in that random fluctuations in the assignment variable Z determine which units in that vicinity get treated, and the internal validity of the method is comparable to an experiment. Unfortunately, the estimate of impact applies only to cases at the cutoff, so external validity is very low.

When the treatment does not turn on from 0 to 1 at one cutoff level, a regression discontinuity method is still possible if the level of treatment or fraction treated jumps discontinuously at a cutoff. In that case, we divide the estimated jump in mean outcomes by the estimated jump in treatment, which is a form of instrumental variables known as a Wald estimator. For both types of regression discontinuity methods, a common approach is to weight the data so that only observations close to the cutoff are used, and observations closer to the cutoff get more weight (using a kernel centered at the cutoff to construct weights).

Thus, we use a local linear regression, or local Wald estimator, around the cutoff to estimate impact at the cutoff. The fact that observations are not all right at the cutoff introduces some bias, which disappears asymptotically as we add more data, and our window of observations shrinks toward the window of width zero around the cutoff.

Ideally, only treatment changes discontinuously at the cutoff. In some cases, we might worry that there are other changes, for example when a benefit changes abruptly at age 18. In that case, we can look before a policy took effect to see if there was any impact when there was no treatment. For example, in the New York Earned Income Tax Credit for Noncustodial Parents, no impact was found before the advent of the policy, and there was a positive impact on collections and earnings after the policy took effect, lending credence to the postpolicy estimate. The external validity of this estimator is low because the impacts estimated apply only to noncustodial parents whose youngest children with child support orders are exactly 18. If parents with older or younger children react differently, we cannot generalize to the whole population of noncustodial parents.