"Good econometrics cannot save a shaky research agenda, but the promiscuous use of fancy econometric techniques sometimes brings down a good one."

Every empirical research project should begin with four Frequently Asked Questions (FAQs).

The most interesting research in social science is about cause and effect.

Why Causal Relationships?

• Useful for making predictions about counterfactual worlds
• Helps predict consequences of policy changes
• Can be derived from economic models

Example: Causal Effect of Education

Units of Analysis

Unit	Research Example
Individuals	Education → Wages (Labor Economics)
Firms	R&D Investment → Productivity
Countries	Colonial Institutions → Economic Growth (Acemoglu et al., 2001)

Contemplate the ideal experiment that could capture the causal effect of interest.

Why Think About Ideal Experiments?

• Helps pick fruitful research topics
• Formulates causal questions precisely
• Highlights forces to manipulate and factors to hold constant

"If you can't devise an experiment that answers your question in a world where anything goes, then the odds of generating useful results with a modest budget and non-experimental survey data seem pretty slim."

Examples

🚫 FUQ'd: Fundamentally Unidentified Questions

Questions that cannot be answered by any experiment are FUQ'd.

What's NOT FUQ'd: Causal Effects of Race/Gender

Race and gender seem hard to manipulate, but labor market discrimination research focuses on perceived race/gender.

• Shakespeare's Rosalind disguised as Ganymede
• Philip Roth's novel - Black professor passing as white
• Audit studies: Experiments with fake resumes (Bertrand & Mullainathan, 2004)

Identification Strategy: The manner in which a researcher uses observational data to approximate a real experiment.

Natural Experiments

Haavelmo (1944)'s Insight

"A design of experiments is an essential appendix to any quantitative theory. Experiments may be grouped into two classes:
(1) Experiments we should like to make to verify certain hypotheses
(2) The stream of experiments that Nature is steadily turning out, which we merely watch as passive observers"

Mode of Statistical Inference (Rubin, 1991)

What to Specify

• The population to be studied
• The sample to be used
• Assumptions made when constructing standard errors

Practical Issues

• Especially important with clustered or grouped data
• Ultimate success of even well-conceived projects turns on inference details

FAQ	Question	Key Point
1	Causal relationship?	Predicting changes in counterfactual worlds
2	Ideal experiment?	Must conceptualize, even if hypothetical
3	Identification strategy?	Approximate experiments with natural variation
4	Mode of inference?	Population, sample, standard error assumptions