Biostatistics & Epidemiology - Part 4

Measurement, Study Design, Causal Reasoning & Improvement

Austin Meyer, MD, PhD, MS, MPH, MS, FACP, FAAP

August 28, 2026

Learning Objectives

By the end of this session, you should be able to:

  • Classify disease frequency, measurement properties, and study designs in at least 8 of 10 PREP vignettes.
  • Choose the design that best answers a descriptive, associative, or causal question.
  • Calculate and interpret r² from a reported correlation coefficient.
  • Distinguish evidence that supports causation from evidence that proves only association.
  • Recognize a nonrandom shift on a run chart after a process change.

How We’ll Work Through Questions

  • Read the clinical vignette and commit to one answer.
  • Take 20-30 seconds for individual thought or pair-share.
  • Submit the answer to reveal the short explanation.
  • Debrief the governing concept, the distractors, and a visual anchor.
  • Separate what the data show from what we are tempted to claim.

Part 1: Measurement Before Modeling

Start with what is being counted - and what a test can actually tell you.

Question 1

The history of treatment strategies for different renal conditions is being discussed among a team of residents. They discuss that the number of children born with cystinosis has not changed over the past 2 decades. With treatments like kidney transplantation and cysteamine, patients are living longer but are not cured of their disease.

Of the following, the BEST description of the overall incidence and prevalence of cystinosis is

AAP PREP source question 10

Longer Duration Raises Prevalence

Correct Answer: C. Stable incidence; increased prevalence

  • Incidence is the flow of new cases over time. The number of affected births is unchanged.
  • Prevalence is the pool of people currently living with disease.
  • Longer survival increases average disease duration, so the pool grows even when inflow is stable.
  • The useful approximation is: prevalence ≈ incidence × average duration.

Caution

Clinical interpretation: More people living with a condition does not necessarily mean more new disease.

Stable Incidence, Expanding Pool

A flat incidence line and a rising prevalence line after survival improves.

Why the Other Choices Fail

  • A and B: Increased incidence would require more new diagnoses or affected births per unit time; the stem says that number is unchanged.
  • B and D: Stable prevalence would require the larger surviving pool to be offset by fewer new cases, cure, death, or out-migration.
  • Better treatment can increase prevalence while improving outcomes - a useful reminder that prevalence is not a quality metric by itself.

Question 2

A laboratory technology company is trying to conduct a study on its new saliva-based test for group A streptococcus. Company investigators plan on obtaining 3 saliva samples from children who seek treatment in an emergency department for sore throat. The specimen obtained with each swab will be tested using a new rapid antigen detection test.

Of the following, the attribute of this test that could BEST be evaluated in the study is

AAP PREP source question 24

Repeat Testing Measures Precision

Correct Answer: C. precision

  • Three samples from the same child test repeatability: do repeated measurements agree?
  • That is precision (or reliability), not necessarily truth.
  • Accuracy, sensitivity, and predictive values require a reference standard that classifies disease status.
  • A test can be consistently wrong: high precision, low accuracy.
Design principle: Repetition estimates variability. Comparison with truth estimates diagnostic accuracy.

Precision Is Not Accuracy

Repeated points form three patterns: tightly clustered off target, spread around target, and tightly clustered on target.

Part 2: Match the Design to the Question

Ask whether the goal is to describe, associate, or estimate a causal treatment effect.

Question 3

A practice seeks to decrease obesity rates as a way to improve population health and is looking to identify risk factors that may be associated with obesity. The team hypothesizes that children who play video games for more than 2 hours a day have higher body mass index (BMI) percentiles. During one month, a survey records BMI percentile and daily video-game hours at each enrolled child’s annual health supervision visit.

The results are: among 200 children with less than 2 hours/day, 40 have BMI above the 95th percentile; among 100 children with more than 2 hours/day, 50 have BMI above the 95th percentile.

Of the following, the study DESIGN used by the practice was

AAP PREP source question 20; stem condensed without changing design-defining facts

One Time Window Means Cross-Sectional

Correct Answer: B. cross-sectional

  • Exposure and outcome are measured in the same short window.
  • The design estimates prevalence and can quantify an association.
  • Because timing is unresolved, it cannot tell whether gaming preceded higher BMI.
  • It is efficient for hypothesis generation, but vulnerable to confounding and reverse causation.

Cross-Sectional = Snapshot

A time axis with one highlighted window containing both exposure and outcome measurement; temporality is not established.

Study Design Starts With Direction

Start with… Then ask… Typical design What it estimates best
A population now Exposure and outcome together Cross-sectional Prevalence; association
An exposure Who later develops outcome? Cohort Incidence; risk ratio
An outcome What prior exposures differed? Case-control Odds ratio
An intervention What happens after random allocation? Randomized trial Causal treatment effect

Caution

Shortcut to avoid: “Retrospective” describes when data were collected, not the design by itself.

Question 4

An 8-year-old boy with asthma has had 3 exacerbations in 6 months. His mother asks about an herbal supplement after reading comments from parents whose children had fewer attacks while taking it.

Of the following, the MOST valid way to assess the causal effect of this herbal supplement in children with asthma is to perform a

AAP PREP source question 22; stem condensed without changing the causal question

Randomization Best Supports a Causal Effect

Correct Answer: C. randomized controlled trial

Eligible children
→
Random allocation
→
Compare prespecified outcomes
  • Randomization makes treatment assignment independent of prognosis on average.
  • A concurrent comparator estimates what would have happened without the supplement.
  • Blinding, allocation concealment, follow-up, adherence, and outcome definition still determine validity.

Why Not the Observational Designs?

  • Cohort: treatment choice may track asthma severity, family preferences, access, and co-interventions.
  • Cross-sectional: exposure and outcome are measured together; temporality is unresolved.
  • Chart review: this is a data-collection method, not a guarantee of a particular design or causal contrast.
  • Statistical adjustment can address measured confounders; randomization also protects against unmeasured confounding on average.

Question 5

An infectious diseases fellow has cared for 3 very different children with probable new hemophagocytic lymphohistiocytosis (HLH) this month. A colleague identifies 5 additional pediatric cases from the past 2 years. There is little pediatric literature, and the team wants to contribute detailed observations about this rare disorder.

Of the following, the MOST appropriate study design is a

AAP PREP source question 23; stem condensed without changing design-defining facts

Rare and Descriptive Favors a Case Series

Correct Answer: B. case series study

Useful for

Detailed phenotype, presentation, management, and hypothesis generation.

Cannot estimate

Incidence, relative risk, or causal effect without a denominator or comparator.

Watch for

Selection, missing records, publication bias, and overgeneralization.

Right tool, right claim: A case series can be the best first design for a rare condition even though it sits low in an intervention-evidence hierarchy.

Question 6

Patients with hypercholesterolemia are randomized by drawing a numbered paper from a bag. Odd numbers receive a very low-fat diet; even numbers receive a statin. A dietitian investigator instructs participants assigned to the diet. Outcomes are cholesterol and body mass index at 6 months.

Of the following study designs, this randomized study is an example of a/an

AAP PREP source question 25; stem condensed without changing masking facts

The Intervention Makes Masking Impossible

Correct Answer: C. open-label study

Trial description Participants know? Treating team knows? Outcome assessor knows?
Open-label Yes Yes May or may not
Participant-masked No Yes May or may not
Double-masked No No Ideally no
  • Diet versus medication is apparent to participants and the clinical team.
  • Randomization and masking solve different problems: allocation bias versus performance/assessment bias.
  • Objective outcomes and blinded outcome assessment can reduce bias when full masking is infeasible.

Part 3: Association, Assumptions & Interpretation

The test statistic is only as honest as the design and the claim attached to it.

Question 7

A public health analysis reports increased autism rates in areas adjacent to cellular telephone towers. Commentators suggest that tower radiation harms the developing brain. Physicians discuss the distinction between correlation and causation.

Of the following, the finding MOST likely to support a causative relationship is

AAP PREP source question 18 - association-only teaching vignette, not evidence of a causal link

Dose-Response Supports, but Does Not Prove

Correct Answer: A. proportional increase with closer proximity

  • A graded relationship between increasing exposure and outcome is a biological gradient.
  • It strengthens a causal argument because the pattern is more specific than simple co-occurrence.
  • It still does not resolve confounding, exposure misclassification, selection, diagnostic ascertainment, or reverse causation.
  • Hill described causal viewpoints, not a checklist that mechanically proves causation.

Caution

Guardrail: The question asks which result would add support. None of these observational findings alone establishes causality.

One Piece of a Causal Argument

Relative outcome frequency rises across five levels of hypothetical exposure; text notes that this supports but does not prove causation.

Question 8

Two physicians test which festival booth better attracts parents of children under age 3. The puppy booth attracts five 3-year-olds, eight 4-year-olds, and five 5-year-olds. The bubble booth attracts three 18-month-olds, nine 24-month-olds, three 30-month-olds, and one child each aged 5, 6, 7, 8, and 9 years. A t test finds no statistically significant difference in mean age.

Of the following, the t test is NOT an appropriate choice because the

AAP PREP source question 15; all ages and counts retained

The Mean Hides a Long Tail

Correct Answer: B. data are not normally distributed

  • Age is continuous enough for this analysis, and the two booth samples are independent.
  • The bubble sample is strongly right-skewed; a few older siblings pull its mean upward.
  • Bubble booth: mean 3.25 years, median 2 years. Puppy booth: mean = median = 4 years.
  • For the PREP question, the severe skew makes the conventional t test a poor summary of the scientific question.
Nuance: The Mann-Whitney U test compares distributions/ranks, not automatically means or medians. Choose the estimand first; then choose a method robust to the data-generating process.

Look at the Distribution First

The bubble-booth ages cluster at 1.5 to 2.5 years with a long tail to 9; the puppy-booth ages are symmetric from 3 to 5.

Question 9

A prospective observational trial in mechanically ventilated, critically ill adult patients with sepsis reports an association between day-1 brain natriuretic peptide (BNP) levels and maximum Sequential Organ Failure Assessment (SOFA) score. The correlation coefficient is r = 0.800.

Of the following, the value that BEST estimates the proportion of variation in BNP levels statistically explained by the SOFA score is

AAP PREP source question 16; adult source population

Square r to Obtain Explained Variance

Correct Answer: B. 0.640

r² = (0.800)² = 0.640 = 64%
  • r gives the direction and strength of a linear association.
  • r² is the fraction of outcome variance explained by that fitted linear relationship in the sample.
  • The remaining 36% is not explained by this one-predictor linear model.
  • Explained variance is not causal attribution. Confounding and shared causes can produce correlation.

Caution

Pediatric distinction: The calculation transfers across populations; the adult sepsis relationship itself does not.

r = 0.80 Does Not Mean 80% Explained

A strong positive scatterplot is paired with bars labeled 64 percent explained and 36 percent unexplained.

Part 4: Improvement Happens Over Time

A before-and-after average can miss the process story.

Question 10

To reduce wait times for patients to receive chemotherapy during oncology clinic visits, a new procedure is implemented for ordering laboratory tests and writing chemotherapy orders before the visit. Six months later, clinic leaders request an update.

Which display is MOST appropriate for determining whether this procedure reduced wait times?

AAP PREP source question 44

Time-Ordered Data Reveal the Process

Correct Answer: D. run chart

  • A run chart plots a measure in time order and adds a centerline, usually the baseline median.
  • It shows whether a change is followed by a sustained, nonrandom pattern.
  • A common shift rule is 6 or more consecutive points on one side of the median; points on the median are skipped.
  • The other tools help plan or prioritize change but do not show performance over time.

A Sustained Shift Is Visible

Wait times fluctuate around 80 minutes before an intervention, then remain below the baseline median and trend toward 50 minutes.

The Distractors Answer Different QI Questions

Tool Best question
Fishbone diagram What categories of causes might explain the problem?
Key driver diagram What system drivers and change ideas link to our aim?
Pareto chart Which categories account for most of the problem?
Run chart Did performance change over time, and was it sustained?

Ten Questions, Four Habits

  • Define the quantity: incidence is not prevalence; precision is not accuracy.
  • Match design to purpose: describe, associate, or estimate a causal effect.
  • Inspect the data before the p-value: distributions and estimands matter.
  • Use cautious language: association and explained variance are not causal attribution.
  • Keep time visible: improvement is a pattern, not a single before-and-after average.

Before you trust the answer, ask: “What exactly was measured, compared, and claimed?”

One-Minute Retrieval Practice

Without looking back, name the answer to each:

  1. Stable incidence + longer survival → what happens to prevalence?
  2. Repeated measurements without a reference standard assess what property?
  3. Exposure and outcome measured simultaneously → what design?
  4. r = 0.80 → what is r²?
  5. Which chart shows whether a process change is sustained?
Answers: increases; precision; cross-sectional; 0.64; run chart.

The End

Thank you. Questions?