Graduation Year

2025

Document Type

Dissertation

Degree

Ph.D.

Degree Name

Doctor of Philosophy (Ph.D.)

Degree Granting Department

Public Health

Major Professor

Henian Chen, Ph.D.

Committee Member

Stephanie Marhefka-Day, Ph.D.

Committee Member

Wei Wang, Ph.D.

Committee Member

Yangxin Huang, Ph.D.

Committee Member

Matthew Valente, Ph.D.

Keywords

Clinical Trials, Effect size, Effect Size Adjustments, Pilot Trials, Sample Size, Trial Simulations

Abstract

Background: The effect size estimated from a pilot trial is often an inaccurate reflection of the true effect size observed in a large trial, leading to either underestimation or overestimation. Published data suggest that effect sizes from large trials are typically smaller than those reported in their corresponding pilot trials. To address this discrepancy, conservative or discount adjustment methods are widely recommended to modify pilot trial effect sizes when calculating sample sizes, thereby maintaining adequate statistical power. This study aims to assess effect sizes from both pilot and large trials and to evaluate the performance of existing adjustment methods.

Methods: Published effect sizes from both pilot trials and large trials were systematically reviewed. Existing adjustment methods for pilot trial effect sizes, including multiplicative, additive, lower confidence limit (LCL), and the Dual-Bound Weighted method, were evaluated through simulated clinical trials across a range of true effect sizes and sample sizes. Both underestimation and overestimation were examined and compared.

Results: When pilot estimates showed upward bias, conservative methods reduced overestimation and improved accuracy. However, when pilot trials underestimated the true effect, these methods exaggerated underestimation, leading to excessively large sample size recommendations. Performance varied by context: multiplicative and 60% LCL adjustments worked well for small and moderate effects under random or upward bias, but none were optimal across all scenarios. The proposed DB-weighted method achieved greater balance and accuracy for larger sample sizes and effect sizes, regardless of bias direction.

Conclusion: Findings challenge the universal use of conservative adjustments of pilot ES estimates. Researchers should tailor adjustment strategies based on bias direction, sample size, and effect size magnitude. The DB-weighted method provides a more balanced alternative when bias direction is uncertain. This study offers practical guidance for selecting adjustment methods based on suspected bias direction, sample size, and effect size magnitude.

Download

Included in

Biostatistics Commons

Share

COinS