Getting started with benchexcal

The benchexcal package gives you two things:

agreement() — compute the three RCT-DUPLICATE / ENCORE agreement metrics (SA, EA, SD) between an RCT and a real-world emulation;
calibrate() — apply the BenchExCal (Wang et al., CPT 2025) calibration to a Stage 2 RWE estimate, using the divergence observed in a Stage 1 RCT-RWE pair as prior information.

Plus tipping-point sensitivity analysis and forest plots.

library(benchexcal)

1. Agreement metrics

The three metrics from Wang et al. (JAMA 2023). Reproduce LEADER (study #1 in Table 1):

agreement(
  rct_hr = 0.87, rct_lower = 0.78, rct_upper = 0.97,
  rwe_hr = 0.82, rwe_lower = 0.76, rwe_upper = 0.87
)
#> RCT-DUPLICATE / ENCORE agreement metrics
#> ----------------------------------------
#>   RCT estimate:  HR = 0.87 (95% CI: 0.78, 0.97)
#>   RWE estimate:  HR = 0.82 (95% CI: 0.76, 0.87)
#>   Trial type:    superiority
#> 
#>   [1] Statistical significance agreement:  YES (full)
#>   [2] Estimate agreement:                  YES
#>   [3] Standardized difference agreement:   YES  (z = 0.90)

The published SD = 0.90 matches.

For a noninferiority trial, pass trial_type and ni_margin so partial agreement (SAP) can fire:

agreement(
  rct_hr = 1.28, rct_lower = 0.59, rct_upper = 2.79,
  rwe_hr = 1.35, rwe_lower = 0.94, rwe_upper = 1.93,
  trial_type = "noninferiority", ni_margin = 1.225
)
#> RCT-DUPLICATE / ENCORE agreement metrics
#> ----------------------------------------
#>   RCT estimate:  HR = 1.28 (95% CI: 0.59, 2.79)
#>   RWE estimate:  HR = 1.35 (95% CI: 0.94, 1.93)
#>   Trial type:    noninferiority (NI margin = 1.23)
#> 
#>   [1] Statistical significance agreement:  YES (full)
#>   [2] Estimate agreement:                  YES
#>   [3] Standardized difference agreement:   YES  (z = -0.12)

summary() returns a tidy data frame, easy to bind into a report table:

res <- agreement(0.87, 0.78, 0.97, 0.82, 0.76, 0.87)
summary(res)
#>                          metric agree                               detail
#> 1 Statistical significance (SA)  full RCT [0.78, 0.97] vs RWE [0.76, 0.87]
#> 2       Estimate agreement (EA)   yes  RWE HR 0.82 in RCT CI [0.78, 0.97]?
#> 3  Standardized difference (SD)   yes      z = 0.90 (threshold |z| < 1.96)

Looping over a library of trials

The package ships with selected RCT-DUPLICATE estimates:

data(rct_duplicate)
head(rct_duplicate)
#>            trial rct_hr rct_lower rct_upper rwe_hr rwe_lower rwe_upper
#> 1         LEADER   0.87      0.78      0.97   0.82      0.76      0.87
#> 2       EMPA-REG   0.86      0.74      0.99   0.83      0.73      0.95
#> 3         CANVAS   0.86      0.75      0.97   0.77      0.70      0.85
#> 4 DECLARE-TIMI58   0.83      0.73      0.95   0.69      0.59      0.81
#> 5      ARISTOTLE   0.79      0.66      0.95   0.68      0.61      0.76
#> 6      ROCKET-AF   0.79      0.66      0.96   0.70      0.62      0.80
#>       trial_type ni_margin
#> 1    superiority        NA
#> 2    superiority        NA
#> 3    superiority        NA
#> 4    superiority        NA
#> 5    superiority        NA
#> 6 noninferiority      1.46

# Apply agreement() to every row
do.call(rbind, lapply(seq_len(nrow(rct_duplicate)), function(i) {
  r <- rct_duplicate[i, ]
  a <- agreement(r$rct_hr, r$rct_lower, r$rct_upper,
                 r$rwe_hr, r$rwe_lower, r$rwe_upper,
                 trial_type = r$trial_type,
                 ni_margin  = if (is.na(r$ni_margin)) NULL else r$ni_margin)
  data.frame(trial = r$trial,
             SA = a$sa$status,
             EA = a$ea,
             SD = a$sd$agree,
             z  = round(a$sd$z, 2))
}))
#>             trial   SA    EA   SD     z
#> 1          LEADER full  TRUE TRUE  0.90
#> 2        EMPA-REG full  TRUE TRUE  0.35
#> 3          CANVAS full  TRUE TRUE  1.34
#> 4  DECLARE-TIMI58 full FALSE TRUE  1.76
#> 5       ARISTOTLE full  TRUE TRUE  1.38
#> 6       ROCKET-AF full  TRUE TRUE  1.05
#> 7           RE-LY full  TRUE TRUE -0.66
#> 8         RECORD1 full  TRUE TRUE  0.94
#> 9    EINSTEIN-DVT   no  TRUE TRUE -0.41
#> 10        AMPLIFY full  TRUE TRUE  0.13
#> 11      PRONOUNCE full  TRUE TRUE -0.12
#> 12       CAROLINA full  TRUE TRUE  0.70

2. BenchExCal calibration

If you forget the inputs:

calibration_inputs()
#> BenchExCal calibration requires three sets of HR + 95% CI:
#> ------------------------------------------------------------
#>   Stage 1 RCT  (existing indication; completed trial):
#>       stage1_rct_hr, stage1_rct_lower, stage1_rct_upper
#> 
#>   Stage 1 RWE  (database study emulating the Stage 1 RCT):
#>       stage1_rwe_hr, stage1_rwe_lower, stage1_rwe_upper
#> 
#>   Stage 2 RWE  (database study for the supplemental indication):
#>       stage2_rwe_hr, stage2_rwe_lower, stage2_rwe_upper
#> 
#> Derived quantities (all on the log HR scale):
#>     xi_hat_1   = log(stage1_rwe_hr) - log(stage1_rct_hr)
#>     xi_hat_2   = sqrt(var2*/var1*) * xi_hat_1
#>     theta_2_cal     = log(stage2_rwe_hr) - xi_hat_2
#>     var_2_cal       = var(theta*_2) + (var(theta*_1) + var(theta_1)) * var(theta*_2)/var(theta*_1)
#> 
#> Reference: Wang SV et al. CPT 2025;117:1820-1828 (doi:10.1002/cpt.3621)

A worked example. Stage 1 = LEADER. Stage 2 = a hypothetical expanded indication where we have only an RWE estimate:

cal <- calibrate(
  stage1_rct_hr = 0.87, stage1_rct_lower = 0.78, stage1_rct_upper = 0.97,
  stage1_rwe_hr = 0.82, stage1_rwe_lower = 0.76, stage1_rwe_upper = 0.87,
  stage2_rwe_hr = 0.85, stage2_rwe_lower = 0.78, stage2_rwe_upper = 0.93
)
print(cal)
#> BenchExCal calibration
#> ======================
#> 
#> Stage 1 benchmark check:
#>   SA: full     EA: yes  SD: yes (z = 0.90)
#> 
#> Stage 1 divergence:
#>   xi_hat_1 (log HR)  = -0.0592
#>   HR ratio (RWE/RCT) = 0.943
#>   Var(xi_hat_1)      = 0.0043
#> 
#> Rescaling to Stage 2:
#>   Scale factor       = 1.301
#>   xi_hat_2 (log HR)  = -0.0770
#>   Prior var(xi_2)    = 0.0073
#> 
#> Stage 2 RWE estimates:
#>   Uncalibrated  HR = 0.850  (95% CI: 0.780, 0.930)
#>   Calibrated    HR = 0.918  (95% CI: 0.760, 1.109)

Note three things in the output:

xi_hat_1 < 0 means the Stage 1 RWE was more protective than the RCT.
The calibrated Stage 2 HR is shifted toward the null to remove that apparent bias.
The calibrated 95% CI is wider than the uncalibrated one — this is the bias-variance trade-off that BenchExCal makes explicit.

Forest plot

if (requireNamespace("ggplot2", quietly = TRUE)) plot(cal)
#> `height` was translated to `width`.

3. Tipping-point sensitivity

tp <- tipping_point(cal)
print(tp)
#> BenchExCal tipping point analysis
#> ---------------------------------
#>   Stage 2 xi_hat_2 (prior mean) = -0.0770  (HR ratio 0.926)
#>   Prior SD of xi_2              = 0.0851
#>   Decision threshold (HR)       = 1.00
#>   Grid range                    = [-0.217, 0.063]
#> 
#>   Tipping point xi_2            = -0.0735
#>   Tipping point HR ratio        = 0.929
#>   Distance from prior mean      = 0.04 prior SD
#> 
#>   -> Tipping point is INSIDE the prior 95% interval.
#>      Conclusion is SENSITIVE to plausible Stage 1 divergence.

if (requireNamespace("ggplot2", quietly = TRUE)) plot(tp)

If the tipping point sits outside the prior 95% interval for ξ₂, the Stage 2 conclusion is robust to plausible Stage 1–type bias. If it sits inside, the regulatory claim should be treated with caution.

References

Wang SV, Schneeweiss S, RCT-DUPLICATE Initiative. Emulation of randomized clinical trials with nonrandomized database analyses: results of 32 clinical trials. JAMA. 2023;329:1376-1385.
Wang SV, Russo M, Glynn RJ, et al. A Benchmark, Expand, and Calibration (BenchExCal) Trial Emulation Approach for Using Real-World Evidence to Support Indication Expansions. Clin Pharmacol Ther. 2025;117:1820-1828.
Weberpals J, Schneeweiss S, et al. Emulating Comparative Oncology Trials With Real-World Evidence Studies (ENCORE). Clin Pharmacol Ther. 2026.