Introduction

ASSISTant is an R package for Adaptive Subgroup Selection In Sequential Trials. This vignette reproduces all the simulations in the original paper of Lai, Lavori and Liao [-@Lai2014191].

library(ASSISTant)
data(LLL.SETTINGS)
str(LLL.SETTINGS)
## List of 3
##  $ trialParameters:List of 4
##   ..$ N         : num [1:3] 300 400 500
##   ..$ type1Error: num 0.05
##   ..$ eps       : num 0.5
##   ..$ type2Error: num 0.2
##  $ scenarios      :List of 11
##   ..$ S0 :List of 2
##   .. ..$ mean: num [1:2, 1:6] 0 0 0 0 0 0 0 0 0 0 ...
##   .. ..$ sd  : num [1:2, 1:6] 1 1 1 1 1 1 1 1 1 1 ...
##   ..$ S1 :List of 2
##   .. ..$ mean: num [1:2, 1:6] 0 0.3 0 0.3 0 0.3 0 0.3 0 0.3 ...
##   .. ..$ sd  : num [1:2, 1:6] 1 1 1 1 1 1 1 1 1 1 ...
##   ..$ S2 :List of 2
##   .. ..$ mean: num [1:2, 1:6] 0 0.3 0 0.3 0 0.3 0 0.3 0 0 ...
##   .. ..$ sd  : num [1:2, 1:6] 1 1 1 1 1 1 1 1 1 1 ...
##   ..$ S3 :List of 2
##   .. ..$ mean: num [1:2, 1:6] 0 0.3 0 0.3 0 0 0 0 0 0 ...
##   .. ..$ sd  : num [1:2, 1:6] 1 1 1 1 1 1 1 1 1 1 ...
##   ..$ S4 :List of 2
##   .. ..$ mean: num [1:2, 1:6] 0 0 0 0 0 0.3 0 0.3 0 0 ...
##   .. ..$ sd  : num [1:2, 1:6] 1 1 1 1 1 1 1 1 1 1 ...
##   ..$ S5 :List of 2
##   .. ..$ mean: num [1:2, 1:6] 0 0.6 0 0 0 0 0 0 0 0 ...
##   .. ..$ sd  : num [1:2, 1:6] 1 1 1 1 1 1 1 1 1 1 ...
##   ..$ S6 :List of 2
##   .. ..$ mean: num [1:2, 1:6] 0 0.5 0 0 0 0 0 0 0 0 ...
##   .. ..$ sd  : num [1:2, 1:6] 1 1 1 1 1 1 1 1 1 1 ...
##   ..$ S7 :List of 2
##   .. ..$ mean: num [1:2, 1:6] 0 0.5 0 0.4 0 0.3 0 0 0 0 ...
##   .. ..$ sd  : num [1:2, 1:6] 1 1 1 1 1 1 1 1 1 1 ...
##   ..$ S8 :List of 2
##   .. ..$ mean: num [1:2, 1:6] 0 0.5 0 0.4 0 0.3 0 0 0 0 ...
##   .. ..$ sd  : num [1:2, 1:6] 1 2 1 1.5 1 1 1 0.5 1 0.5 ...
##   ..$ S9 :List of 2
##   .. ..$ mean: num [1:2, 1:6] 0 0.5 0 0.4 0 0.3 0 0 0 0 ...
##   .. ..$ sd  : num [1:2, 1:6] 1 0.5 1 1 1 1.5 1 2 1 2 ...
##   ..$ S10:List of 2
##   .. ..$ mean: num [1:2, 1:6] 0 0 0 0 0 0 0 0.3 0 0.4 ...
##   .. ..$ sd  : num [1:2, 1:6] 1 1 1 1 1 1 1 1 1 1 ...
##  $ prevalences    :List of 2
##   ..$ table1: num [1:6] 0.167 0.167 0.167 0.167 0.167 ...
##   ..$ table2: num [1:6] 0.2 0.1 0.3 0.1 0.1 0.2

The LLL.SETTINGS list contains all the scenarios used for the null and alternative cases in Lai, Lavori and Liao [-@Lai2014191].

Table 1 Results

The results shown here should closely approximate those in Table 1 of Lai, Lavori and Liao [-@Lai2014191].

Scenario S0

This is the null setting.

scenario <- LLL.SETTINGS$scenarios$S0
designParameters <- list(prevalence = LLL.SETTINGS$prevalences$table1,
                       mean = scenario$mean,
                       sd = scenario$sd)
designA <- ASSISTDesign$new(trialParameters = LLL.SETTINGS$trialParameters,
                            designParameters = designParameters)
print(designA)
## Design Parameters:
##  Number of Groups: 6
##  Prevalence:
## 
##     Group1      Group2      Group3      Group4      Group5      Group6
## ----------  ----------  ----------  ----------  ----------  ----------
##  0.1666667   0.1666667   0.1666667   0.1666667   0.1666667   0.1666667
## 
##  Using Discrete Rankin scores? FALSE
## 
##  Normal Rankin Distribution means (null row, alt. row):
## 
## 
##         Group1   Group2   Group3   Group4   Group5   Group6
## -----  -------  -------  -------  -------  -------  -------
## Null         0        0        0        0        0        0
## Alt          0        0        0        0        0        0
## 
##  Normal Rankin Distribution SDs (null row, alt. row):
## 
## 
##         Group1   Group2   Group3   Group4   Group5   Group6
## -----  -------  -------  -------  -------  -------  -------
## Null         1        1        1        1        1        1
## Alt          1        1        1        1        1        1
## 
## Trial Parameters:
## List of 5
##  $ N         : num [1:3] 300 400 500
##  $ type1Error: num 0.05
##  $ eps       : num 0.5
##  $ type2Error: num 0.2
##  $ effectSize: num 0.0642
## 
## Boundaries:
## 
## 
##     btilde          b          c
## ----------  ---------  ---------
##  -1.460993   2.390404   2.491775
result <- designA$explore(numberOfSimulations = 25000, showProgress = FALSE)
analysis <- designA$analyze(result)
print(designA$summary(analysis))
## P(Reject H0_ITT) = 0.015720; P(Reject H0_subgp) = 0.021520; P(Reject H0) = 0.037240
## P(Early stop for efficacy [futility]) = 0.021680 [0.551640]
## Mean [SD] Randomized N = 422.940000 [75.588259]
## 
## Stage at exit (proportion)
## 
## 
##  exitStage   proportion
## ----------  -----------
##          1      0.19728
##          2      0.37604
##          3      0.42668
## 
## Mean [SD] Lost N = 183.079080 [92.140910]
## Mean [SD] Analyzed N = 239.860920 [97.090027]
## 
## Mean loss by futility stage and subgroup
## 
## 
##  FutilityStage   selectedGroup        mean          sd
## --------------  --------------  ----------  ----------
##              1               1   250.06551    6.505267
##              1               2   200.00203    8.154904
##              1               3   150.16620    8.346300
##              1               4   100.00972    8.101806
##              1               5    49.86783    6.502325
##              2               1   333.39802    7.343570
##              2               2   266.72910    9.561764
##              2               3   199.64773    9.832610
##              2               4   133.08841    9.191855
##              2               5    66.74314    7.428316
##              3               1   415.78818    8.423654
##              3               2   333.04252   10.739814
##              3               3   250.23131   11.191329
##              3               4   166.77711   10.644704
##              3               5    83.37944    8.433986
## 
## Chance of each subpopulation rejected
## 
## 
##  group   count   proportion
## ------  ------  -----------
##      1     217      0.00868
##      2     134      0.00536
##      3      88      0.00352
##      4      63      0.00252
##      5      36      0.00144
##      6     393      0.01572
## 
## Counts by futility stage and subgroup choice
## 
## 
##  FutilityStage     G1     G2     G3     G4     G5
## --------------  -----  -----  -----  -----  -----
##              1   6259   3450   2491   2161   2527
##              2    809    598    528    509    802
##              3    694    635    709    830   1605
## 
## CI Statistics:
## Overall coverage and coverage for rejections:
## 
##  overall   rejection
## --------  ----------
##  0.96276           0
## 
## P(theta_test is in the confidence interval)
## 
## 
##   coverage   selectedCount   rejectedCount
## ----------  --------------  --------------
##  0.9720433            7762             217
##  0.9713859            4683             134
##  0.9763948            3728              88
##  0.9820000            3500              63
##  0.9927037            4934              36
##  0.0000000             393             393
## NULL

Alternative Scenario S1

scenario <- LLL.SETTINGS$scenarios$S1
trueParameters <- list(prevalence = LLL.SETTINGS$prevalences$table1,
                       mean = scenario$mean,
                       sd = scenario$sd)
result <- designA$explore(numberOfSimulations = 25000,
                          trueParameters = trueParameters, showProgress = FALSE)
analysis <- designA$analyze(result)
print(designA$summary(analysis))
## P(Reject H0_ITT) = 0.819480; P(Reject H0_subgp) = 0.041320; P(Reject H0) = 0.860800
## P(Early stop for efficacy [futility]) = 0.738760 [0.004400]
## Mean [SD] Randomized N = 369.756000 [85.128082]
## 
## Stage at exit (proportion)
## 
## 
##  exitStage   proportion
## ----------  -----------
##          1      0.55928
##          2      0.18388
##          3      0.25684
## 
## Mean [SD] Lost N = 30.983720 [80.251024]
## Mean [SD] Analyzed N = 338.772280 [77.840612]
## 
## Mean loss by futility stage and subgroup
## 
## 
##  FutilityStage   selectedGroup        mean          sd
## --------------  --------------  ----------  ----------
##              1               1   250.74803    6.845191
##              1               2   199.69000    8.585512
##              1               3   151.00000    9.284578
##              1               4   101.27536    7.204745
##              1               5    50.47368    6.879765
##              2               1   333.81250    7.642992
##              2               2   266.90323    8.821772
##              2               3   197.07143   10.811272
##              2               4   131.88235   10.641207
##              2               5    68.48485    8.419179
##              3               1   417.05574    8.219051
##              3               2   333.57279   10.804541
##              3               3   249.69643   11.374125
##              3               4   166.51711   10.276570
##              3               5    83.35188    8.242731
## 
## Chance of each subpopulation rejected
## 
## 
##  group   count   proportion
## ------  ------  -----------
##      1     198      0.00792
##      2     202      0.00808
##      3     185      0.00740
##      4     216      0.00864
##      5     232      0.00928
##      6   20487      0.81948
## 
## Counts by futility stage and subgroup choice
## 
## 
##  FutilityStage    G1    G2    G3    G4     G5
## --------------  ----  ----  ----  ----  -----
##              1   127   100    60    69    114
##              2    32    31    28    17     33
##              3   305   419   448   789   1941
## 
## CI Statistics:
## Overall coverage and coverage for rejections:
## 
##  overall   rejection
## --------  ----------
##   0.1392           0
## 
## P(theta_test is in the confidence interval)
## 
## 
##   coverage   selectedCount   rejectedCount
## ----------  --------------  --------------
##  0.5732759             464             198
##  0.6327273             550             202
##  0.6548507             536             185
##  0.7531429             875             216
##  0.8888889            2088             232
##  0.0000000           20487           20487
## NULL

Alternative Scenario S2

scenario <- LLL.SETTINGS$scenarios$S2
trueParameters <- list(prevalence = LLL.SETTINGS$prevalences$table1,
                       mean = scenario$mean,
                       sd = scenario$sd)
result <- designA$explore(numberOfSimulations = 25000,
                          trueParameters = trueParameters, showProgress = FALSE)
analysis <- designA$analyze(result)
print(designA$summary(analysis))
## P(Reject H0_ITT) = 0.444520; P(Reject H0_subgp) = 0.302720; P(Reject H0) = 0.747240
## P(Early stop for efficacy [futility]) = 0.422360 [0.010240]
## Mean [SD] Randomized N = 432.208000 [84.202900]
## 
## Stage at exit (proportion)
## 
## 
##  exitStage   proportion
## ----------  -----------
##          1      0.24532
##          2      0.18728
##          3      0.56740
## 
## Mean [SD] Lost N = 105.824320 [114.155552]
## Mean [SD] Analyzed N = 326.383680 [92.122986]
## 
## Mean loss by futility stage and subgroup
## 
## 
##  FutilityStage   selectedGroup        mean          sd
## --------------  --------------  ----------  ----------
##              1               1   250.06935    6.288181
##              1               2   199.14115    8.462123
##              1               3   150.57754    8.686286
##              1               4   101.57600    8.149609
##              1               5    50.34978    6.742603
##              2               1   332.94286    7.950070
##              2               2   267.72455    8.767866
##              2               3   199.81893   10.605244
##              2               4   133.65535    9.752769
##              2               5    66.58929    6.701558
##              3               1   415.89153    8.565840
##              3               2   333.14062   10.516930
##              3               3   250.22757   11.341056
##              3               4   167.04975   10.530577
##              3               5    84.11860    8.296960
## 
## Chance of each subpopulation rejected
## 
## 
##  group   count   proportion
## ------  ------  -----------
##      1     965      0.03860
##      2    1099      0.04396
##      3    1586      0.06344
##      4    3400      0.13600
##      5     518      0.02072
##      6   11113      0.44452
## 
## Counts by futility stage and subgroup choice
## 
## 
##  FutilityStage    G1     G2     G3     G4     G5
## --------------  ----  -----  -----  -----  -----
##              1   620    503    561    875    223
##              2   175    167    243    383    112
##              3   590   1024   1705   5045   1661
## 
## CI Statistics:
## Overall coverage and coverage for rejections:
## 
##  overall   rejection
## --------  ----------
##  0.25276           0
## 
## P(theta_test is in the confidence interval)
## 
## 
##   coverage   selectedCount   rejectedCount
## ----------  --------------  --------------
##  0.3032491            1385             965
##  0.3512397            1694            1099
##  0.3678756            2509            1586
##  0.4605743            6303            3400
##  0.7404810            1996             518
##  0.0000000           11113           11113
## NULL

Alternative Scenario S3

scenario <- LLL.SETTINGS$scenarios$S3
trueParameters <- list(prevalence = LLL.SETTINGS$prevalences$table1,
                       mean = scenario$mean,
                       sd = scenario$sd)
result <- designA$explore(numberOfSimulations = 25000,
                          trueParameters = trueParameters, showProgress = FALSE)
analysis <- designA$analyze(result)
print(designA$summary(analysis))
## P(Reject H0_ITT) = 0.119920; P(Reject H0_subgp) = 0.483400; P(Reject H0) = 0.603320
## P(Early stop for efficacy [futility]) = 0.310760 [0.036120]
## Mean [SD] Randomized N = 455.568000 [66.467843]
## 
## Stage at exit (proportion)
## 
## 
##  exitStage   proportion
## ----------  -----------
##          1      0.09744
##          2      0.24944
##          3      0.65312
## 
## Mean [SD] Lost N = 214.930040 [116.264237]
## Mean [SD] Analyzed N = 240.637960 [101.810594]
## 
## Mean loss by futility stage and subgroup
## 
## 
##  FutilityStage   selectedGroup        mean          sd
## --------------  --------------  ----------  ----------
##              1               1   249.99010    6.363654
##              1               2   200.69111    8.110211
##              1               3   150.69974    8.354582
##              1               4   100.35740    8.006240
##              1               5    50.32763    6.353959
##              2               1   333.25714    7.515349
##              2               2   266.78197    9.517153
##              2               3   200.83493   10.004269
##              2               4   134.26860    9.824084
##              2               5    67.76437    7.492323
##              3               1   416.56916    8.565990
##              3               2   332.64176   10.645336
##              3               3   249.99475   10.837450
##              3               4   166.25416   10.285499
##              3               5    83.35851    8.447371
## 
## Chance of each subpopulation rejected
## 
## 
##  group   count   proportion
## ------  ------  -----------
##      1    3353      0.13412
##      2    6881      0.27524
##      3    1257      0.05028
##      4     428      0.01712
##      5     166      0.00664
##      6    2998      0.11992
## 
## Counts by futility stage and subgroup choice
## 
## 
##  FutilityStage     G1     G2     G3     G4     G5
## --------------  -----  -----  -----  -----  -----
##              1   2625   3969   1139    554    409
##              2    665   1431    418    242    174
##              3   1446   4871   1713   1141   1205
## 
## CI Statistics:
## Overall coverage and coverage for rejections:
## 
##  overall   rejection
## --------  ----------
##  0.39668           0
## 
## P(theta_test is in the confidence interval)
## 
## 
##   coverage   selectedCount   rejectedCount
## ----------  --------------  --------------
##  0.2920186            4736            3353
##  0.3300555           10271            6881
##  0.6155963            3270            1257
##  0.7790398            1937             428
##  0.9071588            1788             166
##  0.0000000            2998            2998
## NULL

Alternative Scenario S4

scenario <- LLL.SETTINGS$scenarios$S4
trueParameters <- list(prevalence = LLL.SETTINGS$prevalences$table1,
                       mean = scenario$mean,
                       sd = scenario$sd)
result <- designA$explore(numberOfSimulations = 25000,
                          trueParameters = trueParameters, showProgress = FALSE)
analysis <- designA$analyze(result)
print(designA$summary(analysis))
## P(Reject H0_ITT) = 0.117720; P(Reject H0_subgp) = 0.092480; P(Reject H0) = 0.210200
## P(Early stop for efficacy [futility]) = 0.109280 [0.221080]
## Mean [SD] Randomized N = 451.676000 [74.531961]
## 
## Stage at exit (proportion)
## 
## 
##  exitStage   proportion
## ----------  -----------
##          1      0.15288
##          2      0.17748
##          3      0.66964
## 
## Mean [SD] Lost N = 125.679680 [82.598267]
## Mean [SD] Analyzed N = 325.996320 [89.660781]
## 
## Mean loss by futility stage and subgroup
## 
## 
##  FutilityStage   selectedGroup        mean          sd
## --------------  --------------  ----------  ----------
##              1               1   250.10337    6.499087
##              1               2   199.26116    8.388557
##              1               3   150.03446    8.389836
##              1               4   100.10910    8.013028
##              1               5    50.31102    6.601388
##              2               1   333.86538    7.490690
##              2               2   266.46970    9.298833
##              2               3   199.29499   10.203912
##              2               4   133.53675    9.479584
##              2               5    66.78419    7.552403
##              3               1   415.84257    8.287152
##              3               2   332.02041   11.028529
##              3               3   249.39495   12.095256
##              3               4   166.46920   10.588489
##              3               5    83.27237    8.374691
## 
## Chance of each subpopulation rejected
## 
## 
##  group   count   proportion
## ------  ------  -----------
##      1      94      0.00376
##      2      23      0.00092
##      3     283      0.01132
##      4    1525      0.06100
##      5     387      0.01548
##      6    2943      0.11772
## 
## Counts by futility stage and subgroup choice
## 
## 
##  FutilityStage     G1    G2     G3     G4     G5
## --------------  -----  ----  -----  -----  -----
##              1   1693   448   1248   3538   1897
##              2    312    66    339   1347    797
##              3    343    98    752   5324   3855
## 
## CI Statistics:
## Overall coverage and coverage for rejections:
## 
##  overall   rejection
## --------  ----------
##   0.7898           0
## 
## P(theta_test is in the confidence interval)
## 
## 
##   coverage   selectedCount   rejectedCount
## ----------  --------------  --------------
##  0.9599659            2348              94
##  0.9624183             612              23
##  0.8790081            2339             283
##  0.8506220           10209            1525
##  0.9409070            6549             387
##  0.0000000            2943            2943
## NULL

Alternative Scenario S5

scenario <- LLL.SETTINGS$scenarios$S5
trueParameters <- list(prevalence = LLL.SETTINGS$prevalences$table1,
                       mean = scenario$mean,
                       sd = scenario$sd)
result <- designA$explore(numberOfSimulations = 25000,
                          trueParameters = trueParameters, showProgress = FALSE)
analysis <- designA$analyze(result)
print(designA$summary(analysis))
## P(Reject H0_ITT) = 0.112200; P(Reject H0_subgp) = 0.712720; P(Reject H0) = 0.824920
## P(Early stop for efficacy [futility]) = 0.458320 [0.009160]
## Mean [SD] Randomized N = 439.888000 [71.209201]
## 
## Stage at exit (proportion)
## 
## 
##  exitStage   proportion
## ----------  -----------
##          1      0.13364
##          2      0.33384
##          3      0.53252
## 
## Mean [SD] Lost N = 272.505440 [130.984436]
## Mean [SD] Analyzed N = 167.382560 [109.854833]
## 
## Mean loss by futility stage and subgroup
## 
## 
##  FutilityStage   selectedGroup        mean          sd
## --------------  --------------  ----------  ----------
##              1               1   250.61911    6.350835
##              1               2   201.28187    8.096758
##              1               3   151.29754    8.615794
##              1               4   101.04831    7.970674
##              1               5    49.88679    6.507608
##              2               1   333.59851    7.425682
##              2               2   267.68451   10.119995
##              2               3   200.02098   10.522590
##              2               4   134.60294   11.486708
##              2               5    66.31111    6.947276
##              3               1   415.93086    8.274362
##              3               2   333.39952   10.055650
##              3               3   251.62166   11.412356
##              3               4   167.36384   10.148569
##              3               5    83.91832    8.045223
## 
## Chance of each subpopulation rejected
## 
## 
##  group   count   proportion
## ------  ------  -----------
##      1   14881      0.59524
##      2    2049      0.08196
##      3     575      0.02300
##      4     223      0.00892
##      5      90      0.00360
##      6    2805      0.11220
## 
## Counts by futility stage and subgroup choice
## 
## 
##  FutilityStage     G1     G2    G3    G4    G5
## --------------  -----  -----  ----  ----  ----
##              1   7157   1114   447   207   106
##              2   2289    355   143    68    45
##              3   7478   1259   637   437   453
## 
## CI Statistics:
## Overall coverage and coverage for rejections:
## 
##  overall   rejection
## --------  ----------
##  0.17508           0
## 
## P(theta_test is in the confidence interval)
## 
## 
##   coverage   selectedCount   rejectedCount
## ----------  --------------  --------------
##  0.1207161           16924           14881
##  0.2489003            2728            2049
##  0.5313773            1227             575
##  0.6867978             712             223
##  0.8509934             604              90
##  0.0000000            2805            2805
## NULL

Alternative Scenario S6

scenario <- LLL.SETTINGS$scenarios$S6
trueParameters <- list(prevalence = LLL.SETTINGS$prevalences$table1,
                       mean = scenario$mean,
                       sd = scenario$sd)
result <- designA$explore(numberOfSimulations = 25000,
                          trueParameters = trueParameters, showProgress = FALSE)
analysis <- designA$analyze(result)
print(designA$summary(analysis))
## P(Reject H0_ITT) = 0.088200; P(Reject H0_subgp) = 0.647760; P(Reject H0) = 0.735960
## P(Early stop for efficacy [futility]) = 0.438480 [0.023120]
## Mean [SD] Randomized N = 443.132000 [67.783585]
## 
## Stage at exit (proportion)
## 
## 
##  exitStage   proportion
## ----------  -----------
##          1      0.10708
##          2      0.35452
##          3      0.53840
## 
## Mean [SD] Lost N = 262.750280 [123.651895]
## Mean [SD] Analyzed N = 180.381720 [108.608932]
## 
## Mean loss by futility stage and subgroup
## 
## 
##  FutilityStage   selectedGroup        mean          sd
## --------------  --------------  ----------  ----------
##              1               1   250.51889    6.450511
##              1               2   200.90226    8.124082
##              1               3   150.83535    8.403142
##              1               4   100.71105    7.656794
##              1               5    50.50570    6.403047
##              2               1   333.58966    7.190907
##              2               2   267.28571    9.585064
##              2               3   201.29353    9.338544
##              2               4   135.09322    8.838520
##              2               5    67.34343    6.805103
##              3               1   416.17383    8.299780
##              3               2   333.59795   10.716808
##              3               3   250.24868   10.787442
##              3               4   166.92284   10.596297
##              3               5    83.36292    8.215925
## 
## Chance of each subpopulation rejected
## 
## 
##  group   count   proportion
## ------  ------  -----------
##      1   13363      0.53452
##      2    1912      0.07648
##      3     581      0.02324
##      4     242      0.00968
##      5      96      0.00384
##      6    2205      0.08820
## 
## Counts by futility stage and subgroup choice
## 
## 
##  FutilityStage     G1     G2    G3    G4    G5
## --------------  -----  -----  ----  ----  ----
##              1   7516   1463   662   353   263
##              2   2264    448   201   118    99
##              3   6006   1363   760   648   631
## 
## CI Statistics:
## Overall coverage and coverage for rejections:
## 
##  overall   rejection
## --------  ----------
##  0.26404           0
## 
## P(theta_test is in the confidence interval)
## 
## 
##   coverage   selectedCount   rejectedCount
## ----------  --------------  --------------
##  0.1534904           15786           13363
##  0.4160049            3274            1912
##  0.6420209            1623             581
##  0.7837355            1119             242
##  0.9033233             993              96
##  0.0000000            2205            2205
## NULL

Alternative Scenario S7

scenario <- LLL.SETTINGS$scenarios$S7
trueParameters <- list(prevalence = LLL.SETTINGS$prevalences$table1,
                       mean = scenario$mean,
                       sd = scenario$sd)
result <- designA$explore(numberOfSimulations = 25000,
                          trueParameters = trueParameters, showProgress = FALSE)
analysis <- designA$analyze(result)
print(designA$summary(analysis))
## P(Reject H0_ITT) = 0.433400; P(Reject H0_subgp) = 0.454320; P(Reject H0) = 0.887720
## P(Early stop for efficacy [futility]) = 0.469720 [0.002000]
## Mean [SD] Randomized N = 427.720000 [83.818152]
## 
## Stage at exit (proportion)
## 
## 
##  exitStage   proportion
## ----------  -----------
##          1      0.25108
##          2      0.22064
##          3      0.52828
## 
## Mean [SD] Lost N = 144.721680 [140.721373]
## Mean [SD] Analyzed N = 282.998320 [106.573779]
## 
## Mean loss by futility stage and subgroup
## 
## 
##  FutilityStage   selectedGroup        mean          sd
## --------------  --------------  ----------  ----------
##              1               1   250.65160    6.357390
##              1               2   201.11829    7.879522
##              1               3   151.74350    8.617223
##              1               4   101.46341    7.951070
##              1               5    49.38889    5.644890
##              2               1   333.56855    7.207968
##              2               2   267.12276    9.561154
##              2               3   202.55902    9.926593
##              2               4   134.51899   10.117085
##              2               5    68.83333    7.124158
##              3               1   416.90893    8.489855
##              3               2   333.34991   10.374732
##              3               3   250.60999   11.257656
##              3               4   167.80642   11.008029
##              3               5    84.16252    8.974761
## 
## Chance of each subpopulation rejected
## 
## 
##  group   count   proportion
## ------  ------  -----------
##      1    1830      0.07320
##      2    3492      0.13968
##      3    5059      0.20236
##      4     789      0.03156
##      5     188      0.00752
##      6   10835      0.43340
## 
## Counts by futility stage and subgroup choice
## 
## 
##  FutilityStage     G1     G2     G3     G4    G5
## --------------  -----  -----  -----  -----  ----
##              1    752    913    885    164    54
##              2    248    391    449     79    24
##              3   1153   2715   4787   1028   523
## 
## CI Statistics:
## Overall coverage and coverage for rejections:
## 
##  overall   rejection
## --------  ----------
##  0.11228           0
## 
## P(theta_test is in the confidence interval)
## 
## 
##   coverage   selectedCount   rejectedCount
## ----------  --------------  --------------
##  0.1500232            2153            1830
##  0.1311271            4019            3492
##  0.1735011            6121            5059
##  0.3792290            1271             789
##  0.6871880             601             188
##  0.0000000           10835           10835
## NULL

Alternative Scenario S8

scenario <- LLL.SETTINGS$scenarios$S8
trueParameters <- list(prevalence = LLL.SETTINGS$prevalences$table1,
                       mean = scenario$mean,
                       sd = scenario$sd)
result <- designA$explore(numberOfSimulations = 25000,
                          trueParameters = trueParameters, showProgress = FALSE)
analysis <- designA$analyze(result)
print(designA$summary(analysis))
## P(Reject H0_ITT) = 0.267640; P(Reject H0_subgp) = 0.415960; P(Reject H0) = 0.683600
## P(Early stop for efficacy [futility]) = 0.334600 [0.017600]
## Mean [SD] Randomized N = 449.700000 [74.270658]
## 
## Stage at exit (proportion)
## 
## 
##  exitStage   proportion
## ----------  -----------
##          1       0.1508
##          2       0.2014
##          3       0.6478
## 
## Mean [SD] Lost N = 167.374000 [123.307633]
## Mean [SD] Analyzed N = 282.326000 [98.454282]
## 
## Mean loss by futility stage and subgroup
## 
## 
##  FutilityStage   selectedGroup        mean          sd
## --------------  --------------  ----------  ----------
##              1               1   250.12849    6.448017
##              1               2   200.50748    8.145799
##              1               3   150.83234    8.941759
##              1               4   101.12919    8.191214
##              1               5    50.96708    6.061577
##              2               1   333.25620    6.800872
##              2               2   266.26551    9.695436
##              2               3   200.65301   10.049219
##              2               4   134.25532   10.510943
##              2               5    67.35000    7.933177
##              3               1   415.90075    8.739815
##              3               2   333.21799   10.440185
##              3               3   250.18955   10.990600
##              3               4   166.68026   10.518131
##              3               5    83.35916    8.453743
## 
## Chance of each subpopulation rejected
## 
## 
##  group   count   proportion
## ------  ------  -----------
##      1    1887      0.07548
##      2    2315      0.09260
##      3    5207      0.20828
##      4     787      0.03148
##      5     203      0.00812
##      6    6691      0.26764
## 
## Counts by futility stage and subgroup choice
## 
## 
##  FutilityStage     G1     G2     G3     G4     G5
## --------------  -----  -----  -----  -----  -----
##              1   1253   1137   1843    418    243
##              2    363    403    830    188    120
##              3    937   1890   6067   1520   1097
## 
## CI Statistics:
## Overall coverage and coverage for rejections:
## 
##  overall   rejection
## --------  ----------
##   0.3164           0
## 
## P(theta_test is in the confidence interval)
## 
## 
##   coverage   selectedCount   rejectedCount
## ----------  --------------  --------------
##  0.2608696            2553            1887
##  0.3250729            3430            2315
##  0.4042334            8740            5207
##  0.6298213            2126             787
##  0.8609589            1460             203
##  0.0000000            6691            6691
## NULL

Alternative Scenario S9

scenario <- LLL.SETTINGS$scenarios$S9
trueParameters <- list(prevalence = LLL.SETTINGS$prevalences$table1,
                       mean = scenario$mean,
                       sd = scenario$sd)
result <- designA$explore(numberOfSimulations = 25000,
                          trueParameters = trueParameters, showProgress = FALSE)
analysis <- designA$analyze(result)
print(designA$summary(analysis))
## P(Reject H0_ITT) = 0.478400; P(Reject H0_subgp) = 0.465840; P(Reject H0) = 0.944240
## P(Early stop for efficacy [futility]) = 0.510800 [0.000560]
## Mean [SD] Randomized N = 419.260000 [86.464741]
## 
## Stage at exit (proportion)
## 
## 
##  exitStage   proportion
## ----------  -----------
##          1      0.29604
##          2      0.21532
##          3      0.48864
## 
## Mean [SD] Lost N = 147.749440 [154.151954]
## Mean [SD] Analyzed N = 271.510560 [118.438153]
## 
## Mean loss by futility stage and subgroup
## 
## 
##  FutilityStage   selectedGroup        mean          sd
## --------------  --------------  ----------  ----------
##              1               1   250.57766    6.389644
##              1               2   201.62012    8.075404
##              1               3   151.94715    8.924055
##              1               4   101.82979    6.614806
##              1               5    51.33333    5.844129
##              2               1   333.76087    7.203882
##              2               2   268.37566    8.897092
##              2               3   202.06173    9.562771
##              2               4   136.93617    8.501673
##              2               5    66.00000    6.845228
##              3               1   416.32754    8.271757
##              3               2   333.63458   10.359139
##              3               3   250.71265   10.884472
##              3               4   167.93820   10.939887
##              3               5    83.21127    8.272165
## 
## Chance of each subpopulation rejected
## 
## 
##  group   count   proportion
## ------  ------  -----------
##      1    3075      0.12300
##      2    4418      0.17672
##      3    3408      0.13632
##      4     612      0.02448
##      5     133      0.00532
##      6   11960      0.47840
## 
## Counts by futility stage and subgroup choice
## 
## 
##  FutilityStage     G1     G2     G3    G4    G5
## --------------  -----  -----  -----  ----  ----
##              1    985    845    492    94    27
##              2    322    378    243    47     8
##              3   2015   3522   3066   712   284
## 
## CI Statistics:
## Overall coverage and coverage for rejections:
## 
##  overall   rejection
## --------  ----------
##  0.05576           0
## 
## P(theta_test is in the confidence interval)
## 
## 
##   coverage   selectedCount   rejectedCount
## ----------  --------------  --------------
##  0.0743528            3322            3075
##  0.0689146            4745            4418
##  0.1033938            3801            3408
##  0.2825322             853             612
##  0.5830721             319             133
##  0.0000000           11960           11960
## NULL

Alternative Scenario S10

scenario <- LLL.SETTINGS$scenarios$S10
trueParameters <- list(prevalence = LLL.SETTINGS$prevalences$table1,
                       mean = scenario$mean,
                       sd = scenario$sd)
result <- designA$explore(numberOfSimulations = 25000,
                          trueParameters = trueParameters, showProgress = FALSE)
analysis <- designA$analyze(result)
print(designA$summary(analysis))
## P(Reject H0_ITT) = 0.432040; P(Reject H0_subgp) = 0.002920; P(Reject H0) = 0.434960
## P(Early stop for efficacy [futility]) = 0.350120 [0.129760]
## Mean [SD] Randomized N = 422.784000 [87.207761]
## 
## Stage at exit (proportion)
## 
## 
##  exitStage   proportion
## ----------  -----------
##          1      0.29228
##          2      0.18760
##          3      0.52012
## 
## Mean [SD] Lost N = 76.621080 [104.348815]
## Mean [SD] Analyzed N = 346.162920 [104.312231]
## 
## Mean loss by futility stage and subgroup
## 
## 
##  FutilityStage   selectedGroup        mean          sd
## --------------  --------------  ----------  ----------
##              1               1   249.68459    6.224808
##              1               2   200.08596    7.690011
##              1               3   149.12162    9.407011
##              1               4    98.04444    7.517110
##              1               5    48.97054    6.529214
##              2               1   333.07500    7.336613
##              2               2   264.21875    9.159640
##              2               3   195.56000    7.665290
##              2               4   133.75000    8.837739
##              2               5    65.46154    7.110348
##              3               1   417.21726    8.612445
##              3               2   334.16169   10.703262
##              3               3   247.20000    8.558473
##              3               4   165.81653   10.650585
##              3               5    82.94360    8.387382
## 
## Chance of each subpopulation rejected
## 
## 
##  group   count   proportion
## ------  ------  -----------
##      1      38      0.00152
##      2       7      0.00028
##      4       1      0.00004
##      5      27      0.00108
##      6   10801      0.43204
## 
## Counts by futility stage and subgroup choice
## 
## 
##  FutilityStage     G1    G2   G3    G4     G5
## --------------  -----  ----  ---  ----  -----
##              1    967   349   74   225   1290
##              2    280    96   25    72    663
##              3   1008   402   60   496   8192
## 
## CI Statistics:
## Overall coverage and coverage for rejections:
## 
##  overall   rejection
## --------  ----------
##  0.56504           0
## 
## P(theta_test is in the confidence interval)
## 
## 
##   coverage   selectedCount   rejectedCount
## ----------  --------------  --------------
##  0.9831486            2255              38
##  0.9917355             847               7
##  1.0000000             159               0
##  0.9987390             793               1
##  0.9973386           10145              27
##  0.0000000           10801           10801
## NULL

Table 2 Results

Scenario S0

scenario <- LLL.SETTINGS$scenarios$S0
designParameters <- list(prevalence = LLL.SETTINGS$prevalences$table2,
                       mean = scenario$mean,
                       sd = scenario$sd)
designA <- ASSISTDesign$new(trialParameters = LLL.SETTINGS$trialParameters,
                            designParameters = designParameters)
print(designA)
## Design Parameters:
##  Number of Groups: 6
##  Prevalence:
## 
##  Group1   Group2   Group3   Group4   Group5   Group6
## -------  -------  -------  -------  -------  -------
##     0.2      0.1      0.3      0.1      0.1      0.2
## 
##  Using Discrete Rankin scores? FALSE
## 
##  Normal Rankin Distribution means (null row, alt. row):
## 
## 
##         Group1   Group2   Group3   Group4   Group5   Group6
## -----  -------  -------  -------  -------  -------  -------
## Null         0        0        0        0        0        0
## Alt          0        0        0        0        0        0
## 
##  Normal Rankin Distribution SDs (null row, alt. row):
## 
## 
##         Group1   Group2   Group3   Group4   Group5   Group6
## -----  -------  -------  -------  -------  -------  -------
## Null         1        1        1        1        1        1
## Alt          1        1        1        1        1        1
## 
## Trial Parameters:
## List of 5
##  $ N         : num [1:3] 300 400 500
##  $ type1Error: num 0.05
##  $ eps       : num 0.5
##  $ type2Error: num 0.2
##  $ effectSize: num 0.0642
## 
## Boundaries:
## 
## 
##     btilde          b          c
## ----------  ---------  ---------
##  -1.460993   2.371573   2.467677
result <- designA$explore(numberOfSimulations = 25000, showProgress = FALSE)
analysis <- designA$analyze(result)
print(designA$summary(analysis))
## P(Reject H0_ITT) = 0.015480; P(Reject H0_subgp) = 0.021920; P(Reject H0) = 0.037400
## P(Early stop for efficacy [futility]) = 0.022520 [0.562960]
## Mean [SD] Randomized N = 419.416000 [77.279090]
## 
## Stage at exit (proportion)
## 
## 
##  exitStage   proportion
## ----------  -----------
##          1      0.22036
##          2      0.36512
##          3      0.41452
## 
## Mean [SD] Lost N = 177.251640 [87.868516]
## Mean [SD] Analyzed N = 242.164360 [95.059262]
## 
## Mean loss by futility stage and subgroup
## 
## 
##  FutilityStage   selectedGroup        mean          sd
## --------------  --------------  ----------  ----------
##              1               1   239.93296    7.009866
##              1               2   210.06027    7.711790
##              1               3   120.40245    8.572703
##              1               4    89.98533    7.962544
##              1               5    60.20870    7.075884
##              2               1   320.01062    7.861324
##              2               2   279.97753    9.439236
##              2               3   159.90580    9.654945
##              2               4   120.59717    8.877900
##              2               5    80.12467    8.296030
##              3               1   400.11838    8.659173
##              3               2   348.57247   10.783606
##              3               3   200.05105   10.664735
##              3               4   150.01267   10.295253
##              3               5    99.78576    8.844766
## 
## Chance of each subpopulation rejected
## 
## 
##  group   count   proportion
## ------  ------  -----------
##      1     196      0.00784
##      2     175      0.00700
##      3      90      0.00360
##      4      48      0.00192
##      5      39      0.00156
##      6     387      0.01548
## 
## Counts by futility stage and subgroup choice
## 
## 
##  FutilityStage     G1     G2     G3     G4     G5
## --------------  -----  -----  -----  -----  -----
##              1   5817   4015   2614   1977   2597
##              2    753    623    552    494    762
##              3    642    683    764    789   1531
## 
## CI Statistics:
## Overall coverage and coverage for rejections:
## 
##  overall   rejection
## --------  ----------
##   0.9626           0
## 
## P(theta_test is in the confidence interval)
## 
## 
##   coverage   selectedCount   rejectedCount
## ----------  --------------  --------------
##  0.9728231            7212             196
##  0.9671114            5321             175
##  0.9770992            3930              90
##  0.9852761            3260              48
##  0.9920245            4890              39
##  0.0000000             387             387
## NULL

Alternative Scenario S1

scenario <- LLL.SETTINGS$scenarios$S1
trueParameters <- list(prevalence = LLL.SETTINGS$prevalences$table2,
                       mean = scenario$mean,
                       sd = scenario$sd)
result <- designA$explore(numberOfSimulations = 25000, trueParameters = trueParameters,
                          showProgress = FALSE)
analysis <- designA$analyze(result)
print(designA$summary(analysis))
## P(Reject H0_ITT) = 0.822120; P(Reject H0_subgp) = 0.043000; P(Reject H0) = 0.865120
## P(Early stop for efficacy [futility]) = 0.739360 [0.005000]
## Mean [SD] Randomized N = 369.356000 [85.078992]
## 
## Stage at exit (proportion)
## 
## 
##  exitStage   proportion
## ----------  -----------
##          1      0.56208
##          2      0.18228
##          3      0.25564
## 
## Mean [SD] Lost N = 31.487880 [79.797985]
## Mean [SD] Analyzed N = 337.868120 [76.300228]
## 
## Mean loss by futility stage and subgroup
## 
## 
##  FutilityStage   selectedGroup        mean          sd
## --------------  --------------  ----------  ----------
##              1               1   240.48361    6.127075
##              1               2   210.93750    8.054158
##              1               3   120.49412    8.687188
##              1               4    91.45588    9.049505
##              1               5    60.42202    7.395311
##              2               1   320.26923    8.720356
##              2               2   279.89286    9.619961
##              2               3   164.14634   10.563051
##              2               4   122.39130    6.780581
##              2               5    80.21622    7.409210
##              3               1   399.80473    9.178596
##              3               2   350.01205   10.401475
##              3               3   199.81931   10.995747
##              3               4   150.47657   10.841927
##              3               5    99.75089    9.117869
## 
## Chance of each subpopulation rejected
## 
## 
##  group   count   proportion
## ------  ------  -----------
##      1     219      0.00876
##      2     206      0.00824
##      3     237      0.00948
##      4     181      0.00724
##      5     232      0.00928
##      6   20553      0.82212
## 
## Counts by futility stage and subgroup choice
## 
## 
##  FutilityStage    G1    G2    G3    G4     G5
## --------------  ----  ----  ----  ----  -----
##              1   122    80    85    68    109
##              2    26    28    41    23     37
##              3   338   415   642   747   1686
## 
## CI Statistics:
## Overall coverage and coverage for rejections:
## 
##  overall   rejection
## --------  ----------
##  0.13488           0
## 
## P(theta_test is in the confidence interval)
## 
## 
##   coverage   selectedCount   rejectedCount
## ----------  --------------  --------------
##  0.5493827             486             219
##  0.6061185             523             206
##  0.6914062             768             237
##  0.7840095             838             181
##  0.8733624            1832             232
##  0.0000000           20553           20553
## NULL

Alternative Scenario S2

scenario <- LLL.SETTINGS$scenarios$S2
trueParameters <- list(prevalence = LLL.SETTINGS$prevalences$table2,
                       mean = scenario$mean,
                       sd = scenario$sd)
result <- designA$explore(numberOfSimulations = 25000, trueParameters = trueParameters,
                          showProgress = FALSE)
analysis <- designA$analyze(result)
print(designA$summary(analysis))
## P(Reject H0_ITT) = 0.490520; P(Reject H0_subgp) = 0.277040; P(Reject H0) = 0.767560
## P(Early stop for efficacy [futility]) = 0.457520 [0.008720]
## Mean [SD] Randomized N = 425.164000 [86.751796]
## 
## Stage at exit (proportion)
## 
## 
##  exitStage   proportion
## ----------  -----------
##          1      0.28212
##          2      0.18412
##          3      0.53376
## 
## Mean [SD] Lost N = 90.377160 [106.438122]
## Mean [SD] Analyzed N = 334.786840 [86.687693]
## 
## Mean loss by futility stage and subgroup
## 
## 
##  FutilityStage   selectedGroup        mean          sd
## --------------  --------------  ----------  ----------
##              1               1   239.98475    7.150731
##              1               2   211.02058    8.194516
##              1               3   120.37222    8.659738
##              1               4    91.19156    7.720672
##              1               5    60.93333    7.375446
##              2               1   321.01324    7.373816
##              2               2   279.92517    8.537659
##              2               3   161.24201   10.183224
##              2               4   121.04934    8.933982
##              2               5    80.50000    8.145293
##              3               1   399.09626    9.044019
##              3               2   349.41278   10.387528
##              3               3   200.11816   11.298750
##              3               4   150.46675   10.347723
##              3               5   100.57906    9.135956
## 
## Chance of each subpopulation rejected
## 
## 
##  group   count   proportion
## ------  ------  -----------
##      1     796      0.03184
##      2     966      0.03864
##      3    1722      0.06888
##      4    2655      0.10620
##      5     787      0.03148
##      6   12263      0.49052
## 
## Counts by futility stage and subgroup choice
## 
## 
##  FutilityStage    G1    G2     G3     G4     G5
## --------------  ----  ----  -----  -----  -----
##              1   459   486    540    616    240
##              2   151   147    219    304    108
##              3   561   814   2175   4045   1872
## 
## CI Statistics:
## Overall coverage and coverage for rejections:
## 
##  overall   rejection
## --------  ----------
##  0.23244           0
## 
## P(theta_test is in the confidence interval)
## 
## 
##   coverage   selectedCount   rejectedCount
## ----------  --------------  --------------
##  0.3202391            1171             796
##  0.3324119            1447             966
##  0.4130879            2934            1722
##  0.4652568            4965            2655
##  0.6454955            2220             787
##  0.0000000           12263           12263
## NULL

Alternative Scenario S3

scenario <- LLL.SETTINGS$scenarios$S3
trueParameters <- list(prevalence = LLL.SETTINGS$prevalences$table2,
                       mean = scenario$mean,
                       sd = scenario$sd)
result <- designA$explore(numberOfSimulations = 25000, trueParameters = trueParameters,
                          showProgress = FALSE)
analysis <- designA$analyze(result)
print(designA$summary(analysis))
## P(Reject H0_ITT) = 0.104040; P(Reject H0_subgp) = 0.489480; P(Reject H0) = 0.593520
## P(Early stop for efficacy [futility]) = 0.321320 [0.045920]
## Mean [SD] Randomized N = 453.780000 [66.219989]
## 
## Stage at exit (proportion)
## 
## 
##  exitStage   proportion
## ----------  -----------
##          1      0.09496
##          2      0.27228
##          3      0.63276
## 
## Mean [SD] Lost N = 220.699160 [115.906973]
## Mean [SD] Analyzed N = 233.080840 [103.625942]
## 
## Mean loss by futility stage and subgroup
## 
## 
##  FutilityStage   selectedGroup        mean          sd
## --------------  --------------  ----------  ----------
##              1               1   240.35287    6.785554
##              1               2   210.72978    7.798721
##              1               3   120.96732    8.488045
##              1               4    90.57411    7.890977
##              1               5    60.96203    6.886472
##              2               1   319.82549    8.203131
##              2               2   280.60638    9.369014
##              2               3   160.55215    9.237484
##              2               4   120.77720    9.325542
##              2               5    80.30769    8.552307
##              3               1   399.74068    9.112486
##              3               2   349.46174   10.434419
##              3               3   200.41482   11.320456
##              3               4   150.65514   11.042862
##              3               5   100.31903    9.166864
## 
## Chance of each subpopulation rejected
## 
## 
##  group   count   proportion
## ------  ------  -----------
##      1    4140      0.16560
##      2    7051      0.28204
##      3     604      0.02416
##      4     272      0.01088
##      5     170      0.00680
##      6    2601      0.10404
## 
## Counts by futility stage and subgroup choice
## 
## 
##  FutilityStage     G1     G2     G3    G4     G5
## --------------  -----  -----  -----  ----  -----
##              1   3174   4500    918   533    474
##              2    871   1504    326   193    195
##              3   1797   4561   1309   925   1119
## 
## CI Statistics:
## Overall coverage and coverage for rejections:
## 
##  overall   rejection
## --------  ----------
##  0.40648           0
## 
## P(theta_test is in the confidence interval)
## 
## 
##   coverage   selectedCount   rejectedCount
## ----------  --------------  --------------
##  0.2913386            5842            4140
##  0.3326077           10565            7051
##  0.7634156            2553             604
##  0.8352514            1651             272
##  0.9049217            1788             170
##  0.0000000            2601            2601
## NULL

Alternative Scenario S4

scenario <- LLL.SETTINGS$scenarios$S4
trueParameters <- list(prevalence = LLL.SETTINGS$prevalences$table2,
                       mean = scenario$mean,
                       sd = scenario$sd)
result <- designA$explore(numberOfSimulations = 25000, trueParameters = trueParameters,
                          showProgress = FALSE)
analysis <- designA$analyze(result)
print(designA$summary(analysis))
## P(Reject H0_ITT) = 0.171160; P(Reject H0_subgp) = 0.113800; P(Reject H0) = 0.284960
## P(Early stop for efficacy [futility]) = 0.150960 [0.168120]
## Mean [SD] Randomized N = 451.836000 [75.815955]
## 
## Stage at exit (proportion)
## 
## 
##  exitStage   proportion
## ----------  -----------
##          1      0.16256
##          2      0.15652
##          3      0.68092
## 
## Mean [SD] Lost N = 111.862200 [74.557333]
## Mean [SD] Analyzed N = 339.973800 [79.632068]
## 
## Mean loss by futility stage and subgroup
## 
## 
##  FutilityStage   selectedGroup        mean          sd
## --------------  --------------  ----------  ----------
##              1               1   240.09662    7.024022
##              1               2   208.77568    8.647004
##              1               3   120.13578    8.604491
##              1               4    90.28927    7.935467
##              1               5    60.37818    6.883419
##              2               1   318.76437    8.503430
##              2               2   280.01613    8.259107
##              2               3   159.17625   10.185024
##              2               4   120.51129    9.285243
##              2               5    80.20257    8.401125
##              3               1   400.10163    8.756679
##              3               2   351.23958   10.523902
##              3               3   199.70407   10.801328
##              3               4   149.85161   10.359235
##              3               5   100.04298    9.125262
## 
## Chance of each subpopulation rejected
## 
## 
##  group   count   proportion
## ------  ------  -----------
##      1      52      0.00208
##      2      25      0.00100
##      3     611      0.02444
##      4    1570      0.06280
##      5     587      0.02348
##      6    4279      0.17116
## 
## Counts by futility stage and subgroup choice
## 
## 
##  FutilityStage     G1    G2     G3     G4     G5
## --------------  -----  ----  -----  -----  -----
##              1   1035   370   1473   2693   1650
##              2    174    62    522   1107    701
##              3    246    96   1622   5061   3909
## 
## CI Statistics:
## Overall coverage and coverage for rejections:
## 
##  overall   rejection
## --------  ----------
##  0.71504           0
## 
## P(theta_test is in the confidence interval)
## 
## 
##   coverage   selectedCount   rejectedCount
## ----------  --------------  --------------
##  0.9642612            1455              52
##  0.9526515             528              25
##  0.8310755            3617             611
##  0.8228191            8861            1570
##  0.9062300            6260             587
##  0.0000000            4279            4279
## NULL

Alternative Scenario S5

scenario <- LLL.SETTINGS$scenarios$S5
trueParameters <- list(prevalence = LLL.SETTINGS$prevalences$table2,
                       mean = scenario$mean,
                       sd = scenario$sd)
result <- designA$explore(numberOfSimulations = 25000, trueParameters = trueParameters,
                          showProgress = FALSE)
analysis <- designA$analyze(result)
print(designA$summary(analysis))
## P(Reject H0_ITT) = 0.159920; P(Reject H0_subgp) = 0.711600; P(Reject H0) = 0.871520
## P(Early stop for efficacy [futility]) = 0.457600 [0.006000]
## Mean [SD] Randomized N = 438.208000 [73.807633]
## 
## Stage at exit (proportion)
## 
## 
##  exitStage   proportion
## ----------  -----------
##          1      0.15432
##          2      0.30928
##          3      0.53640
## 
## Mean [SD] Lost N = 259.455560 [138.729982]
## Mean [SD] Analyzed N = 178.752440 [111.464756]
## 
## Mean loss by futility stage and subgroup
## 
## 
##  FutilityStage   selectedGroup        mean          sd
## --------------  --------------  ----------  ----------
##              1               1   241.04717    6.792332
##              1               2   211.31729    7.733041
##              1               3   121.17814    8.772222
##              1               4    92.71304    7.192546
##              1               5    61.20408    7.477746
##              2               1   320.45116    7.890766
##              2               2   280.55275    9.052819
##              2               3   162.91111   10.006340
##              2               4   121.36364    8.447074
##              2               5    81.36364    8.957602
##              3               1   399.48654    8.808806
##              3               2   350.13428   10.134943
##              3               3   199.62069   11.530224
##              3               4   149.51361   10.121016
##              3               5   100.92568    9.153005
## 
## Chance of each subpopulation rejected
## 
## 
##  group   count   proportion
## ------  ------  -----------
##      1   14074      0.56296
##      2    3069      0.12276
##      3     406      0.01624
##      4     157      0.00628
##      5      84      0.00336
##      6    3998      0.15992
## 
## Counts by futility stage and subgroup choice
## 
## 
##  FutilityStage     G1     G2    G3    G4    G5
## --------------  -----  -----  ----  ----  ----
##              1   5682   1330   247   115    98
##              2   2068    436    90    44    33
##              3   7915   1832   522   294   296
## 
## CI Statistics:
## Overall coverage and coverage for rejections:
## 
##  overall   rejection
## --------  ----------
##  0.12848           0
## 
## P(theta_test is in the confidence interval)
## 
## 
##   coverage   selectedCount   rejectedCount
## ----------  --------------  --------------
##  0.1015640           15665           14074
##  0.1470261            3598            3069
##  0.5273574             859             406
##  0.6534216             453             157
##  0.8032787             427              84
##  0.0000000            3998            3998
## NULL

Alternative Scenario S6

scenario <- LLL.SETTINGS$scenarios$S6
trueParameters <- list(prevalence = LLL.SETTINGS$prevalences$table2,
                       mean = scenario$mean,
                       sd = scenario$sd)
result <- designA$explore(numberOfSimulations = 25000, trueParameters = trueParameters,
                          showProgress = FALSE)
analysis <- designA$analyze(result)
print(designA$summary(analysis))
## P(Reject H0_ITT) = 0.119600; P(Reject H0_subgp) = 0.666840; P(Reject H0) = 0.786440
## P(Early stop for efficacy [futility]) = 0.441520 [0.014600]
## Mean [SD] Randomized N = 442.452000 [69.501230]
## 
## Stage at exit (proportion)
## 
## 
##  exitStage   proportion
## ----------  -----------
##          1      0.11936
##          2      0.33676
##          3      0.54388
## 
## Mean [SD] Lost N = 254.886520 [128.090445]
## Mean [SD] Analyzed N = 187.565480 [107.934224]
## 
## Mean loss by futility stage and subgroup
## 
## 
##  FutilityStage   selectedGroup        mean          sd
## --------------  --------------  ----------  ----------
##              1               1   240.63833    6.904159
##              1               2   210.91460    7.813404
##              1               3   120.92793    8.394841
##              1               4    90.95902    7.934296
##              1               5    60.85027    7.061103
##              2               1   320.14427    7.821407
##              2               2   280.24689    9.118114
##              2               3   160.94118   10.042826
##              2               4   120.08235    8.696211
##              2               5    79.36486    8.141849
##              3               1   399.35508    8.987618
##              3               2   350.29394   10.106608
##              3               3   200.16940   10.718318
##              3               4   149.91060    9.983231
##              3               5   100.15222    8.916708
## 
## Chance of each subpopulation rejected
## 
## 
##  group   count   proportion
## ------  ------  -----------
##      1   12594      0.50376
##      2    3293      0.13172
##      3     472      0.01888
##      4     194      0.00776
##      5     118      0.00472
##      6    2990      0.11960
## 
## Counts by futility stage and subgroup choice
## 
## 
##  FutilityStage     G1     G2    G3    G4    G5
## --------------  -----  -----  ----  ----  ----
##              1   6340   1733   444   244   187
##              2   1927    563   153    85    74
##              3   6531   1997   732   481   519
## 
## CI Statistics:
## Overall coverage and coverage for rejections:
## 
##  overall   rejection
## --------  ----------
##  0.21356           0
## 
## P(theta_test is in the confidence interval)
## 
## 
##   coverage   selectedCount   rejectedCount
## ----------  --------------  --------------
##  0.1489390           14798           12594
##  0.2329373            4293            3293
##  0.6448457            1329             472
##  0.7604938             810             194
##  0.8487179             780             118
##  0.0000000            2990            2990
## NULL

Alternative Scenario S7

scenario <- LLL.SETTINGS$scenarios$S7
trueParameters <- list(prevalence = LLL.SETTINGS$prevalences$table2,
                       mean = scenario$mean,
                       sd = scenario$sd)
result <- designA$explore(numberOfSimulations = 25000, trueParameters = trueParameters,
                          showProgress = FALSE)
analysis <- designA$analyze(result)
print(designA$summary(analysis))
## P(Reject H0_ITT) = 0.570240; P(Reject H0_subgp) = 0.341000; P(Reject H0) = 0.911240
## P(Early stop for efficacy [futility]) = 0.545080 [0.001640]
## Mean [SD] Randomized N = 411.972000 [87.894142]
## 
## Stage at exit (proportion)
## 
## 
##  exitStage   proportion
## ----------  -----------
##          1      0.33356
##          2      0.21316
##          3      0.45328
## 
## Mean [SD] Lost N = 101.566960 [131.828404]
## Mean [SD] Analyzed N = 310.405040 [101.613328]
## 
## Mean loss by futility stage and subgroup
## 
## 
##  FutilityStage   selectedGroup        mean          sd
## --------------  --------------  ----------  ----------
##              1               1   240.65370    6.281002
##              1               2   210.74820    7.746752
##              1               3   121.99058    8.818821
##              1               4    91.55714    7.828441
##              1               5    62.46296    6.784313
##              2               1   319.28125    7.252995
##              2               2   280.52093    9.574513
##              2               3   161.60000    9.361770
##              2               4   119.47059    9.020760
##              2               5    78.44444    7.064595
##              3               1   399.42247    8.481709
##              3               2   350.72670   10.199406
##              3               3   200.76054   10.775413
##              3               4   151.30739   10.122888
##              3               5   100.90267    8.689463
## 
## Chance of each subpopulation rejected
## 
## 
##  group   count   proportion
## ------  ------  -----------
##      1    1487      0.05948
##      2    2296      0.09184
##      3    3652      0.14608
##      4     816      0.03264
##      5     274      0.01096
##      6   14256      0.57024
## 
## Counts by futility stage and subgroup choice
## 
## 
##  FutilityStage     G1     G2     G3     G4    G5
## --------------  -----  -----  -----  -----  ----
##              1    514    556    531    140    54
##              2    160    215    235     51    18
##              3   1077   1921   3746   1002   524
## 
## CI Statistics:
## Overall coverage and coverage for rejections:
## 
##  overall   rejection
## --------  ----------
##  0.08876           0
## 
## P(theta_test is in the confidence interval)
## 
## 
##   coverage   selectedCount   rejectedCount
## ----------  --------------  --------------
##  0.1507710            1751            1487
##  0.1471025            2692            2296
##  0.1906028            4512            3652
##  0.3160101            1193             816
##  0.5402685             596             274
##  0.0000000           14256           14256
## NULL

Alternative Scenario S8

scenario <- LLL.SETTINGS$scenarios$S8
trueParameters <- list(prevalence = LLL.SETTINGS$prevalences$table2,
                       mean = scenario$mean,
                       sd = scenario$sd)
result <- designA$explore(numberOfSimulations = 25000, trueParameters = trueParameters,
                          showProgress = FALSE)
analysis <- designA$analyze(result)
print(designA$summary(analysis))
## P(Reject H0_ITT) = 0.390640; P(Reject H0_subgp) = 0.333680; P(Reject H0) = 0.724320
## P(Early stop for efficacy [futility]) = 0.391320 [0.014280]
## Mean [SD] Randomized N = 437.304000 [82.257414]
## 
## Stage at exit (proportion)
## 
## 
##  exitStage   proportion
## ----------  -----------
##          1      0.22136
##          2      0.18424
##          3      0.59440
## 
## Mean [SD] Lost N = 122.008800 [116.341514]
## Mean [SD] Analyzed N = 315.295200 [91.545802]
## 
## Mean loss by futility stage and subgroup
## 
## 
##  FutilityStage   selectedGroup        mean          sd
## --------------  --------------  ----------  ----------
##              1               1   240.15817    6.656479
##              1               2   210.54138    7.901309
##              1               3   121.13788    8.266286
##              1               4    91.44940    7.990879
##              1               5    61.10924    7.266814
##              2               1   319.26389    7.791456
##              2               2   280.31780    9.205263
##              2               3   161.29106    9.749710
##              2               4   120.92701    8.862209
##              2               5    81.24762    8.161567
##              3               1   400.02474    9.037408
##              3               2   349.23420   10.544889
##              3               3   200.29170   10.889219
##              3               4   150.25675    9.904950
##              3               5   100.12308    9.052280
## 
## Chance of each subpopulation rejected
## 
## 
##  group   count   proportion
## ------  ------  -----------
##      1    1244      0.04976
##      2    1545      0.06180
##      3    4385      0.17540
##      4     839      0.03356
##      5     329      0.01316
##      6    9766      0.39064
## 
## Counts by futility stage and subgroup choice
## 
## 
##  FutilityStage    G1     G2     G3     G4     G5
## --------------  ----  -----  -----  -----  -----
##              1   765    737   1262    336    238
##              2   216    236    615    137    105
##              3   768   1187   5869   1593   1170
## 
## CI Statistics:
## Overall coverage and coverage for rejections:
## 
##  overall   rejection
## --------  ----------
##  0.27568           0
## 
## P(theta_test is in the confidence interval)
## 
## 
##   coverage   selectedCount   rejectedCount
## ----------  --------------  --------------
##  0.2887364            1749            1244
##  0.2847222            2160            1545
##  0.4339014            7746            4385
##  0.5939013            2066             839
##  0.7825512            1513             329
##  0.0000000            9766            9766
## NULL

Alternative Scenario S9

scenario <- LLL.SETTINGS$scenarios$S9
trueParameters <- list(prevalence = LLL.SETTINGS$prevalences$table2,
                       mean = scenario$mean,
                       sd = scenario$sd)
result <- designA$explore(numberOfSimulations = 25000, trueParameters = trueParameters,
                          showProgress = FALSE)
analysis <- designA$analyze(result)
print(designA$summary(analysis))
## P(Reject H0_ITT) = 0.590720; P(Reject H0_subgp) = 0.369480; P(Reject H0) = 0.960200
## P(Early stop for efficacy [futility]) = 0.569680 [0.000280]
## Mean [SD] Randomized N = 406.648000 [88.837603]
## 
## Stage at exit (proportion)
## 
## 
##  exitStage   proportion
## ----------  -----------
##          1      0.36356
##          2      0.20640
##          3      0.43004
## 
## Mean [SD] Lost N = 114.614480 [150.787216]
## Mean [SD] Analyzed N = 292.033520 [117.101314]
## 
## Mean loss by futility stage and subgroup
## 
## 
##  FutilityStage   selectedGroup        mean          sd
## --------------  --------------  ----------  ----------
##              1               1   241.35735    6.804991
##              1               2   212.41235    7.745659
##              1               3   123.02963    8.327355
##              1               4    92.73438    7.346832
##              1               5    63.33333    5.131601
##              2               1   320.76062    8.401079
##              2               2   281.62308    8.766614
##              2               3   162.69672    9.285988
##              2               4   118.21053    8.593143
##              2               5    76.50000    5.567764
##              3               1   400.14625    8.626274
##              3               2   350.53538   10.280757
##              3               3   201.41770   10.529119
##              3               4   151.14667   10.151037
##              3               5   100.86626    9.140855
## 
## Chance of each subpopulation rejected
## 
## 
##  group   count   proportion
## ------  ------  -----------
##      1    2772      0.11088
##      2    3429      0.13716
##      3    2271      0.09084
##      4     543      0.02172
##      5     222      0.00888
##      6   14768      0.59072
## 
## Counts by futility stage and subgroup choice
## 
## 
##  FutilityStage     G1     G2     G3    G4    G5
## --------------  -----  -----  -----  ----  ----
##              1    694    599    270    64    21
##              2    259    260    122    19     4
##              3   2024   2798   2169   600   329
## 
## CI Statistics:
## Overall coverage and coverage for rejections:
## 
##  overall   rejection
## --------  ----------
##   0.0398           0
## 
## P(theta_test is in the confidence interval)
## 
## 
##   coverage   selectedCount   rejectedCount
## ----------  --------------  --------------
##  0.0688613            2977            2772
##  0.0623462            3657            3429
##  0.1132370            2561            2271
##  0.2049780             683             543
##  0.3728814             354             222
##  0.0000000           14768           14768
## NULL

Alternative Scenario S10

scenario <- LLL.SETTINGS$scenarios$S10
trueParameters <- list(prevalence = LLL.SETTINGS$prevalences$table2,
                       mean = scenario$mean,
                       sd = scenario$sd)
result <- designA$explore(numberOfSimulations = 25000, trueParameters = trueParameters,
                          showProgress = FALSE)
analysis <- designA$analyze(result)
print(designA$summary(analysis))
## P(Reject H0_ITT) = 0.326920; P(Reject H0_subgp) = 0.003120; P(Reject H0) = 0.330040
## P(Early stop for efficacy [futility]) = 0.262840 [0.181480]
## Mean [SD] Randomized N = 430.104000 [84.836531]
## 
## Stage at exit (proportion)
## 
## 
##  exitStage   proportion
## ----------  -----------
##          1      0.25464
##          2      0.18968
##          3      0.55568
## 
## Mean [SD] Lost N = 115.456520 [121.973133]
## Mean [SD] Analyzed N = 314.647480 [114.139067]
## 
## Mean loss by futility stage and subgroup
## 
## 
##  FutilityStage   selectedGroup        mean          sd
## --------------  --------------  ----------  ----------
##              1               1   239.68386    6.985419
##              1               2   209.93041    8.039438
##              1               3   118.55769    8.117079
##              1               4    88.30640    7.419641
##              1               5    58.94565    6.911589
##              2               1   320.09717    8.333726
##              2               2   280.28839    9.672610
##              2               3   160.20000    8.963563
##              2               4   120.60674   10.572926
##              2               5    78.82129    7.938058
##              3               1   400.03851    8.964494
##              3               2   349.99443   10.406245
##              3               3   198.23881   10.888649
##              3               4   149.15813    9.535865
##              3               5    99.84582    8.748413
## 
## Chance of each subpopulation rejected
## 
## 
##  group   count   proportion
## ------  ------  -----------
##      1      51      0.00204
##      2      19      0.00076
##      3       2      0.00008
##      4       1      0.00004
##      5       5      0.00020
##      6    8173      0.32692
## 
## Counts by futility stage and subgroup choice
## 
## 
##  FutilityStage     G1     G2    G3    G4     G5
## --------------  -----  -----  ----  ----  -----
##              1   1512    776   156   297   1380
##              2    494    267    45    89    761
##              3   1662   1077   201   664   7446
## 
## CI Statistics:
## Overall coverage and coverage for rejections:
## 
##  overall   rejection
## --------  ----------
##  0.66996           0
## 
## P(theta_test is in the confidence interval)
## 
## 
##   coverage   selectedCount   rejectedCount
## ----------  --------------  --------------
##  0.9860960            3668              51
##  0.9910377            2120              19
##  0.9950249             402               2
##  0.9990476            1050               1
##  0.9994785            9587               5
##  0.0000000            8173            8173
## NULL

References