# MAMS-trial-simulation-tutorial

#### Ziyan Wang

    library(BayesianPlatformDesignTimeTrend)

## Four arm trial simulation

The ‘BayesianPlatformDesignTimeTrend’ package simulation process requires stopping boundary cutoff screening first (details refers to demo and MAMS-CutoffScreening-tutorial). After the cutoff screening process, we need to record the cutoff value of both efficacy and futility boundary for use in the trial simulation process. The data of each trail replicates will be created sequentailly during the simulation. In this tutorial, the example is a four-arm MAMS trial with one control and three treatment arms. The control arm will not be stopped during the trial. The time trend pattern are set to be ‘linear’. The way of time trend impacting the beginning response probability is set to be ‘mult’ (multiplicative). The time trend strength is set to be zero first and then set to be 0.5 to study the impact of time trend on different evaluation metrics. The model used in this example are fixed effect model (the model with only treatment effect and the model with both treatment effect and discrete stage effect). The evaluation metrics are error rate, mean treatment effect bias, rooted MSE, mean number of patients allocated to each arm and mean total number of patients in the trial.

Firstly, We investigate the family wise error rate (FWER) for different time trend strength and how model with stage effect help control the family wise error rate. The cutoff value was screened for null scenario using model only main effect in order to control the FWER under 0.1. The false positive rate is 0.037 equally for each treatment - control comparison.

    ntrials = 1000 # Number of trial replicates
    ns = seq(120,600,120) # Sequence of total number of accrued patients at each interim analysis
    null.reponse.prob = 0.4
    alt.response.prob = 0.6

    # We investigate the type I error rate for different time trend strength
    null.scenario = matrix(
      c(
        null.reponse.prob,
        null.reponse.prob,
        null.reponse.prob,
        null.reponse.prob
      ),
      nrow = 1,
      ncol = 4,
      byrow = T
    )
    # alt.scenario = matrix(c(null.reponse.prob,null.reponse.prob,null.reponse.prob,null.reponse.prob,
    #                     null.reponse.prob,alt.response.prob,null.reponse.prob,null.reponse.prob,
    #                     null.reponse.prob,alt.response.prob,alt.response.prob,null.reponse.prob,
    #                     null.reponse.prob,alt.response.prob,alt.response.prob,alt.response.prob), nrow=3, ncol = 4,byrow=T)
    model = "tlr" #logistic model
    max.ar = 0.75  #limit the allocation ratio for the control group (1-max.ar < r_control < max.ar)
    #------------Select the data generation randomisation methods-------
    rand.type = "Urn" # Urn design
    max.deviation = 3 # The recommended value for the tuning parameter in the Urn design

    # Require multiple cores for parallel running
    cl = 2

    # Set the model we want to use and the time trend effect for each model used.
    # Here the main model will be used twice for two different strength of time trend c(0,0,0,0) and c(1,1,1,1) to investigate how time trend affect the evaluation metrics in BAR setting.
    # Then the main + stage_continuous model which is the treatment effect + stage effect model will be applied for strength equal c(1,1,1,1) to investigate how the main + stage effect model improve the evaluation metrics.
    reg.inf = c("main", "main", "main + stage_continuous")
    trend.effect = matrix(
      c(0, 0, 0, 0, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1),
      ncol = 4,
      nrow = 3,
      byrow = T
    )

    #
    cutoffearly = matrix(rep(0.994, dim(null.scenario)[1]), ncol = 1)

    K = dim(null.scenario)[2]
    print(
      paste0(
        "Start trial simulation. This is a ",
        K,
        "-arm trial simulation. There are one null scenario and ",
        K - 1 ,
        " alternative scenarios. There are ",
        K ,
        " rounds."
      )
    )
    #> [1] "Start trial simulation. This is a 4-arm trial simulation. There are one null scenario and 3 alternative scenarios. There are 4 rounds."
    cutoffindex = 1

    result = {

    }
    OPC_null = {

    }
    for (i in 1:dim(null.scenario)[1]) {
      trendindex = 1
      for (j in 1:length(reg.inf)){
        restlr = Trial.simulation(
          ntrials = ntrials,
          # Number of trial replicates
          trial.fun = simulatetrial,
          # Call the main function
          input.info = list(
            response.probs = null.scenario[cutoffindex, ],
            #The scenario vector in this round
            ns = ns,
            # Sequence of total number of accrued patients at each interim analysis
            max.ar =  max.ar,
            #limit the allocation ratio for the control group (1-max.ar < r_control < max.ar)
            rand.type = rand.type,
            # Which randomisation methods in data generation.
            max.deviation = max.deviation,
            # The recommended value for the tuning parameter in the Urn design
            model.inf = list(
              model = model,
              #Use which model?
              ibb.inf = list(
                #independent beta-binomial model which can be used only for no time trend simulation
                pi.star = 0.5,
                # beta prior mean
                pess = 2,
                # beta prior effective sample size
                betabinomialmodel = ibetabinomial.post # beta-binomial model for posterior estimation
              ),
              tlr.inf = list(
                beta0_prior_mu = 0,
                # Stan logistic model t prior location
                beta1_prior_mu = 0,
                # Stan logistic model t prior location
                beta0_prior_sigma = 2.5,
                # Stan logistic model t prior sigma
                beta1_prior_sigma = 2.5,
                # Stan logistic model t prior sigma
                beta0_df = 7,
                # Stan logistic model t prior degree of freedom
                beta1_df = 7,
                # Stan logistic model t prior degree of freedom
                reg.inf =  reg.inf[trendindex],
                # The model we want to use
                variable.inf = "Fixeffect" # Use fix effect logistic model
              )
            ),
            Stopbound.inf = Stopboundinf(
              Stop.type = "Early-Pocock",
              # Use Pocock like early stopping boundary
              Boundary.type = "Symmetric",
              # Use Symmetric boundary where cutoff value for efficacy boundary and futility boundary sum up to 1
              cutoff = c(cutoffearly[cutoffindex, 1], 1 - cutoffearly[cutoffindex, 1]) # The cutoff value for stopping boundary
            ),
            Random.inf = list(
              Fixratio = FALSE,
              # Do not use fix ratio allocation
              Fixratiocontrol = NA,
              # Do not use fix ratio allocation
              BARmethod = "Thall",
              # Use Thall's Bayesian adaptive randomisation approach
              Thall.tuning.inf = list(tuningparameter = "Fixed",  fixvalue = 1) # Specified the tunning parameter value for fixed tuning parameter
            ),
            trend.inf = list(
              trend.type = "step",
              # Linear time trend pattern
              trend.effect = trend.effect[trendindex, ],
              # Stength of time trend effect
              trend_add_or_multip = "mult" # Multiplicative time trend effect on response probability
            )
          ),
          cl = 2 # 2 cores required
        )

      trendindex = trendindex + 1
      # The result list can be used for plotting and the OPC table is the summary evaluaton metrics for each scenario
      result = c(result, restlr$result)
      OPC_null = rbind(OPC_null, restlr$OPC)
      }
      cutoffindex = cutoffindex + 1
    }

    print("Finished null scenario study")
    #> [1] "Finished null scenario study"
    save_data = FALSE
    if (isTRUE(save_data)) {
      save(result, file = restlr$Nameofsaveddata$nameData)
      save(OPC_null, file = restlr$Nameofsaveddata$nameTable)
    }

Present the evaluation metrics for null scenario. The FWER is 0.1 when there is no time trend. FWER inflated to 0.1296 when there is a step time trend pattern and modeled by main fixed effect model. The main effect plus stage effect model controls the FWER again under 0.1.

    # Characteristic table
    print(OPC_null)
    #>                                                           Type.I.Error.or.Power
    #> 04040404 Timetrend0000 main stage5                                       0.0920
    #> 04040404 Timetrend01010101 main stage5                                   0.1269
    #> 04040404 Timetrend01010101 main + stage_continuous stage5                0.0942
    #>                                                               Bias.1     Bias.2
    #> 04040404 Timetrend0000 main stage5                        -0.0017950  0.0017528
    #> 04040404 Timetrend01010101 main stage5                     0.0027425  0.0001126
    #> 04040404 Timetrend01010101 main + stage_continuous stage5 -0.0075998 -0.0047135
    #>                                                               Bias.3    rMSE.1
    #> 04040404 Timetrend0000 main stage5                        -0.0047363 0.3071620
    #> 04040404 Timetrend01010101 main stage5                     0.0005532 0.3465008
    #> 04040404 Timetrend01010101 main + stage_continuous stage5 -0.0064670 0.3158225
    #>                                                              rMSE.2    rMSE.3
    #> 04040404 Timetrend0000 main stage5                        0.3216471 0.3210915
    #> 04040404 Timetrend01010101 main stage5                    0.3470985 0.3441727
    #> 04040404 Timetrend01010101 main + stage_continuous stage5 0.3253357 0.3160819
    #>                                                           N.per.arm.1
    #> 04040404 Timetrend0000 main stage5                           149.5994
    #> 04040404 Timetrend01010101 main stage5                       149.8267
    #> 04040404 Timetrend01010101 main + stage_continuous stage5    150.9300
    #>                                                           N.per.arm.2
    #> 04040404 Timetrend0000 main stage5                           150.4572
    #> 04040404 Timetrend01010101 main stage5                       149.7345
    #> 04040404 Timetrend01010101 main + stage_continuous stage5    149.1794
    #>                                                           N.per.arm.3
    #> 04040404 Timetrend0000 main stage5                           149.7173
    #> 04040404 Timetrend01010101 main stage5                       149.7167
    #> 04040404 Timetrend01010101 main + stage_continuous stage5    149.6287
    #>                                                           N.per.arm.4
    #> 04040404 Timetrend0000 main stage5                           149.7461
    #> 04040404 Timetrend01010101 main stage5                       149.8821
    #> 04040404 Timetrend01010101 main + stage_continuous stage5    149.7699
    #>                                                           Survive.per.arm.1
    #> 04040404 Timetrend0000 main stage5                                  59.8228
    #> 04040404 Timetrend01010101 main stage5                              67.3003
    #> 04040404 Timetrend01010101 main + stage_continuous stage5           67.9718
    #>                                                           Survive.per.arm.2
    #> 04040404 Timetrend0000 main stage5                                  60.1033
    #> 04040404 Timetrend01010101 main stage5                              67.2081
    #> 04040404 Timetrend01010101 main + stage_continuous stage5           66.9753
    #>                                                           Survive.per.arm.3
    #> 04040404 Timetrend0000 main stage5                                  59.9199
    #> 04040404 Timetrend01010101 main stage5                              67.2191
    #> 04040404 Timetrend01010101 main + stage_continuous stage5           67.2767
    #>                                                           Survive.per.arm.4
    #> 04040404 Timetrend0000 main stage5                                  59.8687
    #> 04040404 Timetrend01010101 main stage5                              67.2769
    #> 04040404 Timetrend01010101 main + stage_continuous stage5           67.1882
    #>                                                                 N
    #> 04040404 Timetrend0000 main stage5                        599.520
    #> 04040404 Timetrend01010101 main stage5                    599.160
    #> 04040404 Timetrend01010101 main + stage_continuous stage5 599.508

Then, We investigate the other evaluation metrics for alternative scenario with different time trend strength. The cutoff value was the same as the value in precious example since the control arm response probability in alternative scenario are the same as that in null scenario.

    ntrials = 1000 # Number of trial replicates
    ns = seq(120,600,120) # Sequence of total number of accrued patients at each interim analysis
    null.reponse.prob = 0.4
    alt.response.prob = 0.6

    # We investigate the type I error rate for different time trend strength
    alt.scenario = matrix(
      c(
        null.reponse.prob,
        alt.response.prob,
        null.reponse.prob,
        null.reponse.prob,
        null.reponse.prob,
        alt.response.prob,
        alt.response.prob,
        null.reponse.prob,
        null.reponse.prob,
        alt.response.prob,
        alt.response.prob,
        alt.response.prob
      ),
      nrow = 3,
      ncol = 4,
      byrow = T
    )
    model = "tlr" #logistic model
    max.ar = 0.75  #limit the allocation ratio for the control group (1-max.ar < r_control < max.ar)
    #------------Select the data generation randomisation methods-------
    rand.type = "Urn" # Urn design
    max.deviation = 3 # The recommended value for the tuning parameter in the Urn design

    # Require multiple cores for parallel running
    cl = 2

    # Set the model we want to use and the time trend effect for each model used.
    # Here the main model will be used twice for two different strength of time trend c(0,0,0,0) and c(1,1,1,1) to investigate how time trend affect the evaluation metrics in BAR setting.
    # Then the main + stage_continuous model which is the treatment effect + stage effect model will be applied for strength equal c(1,1,1,1) to investigate how the main + stage effect model improve the evaluation metrics.
    reg.inf = c("main + stage_continuous")
    trend.effect = matrix(c(0.1, 0.1, 0.1, 0.1),
                          ncol = 4,
                          nrow = 1,
                          byrow = T)

    #
    cutoffearly = matrix(rep(0.994, dim(alt.scenario)[1]), ncol = 1)

    K = dim(alt.scenario)[2]
    print(
      paste0(
        "Start trial simulation. This is a ",
        K,
        "-arm trial simulation. There are one null scenario and ",
        K - 1 ,
        " alternative scenarios. There are ",
        K ,
        " rounds."
      )
    )
    #> [1] "Start trial simulation. This is a 4-arm trial simulation. There are one null scenario and 3 alternative scenarios. There are 4 rounds."
    cutoffindex = 1

    result = {

    }
    OPCalt = {

    }
    for (i in 1:dim(alt.scenario)[1]) {
      trendindex = 1
      for (j in 1:length(reg.inf)){
        restlr = Trial.simulation(
          ntrials = ntrials,
          # Number of trial replicates
          trial.fun = simulatetrial,
          # Call the main function
          input.info = list(
            response.probs = alt.scenario[cutoffindex, ],
            #The scenario vector in this round
            ns = ns,
            # Sequence of total number of accrued patients at each interim analysis
            max.ar =  max.ar,
            #limit the allocation ratio for the control group (1-max.ar < r_control < max.ar)
            rand.type = rand.type,
            # Which randomisation methods in data generation.
            max.deviation = max.deviation,
            # The recommended value for the tuning parameter in the Urn design
            model.inf = list(
              model = model,
              #Use which model?
              ibb.inf = list(
                #independent beta-binomial model which can be used only for no time trend simulation
                pi.star = 0.5,
                # beta prior mean
                pess = 2,
                # beta prior effective sample size
                betabinomialmodel = ibetabinomial.post # beta-binomial model for posterior estimation
              ),
              tlr.inf = list(
                beta0_prior_mu = 0,
                # Stan logistic model t prior location
                beta1_prior_mu = 0,
                # Stan logistic model t prior location
                beta0_prior_sigma = 2.5,
                # Stan logistic model t prior sigma
                beta1_prior_sigma = 2.5,
                # Stan logistic model t prior sigma
                beta0_df = 7,
                # Stan logistic model t prior degree of freedom
                beta1_df = 7,
                # Stan logistic model t prior degree of freedom
                reg.inf =  reg.inf[trendindex],
                # The model we want to use
                variable.inf = "Fixeffect" # Use fix effect logistic model
              )
            ),
            Stopbound.inf = Stopboundinf(
              Stop.type = "Early-Pocock",
              # Use Pocock like early stopping boundary
              Boundary.type = "Symmetric",
              # Use Symmetric boundary where cutoff value for efficacy boundary and futility boundary sum up to 1
              cutoff = c(cutoffearly[cutoffindex, 1], 1 - cutoffearly[cutoffindex, 1]) # The cutoff value for stopping boundary
            ),
            Random.inf = list(
              Fixratio = FALSE,
              # Do not use fix ratio allocation
              Fixratiocontrol = NA,
              # Do not use fix ratio allocation
              BARmethod = "Thall",
              # Use Thall's Bayesian adaptive randomisation approach
              Thall.tuning.inf = list(tuningparameter = "Fixed",  fixvalue = 1) # Specified the tunning parameter value for fixed tuning parameter
            ),
            trend.inf = list(
              trend.type = "step",
              # Linear time trend pattern
              trend.effect = trend.effect[trendindex, ],
              # Stength of time trend effect
              trend_add_or_multip = "mult" # Multiplicative time trend effect on response probability
            )
          ),
          cl = 2 # 2 cores required
        )
      trendindex = trendindex + 1
      # The result list can be used for plotting and the OPC table is the summary evaluaton metrics for each scenario
      result = c(result, restlr$result)
      OPC_alt = rbind(OPC_alt, restlr$OPC)
      }
      cutoffindex = cutoffindex + 1
    }

    print("Finished alternative scenario study")
    #> [1] "Finished alternative scenario study"
    save_data = FALSE
    if (isTRUE(save_data)) {
      save(result, file = restlr$Nameofsaveddata$nameData)
      save(OPC_alt, file = restlr$Nameofsaveddata$nameTable)
    }

Present the evaluation metrics for alternative scenarios. The power used here is the conjunctive power where the trial will be sucessful only if the effective arms are correctly claimed to be effective and all other null arms are claimed to be ineffective.

    # Characteristic table
    print(OPC_alt)
    #>                                                           Type.I.Error.or.Power
    #> 04060404 Timetrend01010101 main + stage_continuous stage5                0.5148
    #> 04060604 Timetrend01010101 main + stage_continuous stage5                0.6612
    #> 04060606 Timetrend01010101 main + stage_continuous stage5                0.5621
    #>                                                              Bias.1     Bias.2
    #> 04060404 Timetrend01010101 main + stage_continuous stage5 0.1008942 -0.0200847
    #> 04060604 Timetrend01010101 main + stage_continuous stage5 0.1382469  0.1426271
    #> 04060606 Timetrend01010101 main + stage_continuous stage5 0.1195935  0.1226108
    #>                                                               Bias.3    rMSE.1
    #> 04060404 Timetrend01010101 main + stage_continuous stage5 -0.0241128 0.4537455
    #> 04060604 Timetrend01010101 main + stage_continuous stage5 -0.0192185 0.4234063
    #> 04060606 Timetrend01010101 main + stage_continuous stage5  0.1188816 0.4224861
    #>                                                              rMSE.2    rMSE.3
    #> 04060404 Timetrend01010101 main + stage_continuous stage5 0.3757555 0.3831709
    #> 04060604 Timetrend01010101 main + stage_continuous stage5 0.4293264 0.3449225
    #> 04060606 Timetrend01010101 main + stage_continuous stage5 0.4294485 0.4305815
    #>                                                           N.per.arm.1
    #> 04060404 Timetrend01010101 main + stage_continuous stage5    116.5303
    #> 04060604 Timetrend01010101 main + stage_continuous stage5    124.5913
    #> 04060606 Timetrend01010101 main + stage_continuous stage5     79.0515
    #>                                                           N.per.arm.2
    #> 04060404 Timetrend01010101 main + stage_continuous stage5    200.7436
    #> 04060604 Timetrend01010101 main + stage_continuous stage5    153.2433
    #> 04060606 Timetrend01010101 main + stage_continuous stage5    140.7811
    #>                                                           N.per.arm.3
    #> 04060404 Timetrend01010101 main + stage_continuous stage5    141.4709
    #> 04060604 Timetrend01010101 main + stage_continuous stage5    152.4769
    #> 04060606 Timetrend01010101 main + stage_continuous stage5    140.7366
    #>                                                           N.per.arm.4
    #> 04060404 Timetrend01010101 main + stage_continuous stage5    140.8112
    #> 04060604 Timetrend01010101 main + stage_continuous stage5    164.8645
    #> 04060606 Timetrend01010101 main + stage_continuous stage5    141.5108
    #>                                                           Survive.per.arm.1
    #> 04060404 Timetrend01010101 main + stage_continuous stage5           85.8683
    #> 04060604 Timetrend01010101 main + stage_continuous stage5           56.3024
    #> 04060606 Timetrend01010101 main + stage_continuous stage5           34.3717
    #>                                                           Survive.per.arm.2
    #> 04060404 Timetrend01010101 main + stage_continuous stage5          172.9814
    #> 04060604 Timetrend01010101 main + stage_continuous stage5           98.2435
    #> 04060606 Timetrend01010101 main + stage_continuous stage5           90.4030
    #>                                                           Survive.per.arm.3
    #> 04060404 Timetrend01010101 main + stage_continuous stage5          107.4409
    #> 04060604 Timetrend01010101 main + stage_continuous stage5           97.8080
    #> 04060606 Timetrend01010101 main + stage_continuous stage5           90.3950
    #>                                                           Survive.per.arm.4
    #> 04060404 Timetrend01010101 main + stage_continuous stage5          106.9263
    #> 04060604 Timetrend01010101 main + stage_continuous stage5           75.2079
    #> 04060606 Timetrend01010101 main + stage_continuous stage5           90.7811
    #>                                                                 N
    #> 04060404 Timetrend01010101 main + stage_continuous stage5 599.556
    #> 04060604 Timetrend01010101 main + stage_continuous stage5 595.176
    #> 04060606 Timetrend01010101 main + stage_continuous stage5 502.080