Intro to Adaptive Trials, and the EAGLE Visualization Tool

We might think a treatment is effective, but not know how best to implement it.

• How strong is the treatment effect? (How many patients will we need to enroll to detect it?)
• Is it only effective in a subpopulation?
• Is it harmful to a subpopulation?
• What dose should we give?

All of these can correspond to decisions in the trial design stage

In this presentation we'll be focusing on the question of “Who should we enroll?”

• One approach is to conduct a series of traditional trials, each testing in a different subpopulation.
• Another approach is to start enrolling everyone, and use the data from initial patients to inform our decisions of who to keep enrolling as the trial continues.
  • The length, enrollment can take a long time. We will have some measurements before we've enrolled all our patients.
  • Generally, we fix a number of “stages” for the trial ($K$), and do a check at the end of each stage.

In any one trial, you could:

• Fix a subpopulation, run the whole trial.
• Fix a subpopulation, check at the end of each stage to see if you should end the trial early (for efficacy or futility).
  • Group Sequential Trial with Fixed Enrollment
• Split population into two subpopulations. At the end of each stage, check for efficacy and futility in each subpopulation.
  • Group Sequential Trial with Adaptive Enrollment

• Check Z-statistic for treatment effect at the end of each stage.
  • Let $Z_k$ be the test statistic using all the data up to stage $k$.
• If $Z_k$ crosses an efficacy boundary (threshold), stop the trial and declare success.
  • Analagous to the classic “1 stage” trial, where the boundary is just the value 1.96 (for α =.05).
• If $Z_k$ crosses a futility boundary, stop the trial and declare failure. These don't affect $α$.

• People often use efficacy bondaries of the form $cB(k)$, where $c$ is a “proportionality” constant, and $B(k)$ is a funtion of $k$.
  • $B(k)$= constant flat line, or a nice curve shape (last slide). $B(k)$ essentially specifies a member of a class of boundary shapes that are unique up to scaling.
  • For just one stage, could use $B(k)=1$, and $c=1.96$.
  • For multiple stages $c$ is calibrated such that the familywise type I error rate = $α$

$$P_{null}(Z_k>\text{cB(k), for any } k≤K ) <α$$

• One simple way is to simulate the trial a bunch of times with different boundaries, pick the boundary (for the $Z$-stats) that gives us the type I error rate we want.
• We can calculate this boundaries more efficiently though if we derive the null multivariate distribution of $\{Z_1,Z_2,...Z_k\}$.

• Z-stats $\{Z_1,Z_2,...Z_K \}$ have a known correlation matrix.
• Simulate from a MVNormal with mean zero, sd=1, and this correlation matrix.
• Given a boundary shape $B(k)$, use a binary search to find the constant $c$ satisfying

$$P_{null}(Z_k>cB(k) \text{, for any } k≤K ) <α$$

• Suppose we think treatment works better in subpopulation $A$ than in subpopulation $B$.
• We want to know if it works in the combined population $C$, and, if not, if it works in $A$. Thus, we have 2 nulls to test:
  • $H_{0C}$: No effect in the combined population
  • $H_{0A}$: No effect in subpop $A$.

• Again, our nulls are:
  • $H_{0C}$: No effect in the combined population
  • $H_{0A}$: No effect in subpop $A$.
• Let $Z_{k,C}$ and $Z_{k,A}$ be Z-statistics using all the data up through stage $k$, using the combined pop, and subpop $A$ respectively.
• To test $H_{0C}$, make efficacy boundaries for $\{Z_{C,1}, Z_{C,2},... Z_{C,K}\}$.
• To test $H_{0A}$, make efficacy boundaries for $\{Z_{A,1}, Z_{A,2},... Z_{A,K}\}$.

• We've got two boundaries, $c_AB_A(k)$ and $c_CB_C(k)$.
• If we tested both hypotheses at each stage, our family-wise error rate would be

$$P_{null}(Z_{A,k}>c_AB_A(k)\text{ or } Z_{C,k}>c_CB_C(k) \text{, for any } k≤K )$$

• Calibrating $c_A$ and $c_B$ so that this probability is less than $α$ requires us to know the multivariate distribution of $\{Z_{C,1}, Z_{C,2},... Z_{C,K}, Z_{A,1}, Z_{A,2},... Z_{A,K}\}$.

Anyone see where we glossed over more details?

• We said “under the null,” but with multiple subpopulations, it's not immediately clear what that means.
• It gets more complicated when the proportion of people in each subpopulation aren't known.