Roberts, G. on Retrospective Sampling

The talk was delivered sometime in March and concentrated on three main features: restrospective sampling, simulation of diffusions and Bayesian inference from Dirichlet mixture models. Basically problem of interest arises in the situation where it involves infinite dimensional objects. The names MCMC and rejection sampling are already known as modern simulation algorithms which in principle is, simple.

Retrospective sampling, on the other hand is powerful in the context of infinite dimensional. The computation when using this method is inexpensive and exact. A few simulation examples illustrated in the talk are for unnormalized probability density function, alternating series and retrospective MCMC.

Simulation of diffusions is a Markov process described by stochastic differential equation. It involves Brownian motion and Euler approximation. An interest lies in simulating without discretisation error and obtaining realisations of the whole path.

Bayesian Adaptive Non-Inferiority With Safety Assessment: Retrospective Case Study to Highlight Potential Benefits and Limitations of the Approach

According to the research done by Di Masi et al. (2003) 800 million dollar is needed in order to have a new entity reach the market. Another source from the Investor’s Business Daily (2007) stated that the amount spent  is actually 1 billion dollar. The years spent is estimated to be around 15 years. Many attention is now  focused to studies that reduce the cost and the time. Adaptive design is a solution found to this problem, where modification is done in the middle of the trial based on the information accrued. Adaptive design can be applied using both the frequentist and Bayesian method. The intention of the authors of this paper, Spann et al. (2008 ) is to:-

1. Apply Bayesian adaptive to a case study

2. Demonstrate the benefit of it and provide advance understanding of using this approach in regulatory environment.

The authors focused on phase III study, in a trial for patients who had DSM-IV diagnosis of schizophrenia and related disease. The methods advocated here follow all aspects of trial protocol except that a Bayesian approach is employed in the analyses, decision criteria and treatment allocation which is based on predictive probability of treatment response without the side effect. The prior was obtained by combining historical pharmacokinetics data and expert opinon. Patients were randomized first according to the ratio 2:2:1 and then Bayesian adaptive is applied to assign treatment to patients. To assign this treatment, it depends on the calculated joint predictive probability of treatment safe and patient response. This method will reduce patients’ exposure to ineffective treatments or treatments with adverse side effects. MCMC was used to draw since the posterior distribution did not have a closed form.

The authors reached the same conclusion to the analysis previously done, except that they could stop early for efficacy. They required only a half number of patients from the original frequentist study. Since the trial completed earlier, they estimated that 40% of original time and 1 million dollar were saved. Sensitivity of the design with respect to prior distributions was investigated by simulation. If informative priors were used with respect to simulated data, then power is 100% for rejecting false null hypothesis. On the other hand if noninformative priors with respect to simulated data were used, power can be achieved up to 79%.

The authors believed that adaptive design is a favorable approach to ensure that the new entity reach the market sooner and people can obtain the medicine at a lower cost. At the end of this paper, Spann et al. (2008 ) wrote a bit on the difference of frequentist and Bayesian ideas.

Bayesian Sample Size Calculations for a Non-Inferiority Test of Two Proportions in Clinical Trials

In the setting where discrete variables are of interest (i.e. cinical response, complete response, survival in cancer trials or presence of adverse events), proportions are to be compared. For two independent groups, Neyman-Pearson hypothesis testing can be used to find the appropriate sample size. This approach needs the true difference between two groups, type I and type II error and the value of non-inferiority margin. It is usually difficult to find consensus on the margin and what happens in practice is various sample size will be calculated based on various opinions. These sample sizes will be compared to the sample size calculated based on the agreed non inferiority margin.

The purpose of this paper written by Daimon (2008 ) was to demonstrate the use of Bayesian approach to calculate the sample size, by assuming normal prior for the “sum of the true difference and the non-inferiority margin”. Note that in the frequentist approach, uncertainty surrounds the value given to the true difference and the margin. However, when one construct the equation for the sample size, it can be seen that those two values can be taken as one unknown value, which is the “sum of the true difference and the non-inferiority margin”.

Daimon first described the idea of Neyman-Pearson power before giving a detailed account on Bayesian probabilities. In the context of Bayesian approach, he discussed three possible methods to find the necessary sample size:

Hybrid Neyman-Pearson-Bayesian  probability (hNPB): This is also known as the expected power or strength. It takes account of available prior and is used to critically evaluate Neyman-Pearson power formula. It utilizies Neyman-Pearson critical region.

Conditional Bayesian probability (CB): This is a fully Bayesian design where priors are incorporated explicitly. This method allows one to find the probability of obtaining a significant Bayesian result, for any given data.

Unconditional Bayesian probability (UB): Probability of obtaining a significant Bayesian result is not conditional on the pre-specification of the sum of alternative difference and noninferiority margin.

Sample size are calculated and compared in these four approaches: Neyman-Pearson, hNPB, CB dan UB. It is found that sample size from CB is smaller to what has been obtained when using Neyman-Pearson. Sometimes, when using Bayesian approach, there is a possibility of obtaining only 1 subject for the trial, and this is not realistic. So in this situation, Bayesian method is not applicable.

Daimon proposed that this method could possibly be applied to hazard ratios, odds ratios and continuous variables because it uses a large sample aprroximation and the conjugate normal analysis. Daimon suggested that it is worth to check all the sample sizes from those methods and choose the appropriate sample size. It is also a good practice to check the validity of prior specification; for example normal prior here if not reliable can be substituted with beta prior or log normal prior.

A Hybrid Bayesian-Frequentist Approach to Evaluate Clinical Trial Designs for Tests of Superiority and Non-Inferiority

To conduct a phase III trial, one relies on the result of phase II trial. In theory, if the preliminary test in phase II trial resulted in superiority of experimental to reference, then this will lead to testing superiority in the phase III trial. However, if the result in phase II trial showed that experimental is slightly worse than the reference but less toxicity then non-inferiority test will be conducted in phase III.  Sometimes, under certain condition, there is uncertainty surrounding the result of phase II trial; say the uncertainty in the experimental treatment. Therefore, in phase III trial, one is not sure whether to do a superiority or non-inferiority trial. To solve this problem, one will design a trial that will test both, superiority and non-inferiority. Existing designs that consider both objectives are multistage group sequential, adaptive design and conventional single-stage.

Sample size calculation is an essential step in any designed trial.A conventional single-stage design for testing both objectives will choose the sample size based on the primary objective. In this situation, power for the second objective is going to be affected and the potential to reproduce the result is lowered as well. Given these problems, the athours of this paper, Shao et al. (2008 ) proposed a hybrid Bayesian-frequentist method. The essence of this approach is by specifying a probability distribution to the experimental treatment as a way to express uncertainty surrounding the efficacy.

The authors discussed this idea in the context of binomial data of 2-arm trial, involving the experimental and the reference treatment.  Testing for both objectives is done based on closed-testing principle. Based on this principle, there is no inflation of type I error because the null hypotheses are nested (Morikawa and Yoshida, 1995) and (Dunnett and Tamhane, 1997).  However, type II error is inflated and this leads to a severe loss of resources. Shao and Chow (2002) stated that result of the trial in phase III must be able to reproduce and so there will be an issue here when the power of the trial is low.  Kayama and Westfall (2005) favoured this design rather than the test-one-only design, based on their study conducted from the perspective of decision theory.

The authors of this paper applied Bayes formula in tackling the issue of power and reproducibility in this two-tests design. The uncertainty of either experimental or reference treatment is expressed via probability distribution. They considered two situations: first situation is when experimental is uncertain but reference is a known constant and the second situation is when both are uncertain. Power is evaluated with respect to the uncertainty of experimental. There are two types of power: adjusted power and average power.  The sensitivity of the result to the choice of distribution is studied as well. The final analysis however does not rely on any distribution but uses the classical frequentist methods.

25 Years of Bayesian Methods in the Pharmaceutical Industry: A personal, Statistical Bummel

This article by Grieve (2007) was published in Pharmaceutical Statistics. He started off by describing his personal journey into the world of pharmaceutical industry. He first joined Ciba-Geigy as a medical statistician and had an interest in developing the idea of Bayesian in the pharmaceutical R&D.

There is a section in this article, where the author described the basic of Bayesian idea, called “Bayesics“. One must be familiar with the terms such as: probability model for the data, prior probability, posterior probability, predictive distribution for future data, marginal posterior density, bivariate marginal density, posterior moments and highest posterior density.There are many areas of clinical trials where Bayesian has come into the picture. One of them is bioequivalence.

The aim of bioequivalence study is to determine whether two formulations of a drug are equivalent. Westlake (1972) claimed that hypothesis testing is not relevant. His claim was supported by Metzler (1972). Westlake argued that clinical pharmacologists work with ratios rather than difference and so confidence interval is more appropriate in that sense. Bayesian approach started to develop in the bioequivalence area in 1979 by three independent groups: Merck, Sharp and Dohme, Ciba-Geigy (US) and Ciba-Geigy (Swiss). In 1983, the ratio of the area under the curve (AUC) was commonly accepted as 0.8 and 1.2 and that 95% confidence interval is required for that ratio. Bayesian approach advocates the use of credible interval or posterior probability.

In the situation where a small single bioequivalence study is being conducted, proving that two drugs are similar in efficacy would be difficult. So, a 2-stage Bayesian approach is adopted. The first stage is called the screening stage and the sample size chosen here is small. Sample size in the second stage is chosen based on the result of the screening stage. This is an early example of adaptive design and making use of predictive distribution.

The standard approach to bioequivalence today is by conducting two “one-sided t-test”and so simultaneous rejection will indicate that two treatments are equivalent. Bauer and Bauer (1994) raised a question where the means of those two could possibly be close but the variances on the other hand, are large. They proposed testing an extra two “one-sided F test”; so there will be four null hypotheses altogether to be rejected to be able to claim bioequivalence. In the Bayesian perspective, this can be done by determining the joint posterior of say, the difference in means and the ratio of variances.

Grieve did not review the implementation of Bayesian method in non-inferiority trial. The non-inferiority term was first introduced by Blackwelder (1982). However, he quoted a statement from Susan Ellenberg:  “area of non-inferiority is one that really lends itself to the Bayesian approach”, which pretty much summarized up Grieve’s idea.

In the late 1970s, the crossover design received many attention. The basic design is by having 2 treatments with 2 periods. The FDA however, said that it is not a good design when “unequivocal evidence” of a treatment effect was required. In the 1980s, Grieve started to work on Bayesian approach in the crossover design. Bayes factor is used to determine whether carryover effect should be included or not. In this section, Behrens-Fisher distribution came up as well. The problem of Behrens-Fisher is about interval estimation and hypothesis testing betwen the means of two normally distributed population, when the population variances are assumed unequal and come from independent samples.

In the early to mid 1980s, analytic approximation to posterior quantities of interest were being studied. There are four approaches to analytic approximation:

Specific expansion for ratios of integral to determine posterior moments and predictive distributions: This method requires calculation up to third order derivatives of log likelihood function, which is cumbersome. It also provides no correction to variance and covariance.

Laplace approximation to integrals: The advantage of this compared to the first one, is it only evaluates up to second derivative only. It is also possible to find marginal distribution by integrating over a subset of the parameters in the numerator.

Importance sampling: The idea is to choose a density of the parameter that approximates the probability model of the data. The efficiency depends on how well the approximation is.

Iterated Gaussian quadrature: Basic assumption of this method is posterior can be approximated by the multiplication of normal density and low order polynomial. In practice, we would not know the posterior mean and the covariance matrix and so Gaussian quadrature will update the estimates till it converges.

All the approaches described above behave well in small-dimensional problems; that is up to 6-10 parameters. If the parameters increased, the best way is to use Gibbs sampling or MCMC.

In the early 1980s, time was spent to studying the analysis of animal toxicology. The objective of this study is to estimate a dose-response curve and to determine LD50, a dose that kills 50% of the animals exposed to it in a fixed time interval. For this kind of design, logit and probit model are used and parameters of the model are estimated using the maximum likelihood estimator, weighted least squares and Fieller’s theorem. Fieller’s theorem is used to find the confidence interval for the log of LD50. Technical issues are noted to arise when using Fieller’s method: the interval consists of whole real line and as a result, little is learnt about LD50. This is said due to a small number of experimental units being studied. Grieve hoped that Bayesian could circumvent this issue.

Systematic investigation regarding the use of Bayesian in the drug development was carried out in 1982. The work was published in Racine et al. (1986). The obstacles identified in incorporating Bayesian approach are related to numerical implementation, utilizing priors in regulated environment and conservatory attitute among pharmaceutical statisticians and respective colleagues. Compared to the same paper read by John Lewis in 1983, Racine et al. (1986) was received in defeaning silence by the Royal Statistical Society.

The MCMC revolution in the 1990s enable multi-parameter problems to be tackled without having to simplify the models or the assumptions. The general form of MCMC is based on Metropolis-Hastings algorithm which is a special case of Gibbs sampler. The idea of Gibbs sampler is like this: say we have k parameters of interest and we want to sample from the posterior density. Starting values for the parameters are given and sampling is done in sequence for a large number of iterations, say 10 000. In practice, the first say, 1000 iterations are to be discarded, also called as “burn-in”. For a more complex problems the number of discarded iterations could be more than 1000. The mean of values from the sample is the estimate of the posterior mean. The same goes for the posterior variance. The kernel estimate is used to construct the graph of a posterior.

Other than the issue of efficacy and safety, one question that need to be answered is whether the drug can be manufactured consistently in terms of content, uniformity and level of contaminants. So, batches of drugs have to be analyzed and Bayesian approach can be implemented here. Historical data will be taken into consideration and a prior is formed. For example, one could assume that proportion of defectives in a particular batch has a beta distribution. The number to be sampled from the batch will be based on these calculation.

Adaptive dose-finding studies make an appearance in the 21st century, though the idea has already been discussed in 1933 by William Thompson. William described this design as “play-the-winner” design. There are two classes of design: one is to determine maximum tolerated dose in oncology studies and the other is to learn about dose-response curve when it can be non-monotonic.

Pharmacoeconomics is another area where concern arises due to escalating health-care costs. So, demonstration value for money is required and the method is by evaluating the cost data. From the Bayesian point of view, probability statements can be made about populations incremental mean values. John Stevens and Tony O’ Hagan are among the famous statisticians working in this area.

To conclude, Bayesian approach enable direct probability statements about parameters to be made and easier to be interpreted. Full accounting of uncertainty can be seen for example in using Bayes factor in the crossover design. Other than estimating and testing hypothesis, predicting is also of interest. Grieve suggested using predictive ideas. Predictive powers on the other hand is when it is conditioned on the observed data. Grieve quoted a statement from Stephen Senn: “nowhere will Bayesian statistics to be carried out with greater discipline than in the pharmaceutical industry”. Grieve predicted that Bayesian will be heading next to drug safety, playing the role in not just the development but the post-marketing as well.