DQ 1 PLEASE USE ATTACHED ARTICLE ONLY

Applying Bayes'Theorem to clinical trials by Tom Lecklider, Senior Technical Editor Ov er the y ears, m any w rite rs have im p lie d that statistics can provide almost any result that is con­ venient at the time. O f course, honest practitioners use statistics in an attempt to quantify the probability that a certain hypothesis is true or false or to better understand what the data actually means.

The field o f statistics has been developed over more than 200 years by famous mathematicians such as Laplace, Gauss, and Pascal and more recently Markov, Fisher, and Wiener.

Pastor Thomas Bayes (1702-1761) appears to have had little influence on mathematics outside o f statistics where B ayes’ Theorem has found wide application.

As described in the F D A ’s 2010 Guidance... f o r the Use o f Bayesian Statistics in Medical Device Clinical Trials, “Bayes­ ian statistics is an approach for learning from evidence as it accumulates. In clinical trials, traditional (frequentist) statistical methods may use information from previous studies only at the design stage. Then, at the data analysis stage, the information from these studies is considered as a complement to, but not part of, the formal analysis. In contrast, the Bayesian approach uses B ayes’ Theorem to formally combine prior information with current information on a quantity o f interest. The Bayesian idea is to consider the prior information and the trial results as part o f a continual data stream, in which inferences are being updated each time new data becomes available.” 1 B a y e s ' T h e o r e m As explained in the FDA’s Guidance document, prior information about a topic that you wish to investigate in more detail can be combined with new data using Bayes’ Theorem. Symbolically, p(AlB) = p(BIA) x p(A)/p(B) where: p(AIB) = the p o s te rio r p ro b a b ility o f A o c c u rrin g given condition B p(BIA) = the lik elih o o d p ro b a b ility o f c o n d itio n B being true when A occurs p(A) = t h e p r i o r p r o b a b i l i t y o f o u t c o m e A occurring regardless o f condition B p(B) = th e e v id e n c e p r o b a b i l i t y o f c o n d itio n B being true regardless of outcome A Reference 2 discusses the application of Bayes’ Theorem to a horse-racing example. In the past, a horse won five out of 12 races, but it had rained heavily before three o f the five wins.

One race was lost when it had rained. W hat is the probability that the horse will win the next race if it rains?

We want to know p(winning I it has rained). W e know the following:

p(it has rained I winning) = 3/5 = 0.600 p(winning) = 5/12 = 0.417 p(raining before a race) = 4/12 = 0.333M arko v g r a p h o f tra n sitio n p ro b a b ilitie s b e tw e e n states A, B, a n d C7 Courtesy o f Skye Bender-deMoll From Bayes Theorem, p(winning I it has rained) = 0.600 x 0.417/0.333 = 0.75. Taking into account the horse’s prefer­ ence for a wet track significantly changes its odds o f winning compared to 0.417 when rain is not considered.2 Typically, actual situations are not this simple but instead involve many variables and dependencies. Also, the discrete p ro b ab ilities o f the h o rse-racing exam ple are rep laced by probability density functions (PDFs). Common PDFs, such as the fam iliar bell curve o f the norm al distribution, show the likelihood that a variable will have a certain value. Often, researchers need to know that a quantity is larger or smaller than some limit or that it falls within a certain range, which requires integrating part o f the area under the PDF curve.

Dr. John Kruschke, Department o f Psychological and Brain Sciences, Indiana University, described a learning experiment in which a person is shown single words and combinations of two words on a computer screen. The object is to learn which keys to press in response to seeing a word or combination of words. The lengths o f all the response times (RT) between a new word or combination appearing and the correct key being pressed comprise the test data.

All together, there were seven unique words or combinations, called cues, randomly presented to learners often enough that each cue repeated many times. There were 64 learners involved in the study. The objectives were to “ ...estim ate the overall baseline RT, the deflection away from the baseline due to each test item, and the deflection away from the baseline due to each subject.”3 This exam ple is not nearly as large or com plex as many medical trials but still was addressed through the Bayesian inference using the Gibbs sampling (BUGS) computer program initially developed by the Medical Research Council Biosta­ tistics Unit in Cambridge, U.K. A great deal o f information is 22 • EE • M a rc h 2015w w w .e v a iu a tio n e n g in e e rin g .c o m ! " #" $ % $ & & $ % ' & $ $ % () * $ % + % & ' % ! $ Analysis As discussed in Reference 4,“A major limitation towards more widespread implementation o f Bayesian approaches is that ob­ taining the posterior distribution often requires the integration o f high-dimensional functions. This can be computationally very difficult, but several approaches short o f direct integration have been p roposed....” For many studies, each data point is independent. Flere, a data point is the value of a multidimensional vector— the set of answers that a certain respondent gave to a questionnaire. The set of responses given by all the respondents often is considered to be a Markov process, or more particularly a Markov chain because there is a finite number o f discrete states that the vector assumes. In a Markov chain, the next value only depends on the current state— neither the preceding states nor their order is important. The successive states observed when repeatedly flipping a coin comprise a Markov chain.

Another concept that is key to addressing practical appli­ cations o f B ayes’ Theorem is Monte Carlo integration. The Monte Carlo approach can be thought o f as a massively parallel set o f random trials that is evaluated to estimate a solution.

Stanislaw Ulam, who developed the technique when working at Los Alamos in 1946, wrote, “The question was what are the chances that a Canfield solitaire laid out with 52 cards will come out successfully? After spending a lot o f time trying to estimate them by pure combinatorial calculations, I wondered whether a more practical method than ‘abstract thinking’ might not be to lay it out say one hundred times and simply observe and count the number o f successful plays.”5 Monte Carlo integration approximates an integral ^ h ( x ) d x by first d eco m posing the integrand h(x) into the product o f a separate function f(x) and a p ro b ability den sity p(x) [ h ( x ) d x = f f ( x ) p ( x ) d x . If this can be done, then the J a J a second integral is equal to the expected value Ep(x)[f(x)] of f(x) over the interval [a,b]. For large n, this is approximated where each xj is randomly drawn front the p(x) PDF.

Rather than flipping a single coin and recording successive outcomes, the Monte Carlo approach to determ ining that a coin has equal probability o f coming up heads or tails is to sim ultaneously flip thousands o f identical coins and thencompare the num ber o f resulting heads and tails. A Monte C arlo sim ulation, on the o th er hand, w ould accum ulate a very large number o f random-value samples from the interval [0,1], assigning heads to values >0.5 and tails to those <0.5.

The quantities o f each would then be com pared to determ ine bias.

Reference 6 combines the two ideas: “Markov chain Monte Carlo (MCMC) is a collection o f sampling methods that is based on following random walks on Markov chains.” Markov chains for which there is a finite probability o f transitioning from any state to any o ther are term ed ergodic. The PDF describing the frequency with which the various states o f an ergodic chain occur approaches a stationary distribution after a sufficiently large number o f transitions— the so-called burn-in time. The transitions from one state to the next form a multidim ensional path— a random walk.

Gibbs sampling performs a special kind o f random walk in which, “ ...a t each iteration, the value along a randomly selected dim ension is updated according to the conditional distribution.” B ayes’ posterior jo in t probability distribution is defined as the product o f conditional d istrib u tio n s, and Gibbs sam pling is said to work well in this case.6 Review The initial design of a medical survey largely influences the usefulness of the results. Bayes’ Theorem allows results from a previous study to be combined with the current study, and it also provides the opportunity to monitor data as the study progresses. However, correctly analyzing the complex jo in t probability distributions characteristic of this approach requires a statistician trained in the use o f Bayes’ Theorem.

As stated in the F D A ’s G uidance docum ent, “D ifferent choices o f prior information or different choices o f model can produce different decisions. As a result, in the regulatory setting, the design o f a Bayesian clinical trial involves prespecification o f and agreement on both the prior information and the model.

Since reaching this agreement is often an iterative process, we recommend you meet with the FDA early to obtain agreement upon the basic aspects o f the Bayesian trial design.

“A change in the prior information or the model at a later stage o f the trial may imperil the scientific validity o f the trial results. For this reason, formal agreement meetings may be appropriate when using a Bayesian approach.” 1 References 1. Guidance f o r Industry and FDA Staff: Guidance f o r the Use o f Bayesian Statistics in Medical Device Clinical Trials, FDA, Feb. 5, 2010.

2. Boone, K., “Bayesian Statistics for Dummies,” 2010.

3. Kruschke, J. K., “Bayesian data analysis,” WIREs Cogni­ tive Science, 2010.

4. Walsh, B., “Markov Chain Monte Carlo and Gibbs Sam­ pling,” Lecture notes for EEB 581, April 2004.

5. Eckhardt, R., “ Stan Ulam, John Von Neumann, and the Monte Carlo Method,” Los Alamos Science Special Issue, 1987.

6. Lebanon, G., “Metropolis-Hastings and Gibbs Sampling,” November 2006.

7. S kye B e n d e r-d e M o ll, “ In fo rm a tio n , U n c e rta in ty , and Meaning,” May 16, 2001. EE w w w .e v a lu a tio n e n g in e e rin g .c o m M a rc h 2015 • EE • 23 Copyright ofEE: Evaluation Engineering isthe property ofNP Communications, LLCandits content maynotbecopied oremailed tomultiple sitesorposted toalistserv without the copyright holder'sexpresswrittenpermission. However,usersmayprint, download, oremail articles forindividual use.