Statistics.du.se

D-level Essay in Statistics 2009
How to Analyze Change from Baseline:
Absolute or Percentage Change？ Högskolan Dalarna Lars Rönnegård 781 88 Borlänge Tel vx 023-778000 Richard Stridbeck How to Analyze Change from Baseline: Absolute or Percentage Change? In medical studies, it is common to have measurements before and after some medical interventions. How to measure the change from baseline is a common question met by researchers. Two of the methods often used are absolute changeand percentage change. In this essay, from statistical point of view, we will discuss the comparison of the statistical powerbetween absolute change and percentage change. What's more, a rule of thumb for calculation of the standard deviation ofabsolute change is checked in both theoretical and practical way. Simulation is also used to prove both the irrationality of theconclusion that percentage change is statistical ine¢ cient and the nonexistence of the rule of thumb for percentage change.
Some recommendations about how to measure change are put forward associated with the research work we have done.
Key Words: Absolute Change, Percentage Change, Baseline, Follow-up, Statistical Power, Rule of Thumb.
change to evaluate the change of weight. Neovius (2007) alsochose absolute change as the change measurement in their obesity research, while Kim (2009) chose percentage changeto measure the fat lost in di¤erent part of an obese man'sbody in a weight loss program. In a cystic …brosis clinical In medical studies, a common way to measure treatment study, Lavange (2007) used percentage change as well. We e¤ect is to compare the outcome of interest before treat- see that, both of the two methods have been used in di¤erent ment with that after treatment. The measurements before kinds of clinical studies.
and after treatment are known as the baseline (B) and the The properties of absolute change and percentage change follow-up (F ), respectively. How to measure the change from have been discussed by Tönqvist (1985). From his point of baseline is a common question met by researchers. There view, one of the advantages of percentage change is that per- are many methods that can be used as the measure of di¤er- centage change is independent of the unit of measurement.
ence. Two of them, which are used in a lot of clinical studies, For instance, a man who weighs 100 Kg lost 10% of weight are absolute change (C = B F ) and percentage change after a treatment, i.e. 10 Kg. Equivalently, he lost 22.05 F ) B). In di¤erent books and articles, absolute pounds (1Kg = 2:2046 Pounds). 10 Kg and 22.05 pounds are change may also be called change, while percentage change is essentially the same weight, but the absolute change scores also called relative change.
are di¤erent. However, no matter what the unit of measure- There is a simple example that will show us the di¤erence ment is, the percentage change is 10% all the time. More between absolute change and percentage change more clearly: details about the advantage of absolute change can be found Two obese men A and B participate in a weight loss program.
in the article of Tönqvist (1985). Although there are many Their weights at the beginning of the program are 150 Kg and advantages for the two change measurement methods, Tön- 100 Kg, respectively. When they …nish the program, the man qvist (1985) did not give any recommendations about how to A who weighs 150 Kg lost 15 Kg, while another man lost 10 make a choice between absolute change and percentage change Kg. From the example, we see that, the man A lost 5 Kg more based on these properties.
than that the man B, but the percent of weight they lost are10% in both cases. We want to know, which change measure- In other literatures, several suggestions about which ment is best to show the treatment e¤ect of the weight loss method to choose are mentioned. Vickers (2001) suggested avoiding using percentage change. That is because he com- In di¤erent clinical studies, either absolute change or per- pared the statistical power of di¤erent methods by doing centage change may be chosen. In the study of healthy dieting a simulation and concluded that percentage change from and weight control, Waleekhachonloet (2007) used absolute baseline is statistically ine¢ cient. Kaiser (1989) also gave How to Analyze Change from Baseline: Absolute or Percentage Change? some recommendations for making a choice between absolute 2. Comparison of the statistical change and percentage change. He suggested using the change power of absolute change and measurement that has less correlation with baseline scores. Atest statistic developed by Kaiser (1989) was also derived, i.e.
percentage change the ratio of the maximum likelihood of absolute change tothat of percentage change. The absolute change is recom-mended if the value of the test statistic is larger than one, In clinical research, it is common to test whether there is a treatment e¤ect after a medical intervention. In order to while percentage change is preferred when it is less than one.
test the treatment e¤ect, it's necessary to choose a suitable That is, a simple rule helping researchers to make a choice measurement of the di¤erence between baseline and follow-up Actually, the primary consideration for choosing a change From a statistical point of view, an important criterion for measurement method is di¤erent from di¤erent points of view.
a good statistical method is high statistical power. Therefore, From a clinical point of view, we prefer to use a change mea- from the two common change measurement methods, absolute surement that may show the health-improvement for the pa- change and percentage change, the one with a higher statis- tients in a more observable way. For example, in study of tical power will be preferred.
asthma, the primary outcome variable is often FEV (ForcedExpiratory Volume L/s).
The e¢ ciency of a treatment is 2.1 Statistical Power evaluated by calculating the percentage change in FEV frombaseline. In hypertension studies, it is common to use the ab- According to the hypothesis testing theory, statistical power solute change in blood pressure instead of percentage change.
is the probability that a test reject the false null hypothesis.
From statistical point of view, we prefer the method which The de…nition of statistical power can be expressed as has the highest statistical power as Vickers (2001) did.
equation (1).
Another issue concerned by researchers is the standard deviation of the treatment e¤ect (change scores). For two Statistical power = P (reject H0 j H0 is False) medical interventions that may lead to the same expected change, the e¤ect of the intervention that has a smaller stan- 0 is the null hypothesis.
In a t-test, equation (1) can be rewritten as equation (2).
dard deviation seems more stable and e¤ective. And clini-cians may always prefer that medical intervention. Since itis not practical for researchers to get all the interested exper- Statistical power P (reject H0 j H0 is False) imental datasets which recorded the details of baseline and follow-up scores for each patient, there is a rule of thumb 21 . It may help calculating the stan- dard deviation of change scores from the standard deviation where t =2 is the t-value under the signi…cant level of baseline scores.
two-side t-test, and pt is the p-value of the t-test.
The aim of this essay is to show, the statistical e¢ ciency From expression (2), we may see that the larger the ex- of percentage change under some conditions in contrast with pected absolute value of the t-statistic is, the higher the sta- Vickers' (2001) conclusion, and the rationality of the rule of tistical power will be.2 Equivalently, the smaller the expected thumb. The essay is organized as follows. The second section p-value is, the higher the statistical power will be. Therefore, is the comparison of the statistical power of absolute change from di¤erent measurement methods, we will choose the one and percentage change by constructing a test statistic under that has a larger expected absolute value of t-statistic or a certain distribution assumption. In the third section, the ra- smaller expected p-value.
tionality of rule of thumb is discussed in a theoretical way.
Simulations of some of the issues discussed in section two and 2.2 A Clinical Example of Blood Pressure three are carried out in the fourth section. The …fth section is an empirical investigation of the usefulness of the rule ofthumb by using some real datasets. Finally, in the discussion To easily interpret the di¤erence of two measurement meth- section, we discuss the results got from the previous sections, ods, an example from a clinical trial is shown in Table 1. In and give some suggestions.
the table, there are the records of the supine systolic bloodpressures (in mmHg) for 5 patients before and after takingthe drug captopril.
Let (Bj; Fj) denote a baseline/follow-up pair of scores for patient j in the treatment group, j = 1; 2; 1 Personal communication Prof. Johan Bring, E-mail: [email protected] However, it must be emphasized that, in this essay, "statistical power" actually means something slightly di¤erent from this.
Zhang, L. and Han, K. (2009) How to Analyze Change from Baseline: Absolute or Percentage Change? can get absolute change Cj = Bj Fj and percentage change absolute change and percentage change are asymptotic nor- Bj for patient j by calculating from the mally distributed, i.e.
baseline Bj and follow-up Fj scores immediately.
In this example, j is the patients' ID number, and here n = 5. In columns 2 and 3, there are baseline and follow- up scores for each patient. Absolute change and percentage where C and C are the mean and the standard deviation change that calculated from baseline and follow-up scores are P are the mean and the standard deviation of shown in column 4 and 5, respectively.
P , respectively.
In this case, t-test can be used for both absolute change Table 1. Supine systolic blood pressure (in mmHg) for 5 and percentage change. For absolute change, the null hypoth- patients with moderate essential hypertension, immediately esis of t-test is H0 : C = 0. From C before and after taking the drug captopril3 the t-statistic for an absolute change t-test is C is an estimate of the standard deviation of ab- solute change.
Similarly, in the percentage case, the null hypothesis of t-statistic for a percentage change t-test is From Table 1, we see that there is a decreasing e¤ect for the blood pressure of each patient after taking the drug cap- topril. Absolute change and percentage change show the de- crease in di¤erent ways. From a statistical point of view, we where bP is an estimate of the standard deviation of per- should compare the statistical power for the two methods.
centage change.
We have mentioned the relation between statistical power and the absolute value of t-statistic. We know that, when 2.3 Comparison of Statistical Power the signi…cant level is …xed, if the expected absolute value We have mentioned that Vickers (2001) compared the statis- of the t-statistic of absolute change is larger, the statistical tical power of di¤erent methods by doing a simulation. How- power of that will be higher. The opposite is also true, i.e.
ever, his conclusion just based on an ideal simulation proce-dure, and he did not compare the statistical power theoreti-cally. Kaiser (1989) developed a test statistic which comparedthe maximum likelihood of the two methods. It has nothing to do with statistical power. But Kaiser (1989) gave an idea that it is easier to do comparison by constructing a ratio test Statistical P ower of Absolute Change For comparison of the statistical power of the two meth- Statistical P ower of P ercentage Change ods, we construct a ratio test statistic by using the test sta-tistic or p-value of the treatment e¤ect test. Before that, we where E (jtCj) and E (jtP j) are the expected absolute value need to know the distributions of absolute change and per- of the t-statistic of absolute change and percentage change, centage change. That is because, for di¤erent distributions of absolute change or percentage change, di¤erent test methods So, when R > 1, absolute change has higher statisti- will be used. In order to construct a ratio test statistic, we cal power than percentage change, and we choose absolute should know the test statistics used in both numerator and change. If R < 1, the percentage change with the higher denominator of the ratio test statistic.
statistical power is preferred.
In the case of small sample size, it is common to assume that one of the distributions of absolute change and percent- 2.3.1 When t-test is Suitable for both Absolute age change is normal. In some speci…c situation, both ab- Change and Percentage Change solute change and percentage change may be normally dis- When the sample size n of the clinical experiment is large, tributed. Even though for a dataset that is not normally dis- according to the Central Limit Theorem, both the mean of tributed, if the distribution is close to normal distribution or 3 Hand, DJ, Daly, F, Lunn, AD, McConway, KJ and Ostrowski, E (1994): A Handbook of Small Data Sets. London: Chapman and Hall.
Zhang, L. and Han, K. (2009) How to Analyze Change from Baseline: Absolute or Percentage Change? the distribution is symmetric without extreme observations, statistical power than percentage change, and we choose ab- t-test may also be used. In that situation, the test statistic R solute change. For R0 > 1, the percentage change is preferred.
is also applicable.
In this section, another ratio test statistic R0 for non- If we simulate some datasets, by using the ratio test sta- normal distribution situation is discussed. This is the supple- tistic R, we can compare the statistical power of the absolute ment of the normal distribution case. In the following simula- change and percentage change of the datasets that we simu- tion part, we will concentrate more on the normal distribution lated. In contrast with Vickers' (2001) claim, some datasets case shown in subsection 2.3.1, and the details in subsection with R < 1 will be shown, which re‡ects that percentage 2.3.2 will not be discussed any more.
change has higher statistical power than absolute change un-der some conditions. In the simulation section, we will talkmore about the comparison of the statistical power of the twomethods.
3. Rule of Thumb for the Standard 2.3.2 When the t-test is not suitable for At Least One Deviation of Change Scores For the cases when the assumptions for the t-test are notsatis…ed for at least one of the tests, another test should be The standard deviation of the treatment e¤ect is an im- portant parameter that is of interest in the planning of considered. Wilcoxon rank sum test4 is an alternative method studies. The standard deviation of the change scores is the proposed by Wilcoxon (1945). Bonate (2000) mentioned that focus in the second section of this essay. For the case when it is the non-parametric counterpart to the paired samples t-test may be used instead of a non-parametric test, in or- t-test and should be used when normal assumptions are vio- der to calculate the ratio test statistic R, we should work out lated. He suggested that the Wilcoxon rank sum test is always both the mean and the standard deviation of absolute change a better choice when the distribution of the data is unknown and percentage change …rst. It is easy to get these values in or uncertain. Therefore, when the paired samples t-test does case the datasets of the experiment which give the baseline not work, we choose Wilcoxon rank sum test instead.
and follow-up scores for each patient are known. However, in Since we can not use t-statistic to construct the ratio test clinical research, it is not always possible and practical to get statistic any more, we may choose to use the expected p-value the scores for each patient, especially in the planning phase of of the treatment e¤ect test.
a study. If we require some datasets to support our research Similarly to (5), according to what is mentioned in equa- work, we may …nd some experiment datasets interesting for tion (2), we may construct another ratio test statistic R0 by our research from experiments that someone else has done.
taking the ratio of expected p-value, i.e.
One of the good ways to …nd the datasets is searching from published clinical articles. Most of the time, we may …nd some examples in these articles which show us the summary of the baseline scores, the follow-up scores, and their standard devi- ations. And we may get relevant datasets from these tables.
Statistical P ower of Absolute Change However, the scores for each patient in these clinical research Statistical P ower of P ercentage Change articles are seldom published. In this case, how can we know the standard deviation of the change scores? C ) and E (pP ) are the expected p-value of ab- solute change and percentage change, respectively.
There is a rule of thumb which describes the relationship For the cases that t-test still works, E (p between the standard deviation of the change scores and that of the baseline scores.
pected p-value of the t-test for absolute change and percentage change, respectively. When Wilcoxon rank sum test is used instead of t-test, E (p W ilcoxon) and E (pP ) = E (pP W ilcoxon). E (pC W ilcoxon) and E (pP W ilcoxon) arethe expected p-value of the Wilcoxon rank sum test for ab- 3.1 Theoretical Derivation of the Rule of solute change and percentage change, respectively. Therefore, Thumb for Absolute Change we may get three di¤erent alternative forms for the ratio test The general expression for the rule of thumb of absolute statistic R0.
We have talked about that the smaller the expected p- value is, the higher the statistical power will be. When the sig- is …xed, if R0 < 1, absolute change has higher 4 Wilcoxon rank sum test is a non-parametric test for assessing whether two independent samples of observations come from the same distribution.
Zhang, L. and Han, K. (2009) How to Analyze Change from Baseline: Absolute or Percentage Change? where k is a constant that should be determined; SD (C) If we assume that SD (B) = 1, then we get SD (C) = and SD (B) are the standard deviation of absolute change 2r. The smooth curve in Figure 1 shows the relation- and baseline scores, respectively.
ship between the standard deviation of the absolute change The relationship between the standard deviation of ab- SD (C) and the correlation coe¢ cient r. We see that SD (C) solute change scores and that of the baseline scores can be decreases from 1:4 to 0 as the correlation coe¢ cient increases derived from properties of the variance of C, from 0 to 1. When r 6 0:8, SD (C) roughly has a linear de-crease. After that, when the correlation coe¢ cient r tends to1, the ratio decreases quickly to 0.
where r = cov (B; F ) V ar (B) V ar (F ) is the correla- tion coe¢ cient between baseline and follow-up scores.
We assume that V ar (F ) = mV ar (B), where m is the ratio of the variance of follow-up scores to that of baseline scores. In a speci…c case, m is a constant which may be cal-culated from the known dataset. Then, equation (9) can be V ar (B) V ar (F )(10) V ar (B) mV ar (B) Correlation Coefficient r Equivalently, from equation (10), we get the relation equa- Figure 1. Relation curve between the standard deviation of absolute change SD (C) and the correlation coe¢ cient r when SD (B) = 1.
Thus, in equation (8), k = 1 shows that the rule of thumb is determined by the correlation Therefore, when r 0:75, the empirical form of the rule coe¢ cient r and the ratio m.
2 holds. If the correlation When the baseline and follow-up scores have the same coe¢ cient r changes to another value, the form of the rule variance, m = 1, then we get k = 1 of thumb will be also changed. When r tends to 1, a little pression of the rule of thumb becomes change in r may result in a signi…cant change in the standarddeviation of absolute change.
3.3 Rule of Thumb for Percentage Change Therefore, the standard deviation of the absolute change is connected to the standard deviation of the baseline scores Earlier we stated that P = (B via the correlation coe¢ cient r.
centage change is the ratio of absolute change to the baselinescore. Then we get 3.2 The Relation between the Standard Devi-ation of Absolute Change and the Correlation The empirical form of the rule of thumb is shown in expression Since percentage change is a ratio of two variables, its dis- tribution is uncertain. It is hard to derive the expression of (12) shows the relation between the standard deviation of the the standard deviation of percentage change from the stan- absolute change and that of the baseline scores. So, how does dard deviation of baseline scores as we did in equation (10).
the standard deviation of absolute change depend on the cor- We have discussed the rule of thumb for SD (C), and we relation coe¢ cient? see that the standard deviation of percentage change depends Zhang, L. and Han, K. (2009) How to Analyze Change from Baseline: Absolute or Percentage Change? not only on the absolute change C but also on the baseline the patients in the treatment group, the …nal follow-up scores score B. If we …x the value of C, B may also keep on changing F all have an absolute decrease of 5 units from F 0 after the from one sample to another. As a result, it seems there is no medical intervention, while there is no change of the follow-up stable relationship between the standard deviation of percent- scores for patients in the control group, i.e.
age change SD (P ) and the baseline score B. A rule of thumbfor percentage change may, therefore, not be stated. This con- clusion will be proved in the following simulation part.
He changed the correlation coe¢ cient r, and got di¤erent simulation results under di¤erent correlation coe¢ cient. Us-ing these simulation results, he calculated statistical power for each method and made the statistical ine¢ ciency conclusion.
In the following subsections, we will do simulations based on Vickers' (2001) method. But some change and improve- In section 2, we discussed the comparison of the statistical ment will be done to his code.
power of absolute change and percentage change, by con- structing a ratio test statistic based on normal distribution.
In the third section, we discussed the rule of thumb for ab- 4.1.1 A Case that Percentage Change Has Higher Sta- solute change and percentage change theoretically.
This section will do some simulations to show the prob- In Vickers'(2001) simulation method, a …xed absolute change lems that we have discussed in a practical way. The …rst from the simulated follow-up scores F 0 to the …nal follow-up thing we want to prove is, in contrast with Vickers' (2001) scores F was set to each patient in the treatment group. If we conclusion, that percentage change can be statistically e¢ - change the …xed absolute change to a …xed percentage change, cient under some conditions. The second thing that will be maybe we will get something di¤erent. To be more random- proved is the di¢ culty of de…ning a rule of thumb for percent- ized, just like what may happen in practice, we use a random percentage change instead of …xed percentage change. Thepercentage changes P are simulated from a normal distribu- Percentage tion. We should notice that the changes we did to Vickers' Change under Some Conditions (2001) simulation will result in the change of the correlationcoe¢ cient between the baseline and follow-up scores. The cor- Vickers (2001) suggested avoiding using percentage change, relation coe¢ cient of the baseline and follow-up scores in the because of his conclusion that percentage change from base- simulation result is not the r that we used in equation (13) any line is statistically ine¢ cient. He made that conclusion based more, even though the real value of the correlation coe¢ cient on the comparison of statistical power calculated from his may be very close to the value we used in the simulation.
simulation results.
However, since the correlation coe¢ cient will not a¤ect the Vickers (2001) did the simulation in the following way.
comparison of the statistical power for the two methods, we First, he simulated 100 pairs of baseline and follow up scores will give the value of used in each simulation procedure, but for 100 patients. The baseline scores B are simulated from do not talk more about it. What's more, in the following sim- a normal distribution, i.e. B N (50; 10). In order to get ulation, we just concentrate on the patients in the treatment 100 scores B, he simulated 100 B0 …rst, B0 he got B from the equation B = B0 + 50. He also simu- We have developed a ratio test statistic R in section 2, and lated another 100 scores Y , Y N (0; 10), which are de…ned we will use it to do the comparison between absolute change as the post-treatment scores of the control group. Then the and percentage change. The simulation can be divided into follow-up scores F 0 are simulated from B0 and Y by using two steps. In step 1, we simulate 100 pairs of baseline/follow- the equation (13). We should note that F 0 is not the …nal up scores. In the second step, the test statistic R is calculated follow-up scores.
based on the scores we simulated in step 1.
From equation (5), we know that, in order to simulate a dataset such that R < 1, we should let the percentage N (0; 10), we obtain that change have a large mean and small standard deviation. So, N (50; 10). Finally, Vickers (2001) simulated 100 g from in this case, we simulate P from the normal distribution Binomial (1; 0:5) for each patient. These patients who got N (0:5; 0:01). We set r = 0:75 and simulate B from the dis- g = 1 were put into the treatment group, and the other pa- tribution N (200; 20). According to Vickers (2001) simulation tients were put in the control group. So, there are nearly method, we obtain a dataset of scores. Figure 2 shows a part 50 patients in both treatment group and control group. For of the simulation results of baseline and follow-up scores in Zhang, L. and Han, K. (2009) How to Analyze Change from Baseline: Absolute or Percentage Change? step 1. From Figure 2, we see that there is a nearly 50% In order to show a more general result, we repeat the pro- decrease from the baseline score for each patient.
cedure in both step 1 and step 2 100 times, and check thedistribution of R. As shown in Figure 3, the solid line on theleft is the distribution of R based on the datasets we simu-lated. We see that, when P N (0:5; 0:01), the value of R is much less than 1. As a result, in this case, percentage changehas a higher statistical power.
In this simulation, we set the percentage change P nor- mally distributed with a large mean and small standard de-viation. However, it is unreasonable to have such a small standard deviation in practice. If we increase the standarddeviation of P , what kind of result will come to us? Figure 3 shows the distribution of R under di¤erent stan- dard deviation of P . We see that, the value of the test sta- tistic R increases as the standard deviation of P increases.
Although R increases, it is still less than 1. In this case, weprefer percentage change to absolute change.
4.1.2 A Case that Absolute Change Has A LittleHigher Statistical Power B N(200,20), P N(0.5,0.01) In the last section, we simulated a case where percentagechange had a higher statistical power, which is in contrast Figure 2. Change from Baseline Scores to Follow-up Scores with Vickers' (2001) conclusion. If we consider more about the simulation method, we should notice that we used per-centage change to do simulation in that case. It may be afactor which a¤ects the simulation results such that percent-age change has a higher statistical power.
B N(200,20), P N(0.5,SD(P)) Figure 3. Distribution of R (r = 0:75).
B N(200,20), C N(100,5) Since the test statistic R is the ratio of two expected val- Figure 4. Change from Baseline Scores to Follow-up Scores ues, in order to calculate the expected value, we repeat the score simulation procedure in step 1 100 times, and then weget 100 datasets of scores. In the second step, using the 100datasets, we work out the value of R, and check if it is less Now, we just change P N (0:5; 0:01) to C and keep other conditions the same. Part of the baseline and Zhang, L. and Han, K. (2009) How to Analyze Change from Baseline: Absolute or Percentage Change? follow-up scores are shown in Figure 4. It seems similar withthe scores in Figure 2. This is because we set the expectedabsolute change to 100, which is 50% of baseline scores. So,in a similar absolute change case, what kind of result we willget? B N(200,SD(B)), C N(Mean(C),10) Figure 6. Distribution of R (r = 0:75).
We have done 3 simulations based on a modi…cation of Vickers'(2001) method so far. The …rst one shows that per-centage change has higher statistical power. The other two B N(200,20), P N(100,SD(C)) show that percentage has nearly the same statistical power Figure 5. Distribution of R (r = 0:75).
with absolute change.
All of them proved that percent- age change can be statistical e¢ cient under some conditions.
Therefore, Vickers (2001) conclusion is not correct.
Figure 5 shows the distribution of R under di¤erent stan- 4.2 Nonexistence of Rule of Thumb for Per- dard deviation of C. Comparing with the distributions in Figure 3, we …nd it has a di¤erent kind of change when thestandard deviation of C changes. The distributions in Figure We have discussed the rule of thumb for percentage change 3 mainly perform a location di¤erence, while the distributions theoretically in section 3. In this section we will simulate in Figure 5 have di¤erent kurtosis and spread.
another dataset to check if the rule of thumb for percentage Even though the expected value of R in Figure 5 is larger change exists. The simulation will show how the standard de- than 1, it is really close to 1. In this case, it seems both viation of percentage change SD (P ) depends on the baseline absolute change and percentage change can be used. The dif- ference between the statistical powers of the two methods is This simulation is also based on Vickers' (2001) sim- ulation method.
In this case, the baseline scores follows N (50; 10), and the percentage change has a distribution N (0:1; 0:02). Following the simulation steps, we will 4.1.3 Another Case that Percentage Change Has Lit- get a score dataset, and the standard deviations of absolute tle Di¤erence with Absolute Change change and percentage change can be calculated.
After we get the baseline and follow-up scores, we make a In this case, we reduce the standard deviation of the baseline simple transformation that both baseline and follow-up scores scores to 10 and compare the results with that of the previous decrease 5 units, i.e.
From Figure 6, we …nd that the expected value of the ratio test statistic is much more close to 1. If we also reduce themean of C, then R is completely less than 1. This is another After transformation, we get a new dataset of baseline and case that shows, under some conditions, the statistical powers follow-up scores, and calculate the standard deviations of ab- of the two methods are nearly the same.
solute change and percentage change of new scores. Repeat Zhang, L. and Han, K. (2009) How to Analyze Change from Baseline: Absolute or Percentage Change? the simulation procedure 100 times, each time we may get standard deviation of absolute change does not change. Ac- 4 standard deviations, SD (C), SD tually, we can prove that in a theoretical way.
Then, we calculate the mean of 100 simulation results for the 4 standard deviations, respectively. When the correlation co-e¢ cient changes, we get the relation curve between the stan- After the transformation, C does not change. Therefore, dard deviation of change scores and the correlation coe¢ cient the standard deviation of absolute change will not change, both before and after transformation.
From …gure 7, we also see that, the standard deviations of percentage change under di¤erent correlation coe¢ cients be- come larger after transformation. The smaller the correlationcoe¢ cient is, the larger the change of the standard deviation of percentage change will be.
We have mentioned that SD (P ) = SD (C case, C does not change, but B becomes smaller. As a result,the standard deviation of percentage change becomes larger.
This re‡ects that the standard deviation of percentage change Before T ransformation depends on the baseline scores. Therefore, it's di¢ cult to have After T ransformation a rule of thumb for the standard deviation of the percentagechange based on only the standard deviation of the baseline Correlation Coefficient r B N(50,10), P N(0.1,0.02) 5. Demonstration of the Rule of Thumb for Absolute Change We have discussed the rule of thumb for absolute change theoretically in the third section of this essay. A gen- eral expression of the rule of thumb is given in equation (12).
We know that, when r 0:75, the empirical form of the rule of thumb in expression (7) holds. In this section, we will concen- trate on the demonstration of the rule of thumb for absolute change. We collect some real datasets to check whether the Before T ransformationAfter T ransformation rule of thumb works well.
The real datasets are searched from examples of clinical research articles and medical literatures. As shown in Table 2, we have collected two kinds of datasets. The datasets of the …rst 10 cases contain the scores for each patient, while thedata sets of the last 5 cases just contain some data summary, e.g. mean, standard deviation, etc., of the baseline scores and Correlation Coefficient r absolute change scores.
B N(50,10), P N(0.1,0.02) If we know the scores for each patient, we can calculate Figure 7. Relation curve between the standard deviation of not only the standard deviations but also the correlation co- absolute change (above) or percentage change (below) and e¢ cient between the baseline scores and the follow-up scores.
the correlation coe¢ cient r before and after transformation.
So, for the …rst 10 cases, we also get the value for the general From …gure 7, we observe that, after transformation, the form of the rule of thumb.
Zhang, L. and Han, K. (2009) How to Analyze Change from Baseline: Absolute or Percentage Change? Table 2. Comparison of the real standard deviation and the value got from the rule of thumb for absolute change Comparing the values of SD (C) and SD (B) In this case, SD (C) = 2:68 and are obvious di¤erences between the two values for these real 2rSD (B) = 2:66, the two values are nearly the same, datasets. In some cases, the di¤erence between SD (C) and which re‡ect that the rule of thumb are also a¤ected by the 2 is very large.
For the cases that has a correlation coe¢ cient between 0.6 If we also take m into account, we will get the real value of and 0.9, which is close to 0.75, the di¤erence between SD (C) the standard deviation of absolute change. Actually, we have 2 may be acceptable. For example, in case 1, proved that in equation (11) of section 3.
r = 0:63, it is close to 0.75. In this case, SD (C) = 8:99 and When we know nothing about the correlation between the 2 = 8:09 , it seems that the two values are close baseline and follow-up scores, just like the last …ve cases in to each other. However, in case 5, when r = 0:17, the value Table 2, the rule of thumb may not be suitable. In this case, of SD (C) is nearly three times of the value of SD (B) we should be more careful.
This is not acceptable. These facts show that the rule of From the analysis based on real datasets in this section, thumb in expression (7) is valid when r 0:75 or when r is we learned that when the ratio m = V ar (F ) close to that value.
to 1 and the correlation coe¢ cient is nearly 0.75, the rule of If we take the correlation coe¢ cient r into account, by 2 will be practical. If these con- comparing the values of SD (C) and ditions are not satis…ed, it is not a good rule to follow. If we …nd that the values of 2rSD (B) are closer to SD (C) ignore the two conditions and insist on using the rule, as we 2, especially when m know from Table 2, it may result in a big mistake.
5 Douglas G.Alman (1991): Practical Statistics for Medical Research. London: Chapman and Hall. Page 4756 Douglas G.Alman (1991): Practical Statistics for Medical Research. London: Chapman and Hall. Page 4757 Pagano M, Gauvreau K (2000): Principles of Biostatistics, Second Edition, Duxbury. Table B.158 Pagano M, Gauvreau K (2000): Principles of Biostatistics, Second Edition, Duxbury. Table B.159 Bradstreet, T.E. (1994) "Favorite Data Sets from Early Phases of Drug Research - Part 3." Proceedings of the Section on Statistical Education of the American Statistical Association. <http://www.math.iup.edu/ tshort/Bradstreet/part3/part3-table3.html> 2009-06-08 1 0 Hand, DJ, Daly, F, Lunn, AD, McConway, KJ and Ostrowski, E (1994): A Handbook of Small Data Sets. London: Chapman and Hall.
1 1 Ryan, Joiner, Cryer (1985): Minitab Handbook, Second Edition. PWS-KENT Publishing Company. Page 318, Pulse Data1 2 Bonate P (2000): Analysis of Pretest-Posttest Design. Boca Raton: Chapman and Hall/CRC. Table 3.11 3 Bonate P (2000): Analysis of Pretest-Posttest Design. Boca Raton: Chapman and Hall/CRC. Table 3.41 4 Bonate P (2000): Analysis of Pretest-Posttest Design. Boca Raton: Chapman and Hall/CRC. Table 9.11 5 Waleekhachonloet O, Limwattananon C, Limwattananon S, Gross C (2007): Group behavior therapy versus individual behavior therapy for healthy dieting and weight control management in overweight and obese women living in rural community. Obesity Research & Clinical Practice,1: 223-232. Table 3 1 6 Neovius M, Rössner S (2007): Results from a randomized controlled trial comparing two low-calorie diet formulae. Obesity Research & Clinical Practice, 1: 165-171. Table 1 & Table 2 1 7 Neovius M, Rössner S (2007): Results from a randomized controlled trial comparing two low-calorie diet formulae. Obesity Research & Clinical Practice, 1: 165-171. Table 1 & Table 2 1 8 Neovius M, Rössner S (2007): Results from a randomized controlled trial comparing two low-calorie diet formulae. Obesity Research & Clinical Practice, 1: 165-171. Table 1 & Table 2 1 9 Neovius M, Rössner S (2007): Results from a randomized controlled trial comparing two low-calorie diet formulae. Obesity Research & Clinical Practice, 1: 165-171. Table 1 & Table 2 Zhang, L. and Han, K. (2009) How to Analyze Change from Baseline: Absolute or Percentage Change? in more detail. It has higher statistical power than the twomethods we talked about. But for the people who are not 6. Discussion and Conclusions statisticians, this method can not be understood as easily asabsolute change or percentage change.
In this essay we compared the use of absolute change and From a clinical point of view, clinicians may prefer to percentage change. According to the de…nition of statis- choose the method that will show the health-improvement tical power, we developed a ratio test statistic R under cer- more obviously. Some researchers may choose the method tain distribution assumption, which can help us decide which that may be understood by most people that are interested method will be used, absolute change or percentage change.
in his research. Sometimes, we may make a choice just based When R > 1, absolute change has a higher statistical power.
on some empirical information. Most of the time, the choice In that case, we prefer absolute change to percentage change.
depends more on the research work and the researcher's own If R < 1, we choose percentage change.
Based on Vickers'(2001) simulation method, with the help of the ratio test statistic, we did some simulations to com-pare the statistical power of the two methods. In contrastwith Vickers' (2001) conclusion that the percentage change is statistical ine¢ cient, we simulated some datasets in whichpercentage change has higher statistical power, or has nearlythe same statistical power with absolute change. In this way, Bonate P. (2000): Analysis of Pretest-Posttest Design.
we showed that percentage can be statistical e¢ cient under Boca Raton: Chapman and Hall/CRC.
some conditions.
Kaiser L. (1989): Adjusting for baseline: change or Another issue often concerned by researchers is the stan- percentage change? Statistics in Medicine, 10: 1183- dard deviation of change scores. There is a rule of thumb that may help us get the standard deviation of change scoresquickly from the standard deviation of the baseline scores.
Kim M. et al. (2009): Comparison of epicardial, ab- The general form of the rule of thumb for absolute change can dominal and regional fat compartments in response to be derived in a theoretical way. From the derivation in section weight loss.
Nutr Metab Cardiovasc Dis, 1:7.
3, we know that, when the ratio m = V ar (F ) 0:75, the empirical form of the rule of thumb 2 holds. We also checked these con- avange L., Engels J. and Accurso F. (2007): An- alyzing percent change in cystic …brosis clinical trials.
ditions in a practical way by collecting some real data to The 21st Annual North American Cystic Fibrosis Con- compare the real value and the value got from the rule of ference, Anaheim, California.
thumb. And we got the same conclusion. So, if we knownothing about the correlation between baseline and follow-up Neovius M. and Rössner S. (2007): Results from a scores, we should be more careful to use the rule.
randomized controlled trial comparing two low-calorie For percentage change, a rule of thumb does not exist.
diet formulae. Obesity Research & Clinical Practice, 1: That is because the standard deviation of percentage change depends on the baseline scores, and it is very hazardous tostate a rule. We also proved this by doing a simulation with Törnqvist L., Vartia P. and Vartia Y. (1985): How a simple transformation. The simulation result showed how should relative changes be measured? American Statis- the standard deviation of percentage change depends on the tician, 39: 43-46.
baseline scores.
Vickers A. (2001): The use of percentage change from In this essay, we didn't give any rules to make a choice baseline as an outcome in a controlled trial is statis- between absolute change and percentage change. We devel- tically ine¢ cient: a simulation study.
oped a ratio test statistic which may be helpful, but it is not a Research Methodology, 1:6.
good rule to tell us how to make a choice. That is because, inthe ratio test statistic, expected value should be used. For a speci…c dataset in practice, we can not calculate the expected Limwattananon S. and Gross C. (2007): value. That is the limitation of the test statistic.
behavior therapy versus individual behavior therapy Actually, there is not a most optimal method to tell us for healthy dieting and weight control management in which method to choose. From a statistical point of view, overweight and obese women living in rural community.
we would like to choose the method with higher statistical Obesity Research & Clinical Practice, 1: 223-232.
power. Beside the two change measurement methods from Wilcoxon F. (1945): Individual comparisons by rank- baseline, there are also some other methods. One of them ing methods, Biometrics Bull., 1, 80.
is analysis of covariance, which is mentioned by both Vick-ers (2001) and Kaiser (1989). Bonate (2000) discussed thisZhang, L. and Han, K. (2009) How to Analyze Change from Baseline: Absolute or Percentage Change? #correlation coefficient #percentage change method="stack",main="Change from Baseline to Follow-up")for(i in (1:length(b))){ }mtext(side=1,line=3, "B N(200,20), P N(0.5,0.01)") 2 0 Responsible Programmer: Ling Zhang.
Zhang, L. and Han, K. (2009) How to Analyze Change from Baseline: Absolute or Percentage Change? #correlation coefficient f<-b*r+y*(1-r 2) 0.5+200h<-rnorm(n,0.5,s) #percentage change #absolute value of tc, i.e.
#absolute value of tc, i.e.
#expected value of jtcj #expected value of jtpj #value of the ratio test statistic R Zhang, L. and Han, K. (2009) How to Analyze Change from Baseline: Absolute or Percentage Change? #correlation coefficient f<-b*r+y*(1-r 2) 0.5+200h<-rnorm(n,100,5) method="stack",main="Change from Baseline to Follow-up") for(i in (1:length(b))){ }mtext(side=1,line=3, "B N(200,20), C N(100,5)") #correlation coefficient f<-b*r+y*(1-r 2) 0.5+200h<-rnorm(n,100,s) #absolute value of tc, i.e.
#absolute value of tp, i.e.
#expected value of jtcj #expected value of jtpj #value of the ratio test statistic R Zhang, L. and Han, K. (2009) How to Analyze Change from Baseline: Absolute or Percentage Change? for(muc in c(100,50)){ #correlation coefficient f<-b*r+y*(1-r 2) 0.5+200h<-rnorm(n,muc,10) #absolute value of tc, i.e.
#absolute value of tp, i.e.
#expected value of jtcj #expected value of jtpj #value of the ratio test statistic R Zhang, L. and Han, K. (2009) How to Analyze Change from Baseline: Absolute or Percentage Change? xlab="B N(200,SD(B)), C N(Mean(C),10)",ylab="Density",main="Distribution of R") #correlation coefficient #percentage change #follow-up score before transformation #follow-up score after transformation #baseline score before transformation #baseline score after transformation Zhang, L. and Han, K. (2009) How to Analyze Change from Baseline: Absolute or Percentage Change? mtext(side=1,line=4, "B N(50,10), P N(0.1,0.02)")for(i in 1:21){ mtext(side=1,line=4, "B N(50,10), P N(0.1,0.02)") Zhang, L. and Han, K. (2009)

Source: http://www.statistics.du.se/essays/D09_Zhang%20Ling%20&%20Han%20Kun.pdf