One of the main aims in planning a clinical trial is to provide 'conclusive evidence of efficacy' (Thrushfield 2007). However, as field vets and scientists we should also recognise that a procedure may be deemed efficacious but relatively ineffective if the outcome is not generally applicable to the target population or is not widely adopted outside the confines of the trial environment. Aspects which commonly affect this outcome are; timing of the trial and trial design, animal selection and product selection. The main aims of a trial should consider which outcomes are likely to be the most clinically and economically important. There are many different ways of conducting a trial and much of the decision making in a practical environment will be influenced by a few factors; the main aims, the availability of animals and the circumstances of the trial.
The various aspects for consideration in planning a field trial are many and can become very involved and are well beyond the 'tips and tools' approach of this paper. The main concepts to consider are summarised in the table below (modified from Thrushfield 2007).
|GENERAL INFORMATION||JUSTIFICATION AND OBJECTIVES|
|Title/type of trial
Identity and appropriateness of trail site
Source and level of funding available
Animal ethics approval
|Reason for conducting the trial
Primary hypothesis to be tested
Primary end point
|EXPERIMENTAL POPULATION||THERAPEUTIC PROCEDURE|
|Sample size determination
Composition (sex, breed etc)
Definition of case
Identification of case
Selection of controls
Product (combination vs single product)
Method of administration (oral vs injection)
Animal and operator safety
Level of measurement
Definition of efficacy
|Start and finish dates/timeline
Duration of disease/effect under study
Period of recruitment of cases
Duration of treatment
Decision rules for termination of trial/removal of animals
|TYPE OF TRIAL||
Stratification of variables
Implementation of allocation process
|Data to be collected
Frequency of data collection
Method for recording/identifying adverse reactions
Identification of experimental units
Standardisation of data collection and recording
For the purposes of this presentation and limiting discussion to a reasonable length, we will focus on a couple of electronic tools to assist in selection of animals for inclusion and determining sample size. The B12 trial design will be used as an example.
Determining the sample size for a treatment or control group will be influenced by the animals and resources available. Commonly the ideal may not be achievable. However, understanding what factors are central to achieving the primary aims of the trial and how the selection of animals may influence the validity/interpretation/extension of the final outcomes is important to avoid disappointment and wastage of time and resources. Careful planning, including the consultation of an epidemiologist or statistician is often important.
The following are sample size estimation tools that are readily available on the web. The approach will differ depending on whether the aim is to investigate a difference in proportions – eg disease prevalence in groups of animals under different conditions (eg vaccinated/non vaccinated stock) or if a continuous variable is the main measured outcome, for example determining the difference in growth rate of lambs receiving supplementation vs no supplementation.
The links to the sites are provided below and examples will be worked through in the presentation.
To compare proportions – eg comparing disease prevalence or mortality rate in two comparable populations of animals; www.stat.ubc.ca (compare proportions).
To compare two means - eg mean difference in growth rate between treatment and control groups (comparing continuous variables ie measured over time) epitools.ausvet.com.au
Both calculators allow adjustments of power and calculate sample size
The main AusVet site has many other epidemiological tools available; epitools.ausvet.com.au
Another good site with many tools is 'Epicentre', Massey University; epicentre.massey.ac.nz
The default setting/accepted level of power is 0.8 or 80%. Power will be increased by reducing variability in measured outcomes and/or increasing numbers. If a trial has low power, even though a difference in the treatment outcomes may be real, it may not reach the 95% level of significance required to 'prove' that a true difference exists. If it is anticipated that there will be a wide variation in outcomes measured (eg rate of live weight gain) and it is not possible to allocate animals to treatment groups at the start of the trial to reduce this effect, then larger numbers per groups will be required to demonstrate a statistically significant outcome.
Other considerations may be required in selection of animals and identification/classification of experimental units. The experimental unit is defined as the smallest independent unit to which a treatment is randomly allocated. If animals are being selected to be included in a treatment group, a number of methods/techniques may be applied to control for certain influential variables eg sex, age, live weight etc. Stratification is commonly employed. This also allows for 'matching' between treatment and control groups. In other trial designs where animals are all comparable (eg Merino females 12-14 months old), random allocation, using generation of random numbers and sequential allocation is appropriate. This is based on the calculation of sample size ie treatment group size.
A B12 response trial is planned for late 2011 early 2012 in the Hume board. We have a reliable co-operator and access to a mob of approximately 600 lambs. The lambs selected for the trial will be drafted based on live weight to reduce the variability in the treatment groups at the start of the trial. Lambs in these groups will be further drafted into male/female and lambs will be stratified based on sex and body weight to create treatment and control groups with the same number of sheep, matched for sex and matched for weight (or with the same range of variability in start weight). Lambs will be allocated to treatments and controls to produce 4 groups; two treatments (lighter and heavier starting weight) and two control groups (lighter and heavier starting weight). This will facilitate weight change comparisons between treatments and controls for each live weight group. We are able to include up to 150 lambs in each treatment group.
These larger group sizes may facilitate examination of data in an attempt to show a significant relationship between supplementation with B12 and a greater rate of live weight gain, even if only a small growth rate response to treatment (eg 0.6kg) is measured over the timeframe of the study. The higher numbers of animals entering the trial on day one will also allow for data to be 'lost' during the trial if animals are removed as they reach a saleable weight. This possibility has been considered and compensated for to reduce the impact on decision making at a commercial level for the duration of the trial. For the final analyses of data it is important to demonstrate that there is a cost benefit of the treatment; that is the additional weight gain achieved in the treatment group renders the use of the supplement economically sound option. If this is not the case the outcome may be deemed ineffective with reduced uptake or extension to the broader farming community. In this case, the resources allocated to a trial with large sample size may be better applied to duplication of the trial to increase validity of outcomes or to allow the trial to proceed under a more severe set of constraints.
EXAMPLE 1: SAMPLE SIZE TO DETECT A SIGNIFICANT DIFFERENCE BETWEEN 2 MEANS WITH EQUAL SAMPLE SIZES AND VARIANCES
|Input Values||Mean in population 1:||This utility calculates the
sample size required to detect a statistically significant difference between
two sample means with specified levels of confidence and power, assuming
equal sample sizes and equal variances.
Inputs are the assumed true values for the two means, the desired level of confidence and the desired power for the detection of a significant difference and the estimated combined variance. By default, calculations assume that a two-tailed statistical test will be used ie the mean of one population can be smaller or larger than the other. However a one-tailed test can also be specified if preferred ie the hypothesis is that the mean of one population can either be smaller or larger than the other, but not both.
The program outputs the sample size required to detect the specified difference with desired power and confidence, for a one-tailed or two-tailed statistical test, as specified.
|Mean in population 2:|
|Use 1 or 2-tailed test:||1 tailed|
EXAMPLE 2; SAMPLE SIZE TO DETECT A SIGNIFICANT DIFFERENCE BETWEEN TWO MEANS (eg mean difference in weight gain between treatment and control groups)
|Sample size (per group):||79|
|Total sample size (both groups):||158|
Interpretation: To detect a significant difference of 1 kg body weight between treatment and control groups, with a confidence of 95%, each group of sheep should have at least 79 head at the end of the trial period.