CASE NOTES
TIPS AND TOOLS FOR DECISION MAKING AND CLINICAL / FIELD TRIAL PLANNING—SAMPLE SIZE
Helen McGregor, DV Hume LHPA
Posted Flock & Herd March 2011
Introduction
One of the main aims in planning a clinical trial is to provide 'conclusive evidence of efficacy' (Thrushfield 2007). However, as field vets and scientists we should also recognise that a procedure may be deemed efficacious but relatively ineffective if the outcome is not generally applicable to the target population or is not widely adopted outside the confines of the trial environment. Aspects which commonly affect this outcome are; timing of the trial and trial design, animal selection and product selection. The main aims of a trial should consider which outcomes are likely to be the most clinically and economically important. There are many different ways of conducting a trial and much of the decision making in a practical environment will be influenced by a few factors; the main aims, the availability of animals and the circumstances of the trial.
The various aspects for consideration in planning a field trial are many and can become very involved and are well beyond the 'tips and tools' approach of this paper. The main concepts to consider are summarised in the table below (modified from Thrushfield 2007).
| GENERAL INFORMATION | JUSTIFICATION AND OBJECTIVES |
| Title/type of trial Contributors Identity and appropriateness of trail site Source and level of funding available Animal ethics approval |
Reason for conducting the trial Primary hypothesis to be tested Primary end point Secondary hypotheses |
| EXPERIMENTAL POPULATION | THERAPEUTIC PROCEDURE |
| Sample size determination Experimental unit Composition (sex, breedetc.) Inclusion/exclusion criteria Withdrawal criteria Definition of case Identification of case Selection of controls |
Dosage Product (combination vs single product) Placebo/control Method of administration (oral vs injection) Animal and operator safety Blinding Compliance monitoring |
| DESIGN | DURATION |
| Response variables; Level of measurement Scoring system Definition of efficacy |
Start and finish dates/timeline Duration of disease/effect under study Period of recruitment of cases Duration of treatment Drug withholding Decision rules for termination of trial/removal of animals |
| TYPE OF TRIAL | DATA COLLECTION |
| Randomisation Stratification of variables Implementation of allocation process |
Data to be collected Frequency of data collection Method for recording/identifying adverse reactions Identification of experimental units Standardisation of data collection and recording Confidentiality |
For the purposes of this presentation and limiting discussion to a reasonable length, we will focus on a couple of electronic tools to assist in selection of animals for inclusion and determining sample size. The B12 trial design will be used as an example.
HOW DO I DETERMINE SAMPLE SIZE?
Determining the sample size for a treatment or control group will be influenced by the animals and resources available. Commonly the ideal may not be achievable. However, understanding what factors are central to achieving the primary aims of the trial and how the selection of animals may influence the validity/interpretation/extension of the final outcomes is important to avoid disappointment and wastage of time and resources. Careful planning, including the consultation of an epidemiologist or statistician is often important.
The following are sample size estimation tools that are readily available on the web. The approach will differ depending on whether the aim is to investigate a difference in proportions –e.g.disease prevalence in groups of animals under different conditions (eg vaccinated/non vaccinated stock) or if a continuous variable is the main measured outcome, for example determining the difference in growth rate of lambs receiving supplementation vs no supplementation.
The links to the sites are provided below and examples will be worked through in the presentation.
To compare proportions –e.g.comparing disease prevalence or mortality rate in two comparable populations of animals; www.stat.ubc.ca (compare proportions).
To compare two means -e.g.mean difference in growth rate between treatment and control groups (comparing continuous variablesi.e. measured over time) epitools.ausvet.com.au
Both calculators allow adjustments of power and calculate sample size
The main AusVet site has many other epidemiological tools available; epitools.ausvet.com.au
Another good site with many tools is 'Epicentre', Massey University; epicentre.massey.ac.nz
The default setting/accepted level of power is 0.8 or 80%. Power will be increased by reducing variability in measured outcomes and/or increasing numbers. If a trial has low power, even though a difference in the treatment outcomes may be real, it may not reach the 95% level of significance required to 'prove' that a true difference exists. If it is anticipated that there will be a wide variation in outcomes measured (eg rate of live weight gain) and it is not possible to allocate animals to treatment groups at the start of the trial to reduce this effect, then larger numbers per groups will be required to demonstrate a statistically significant outcome.
Other considerations may be required in selection of animals and identification/classification of experimental units. The experimental unit is defined as the smallest independent unit to which a treatment is randomly allocated. If animals are being selected to be included in a treatment group, a number of methods/techniques may be applied to control for certain influential variablese.g.sex, age, live weightetc. Stratification is commonly employed. This also allows for 'matching' between treatment and control groups. In other trial designs where animals are all comparable (eg Merino females 12-14 months old), random allocation, using generation of random numbers and sequential allocation is appropriate. This is based on the calculation of sample sizei.e. treatment group size.
OUTLINE OF SAMPLE SIZE DESISION MAKING FOR A PROPOSED B12 RESPONSE TRIAL IN THE HUME LHPA
A B12 response trial is planned for late 2011 early 2012 in the Hume board. We have a reliable co-operator and access to a mob of approximately 600 lambs. The lambs selected for the trial will be drafted based on live weight to reduce the variability in the treatment groups at the start of the trial. Lambs in these groups will be further drafted into male/female and lambs will be stratified based on sex and body weight to create treatment and control groups with the same number of sheep, matched for sex and matched for weight (or with the same range of variability in start weight). Lambs will be allocated to treatments and controls to produce 4 groups; two treatments (lighter and heavier starting weight) and two control groups (lighter and heavier starting weight). This will facilitate weight change comparisons between treatments and controls for each live weight group. We are able to include up to 150 lambs in each treatment group.
These larger group sizes may facilitate examination of data in an attempt to show a significant relationship between supplementation with B12 and a greater rate of live weight gain, even if only a small growth rate response to treatment (eg 0.6kg) is measured over the timeframe of the study. The higher numbers of animals entering the trial on day one will also allow for data to be 'lost' during the trial if animals are removed as they reach a saleable weight. This possibility has been considered and compensated for to reduce the impact on decision making at a commercial level for the duration of the trial. For the final analyses of data it is important to demonstrate that there is a cost benefit of the treatment; that is the additional weight gain achieved in the treatment group renders the use of the supplement economically sound option. If this is not the case the outcome may be deemed ineffective with reduced uptake or extension to the broader farming community. In this case, the resources allocated to a trial with large sample size may be better applied to duplication of the trial to increase validity of outcomes or to allow the trial to proceed under a more severe set of constraints.
EXAMPLE 1: SAMPLE SIZE TO DETECT A SIGNIFICANT DIFFERENCE BETWEEN 2 MEANS WITH EQUAL SAMPLE SIZES AND VARIANCES
Input values
| Input Values | ||||||||
|---|---|---|---|---|---|---|---|---|
| Mean in population 1: | This utility calculates the sample size required to detect a statistically
significant difference between two sample means with specified levels of confidence and power, assuming
equal sample sizes and equal variances. Inputs are the assumed true values for the two means, the desired level of confidence and the desired power for the detection of a significant difference and the estimated combined variance. By default, calculations assume that a two-tailed statistical test will be usedi.e. the mean of one population can be smaller or larger than the other. However a one-tailed test can also be specified if preferredi.e. the hypothesis is that the mean of one population can either be smaller or larger than the other, but not both. The program outputs the sample size required to detect the specified difference with desired power and confidence, for a one-tailed or two-tailed statistical test, as specified. |
|||||||
| Mean in population 2: | ||||||||
| Variance: | ||||||||
| Confidence level: | ||||||||
| Power: | ||||||||
| Use 1 or 2-tailed test: |  1 tailed |
|||||||
 2 tailed |
||||||||
 |
||||||||
EXAMPLE 2; SAMPLE SIZE TO DETECT A SIGNIFICANT DIFFERENCE BETWEEN TWO MEANS (eg mean difference in weight gain between treatment and control groups)
Inputs:
| Mean 1 | 49 |
| Mean 2 | 50 |
| Variance | 5 |
| Confidence level | 0.95 |
| Power | 0.8 |
| Tails | 2 |
Results:
| Sample size | |
|---|---|
| Sample size (per group): | 79 |
| Total sample size (both groups): | 158 |
Interpretation: To detect a significant difference of 1 kg body weight between treatment and control groups, with a confidence of 95%, each group of sheep should have at least 79 head at the end of the trial period.
References
- Thrusfield, M. (2007). Veterinary epidemiology (3 ed.). United Kingdom: Blackwell publishing
- epitools.ausvet.com.au