Introduction
Critiquing skill is necessary to determine a study design to answer a research question. Cohort study design and randomized controlled design are the most popular designs many researchers employ. This paper seeks to compare the features and principles of these study designs with respect to two study examples that employed either of the mentioned study design. This paper will compare the features and principles of the cohort study design against those of randomized controlled trial design used in two examples of studies that employed one of the two designs.
A Cohort Study to Evaluate the Relationship between Leg-inequality and Knee Osteoarthritis by Harvey et al. (2010)
Harvey et al. (2010) used radiography data to evaluate inequality in leg length of 3026 adult participants. A 30-months follow-up revealed a relationship between leg length inequalities of 1 cm or more and clinical variables of the progressive and symptomatic knee osteoarthritis, including prevalence and incidence. In addition, the findings of the study associated the variables more strongly in the shorter leg.
This study had two objectives. One objective was to portray the association between leg-length inequality radiographs and widespread knee osteoarthritis. The other objective was to evaluate the association between inequality in leg length, and occurrence and deteriorating knee osteoarthritis. With these objectives, the researchers expected to establish whether or not a more relevant clinical cut-off value of leg-inequality.
The researchers performed this study in Multicenter Osteoarthritis Study (MOST), which is a community-based study of 3026 subjects of age prone to developing knee osteoarthritis or who already have knee osteoarthritis. These subjects, who were aged between 50 and 79 years old, included those who manifested risk factors for knee osteoarthritis, such as knee surgery, knee injury, obesity, or knee pain.
The setting of this study was Multicenter Osteoarthritis Study (MOST). This facility was specialized in characterizing predictors for knee osteoarthritis worsening and occurrence (Harvey et al. 2010). Therefore, the data used for this study must be valid and reliable because of the radiographic machine and professions employed in the facility.
Designers of cohort studies often require determining the risk characteristics they will investigate long before outcome(s) emerge (Schoenbach 2011). In a cohort study, the sampling of subjects may be tied to exposure. The study may start with all exposed or unexposed subjects or both. The subjects must be free of the outcome of interest at the beginning of the study and are followed up over a specific period to evaluate the incidence of the outcome of interest. Exposures can have broad characteristics, including sex or age; an intervention; predictors, like alcohol use or cigarette smoking. A sampling of participants depending on exposure and evaluation of outcome in the follow-up is the most prominent characteristic of cohort study design (Vandenbroucke, von Elm and Altman 163).
In the study by Harvey et al. (2010), the researchers relied on the demographic information, radiographs of both complete limb and knee, collected during the parent study. Harvey and his colleagues then reassessed the subjects over 30 months. They did this with regular visits. Prior to conducting the assessment, the researchers obtained written informed consent as the institutional review boards required.
Cohort studies may involve healthy subjects or may begin by sampling subjects with disease (Schoenbach, 2011). In this regard, the study of the relationship between leg-length inequality and occurrence and progressive knee osteoarthritis involves both healthy and diseased subjects with relevance to knee osteoarthritis. Therefore, Harvey and colleagues (2010) have incorporated this feature of a cohort study design in their research.
This study experienced a high adherence to study protocols as 89. 6% of subjects underwent complete 30-month follow-up visits (Harvey et al. 2010). Thus, the study results are trustworthy. Bias could not affect the observation of the subjects, unlike the study by Bennell et al. (2005) to evaluate the effectiveness of physiotherapy on knee osteoarthritis discussed below.
The exposure variable Harvey et al (2010) used for the study was leg-length inequality. They calculated leg-length inequality from the complete limb radiographs used at baseline. In addition, the researchers adopted a leg-specific definition, facilitating a knee-based analysis to establish which of the two unequal legs had a high risk of developing knee osteoarthritis. At first, the researcher standardized leg-length inequality to mean a difference of 1cm or more between the subject’s left and right legs. However, the researchers performed further analysis using a cut-off value for leg-inequality of 2 cm or more. Harvey et al. (2010) analyzed the data to determine whether there was a threshold measurement above which the incidence of knee osteoarthritis was highest.
The results of this research indicated two main conditions; Progressive knee osteoarthritis and prevalent knee osteoarthritis. Definitions of Knee Osteoarthritis fell under the presence of Kellgren and Lawrence grade 2 and higher or new complete replacement of the knee.
Progressive knee osteoarthritis meant an advance in narrowing score of joint-space from baseline relative to 30-month follow-up, or a new whole knee replacement unreported at baseline. This outcome definition was appropriate because the progression of Knee Osteoarthritis based on Kollegren and Lawrence’s score is insensitive to obvious models of progression. This helps eliminate bias attributed to such models of progression, thereby increasing the trustworthiness of the results of this study further.
Harvey et al (2010) calculated the means for subjects’ characteristics. The researchers measured the relationship between leg-length inequality and the clinical variables of knee osteoarthritis using logistic regression to explain the correlation between the left knee and right knee of a subject.
Harvey and his colleagues adjusted for potential confounders, such as alignment, BMI, body height, sex, and age, using baseline visit data. Alignment was measured in degrees as a continuous variable. This meant that alignment could take an infinite number of various values; for example 1, or 1. 8, or 3. 938493873 (Howell, 2010) and so forth. The researchers explored the relationship between the alignment and the outcomes, in their statistical models.
The researchers checked the rate of attrition to follow-up by leg-length inequality to establish whether leg-length inequality group biased participation. The differential loss to follow-up was not significant, which helps ascertain the validity and reliability of the results of the study.
Out of the 3026 subjects enrolled for the study at baseline, 2964 subjects did not have missing data for the value of leg-length inequality. The dismal quality of full-limb radiographs was responsible for missing data. Harvey et al (2010) observed a relationship between leg-length inequality and prevalent knee osteoarthritis at baseline. These researchers controlled for confounding factors allowing them to identify the mentioned relationship without any difficulty. The results of the analysis showed that a higher value of leg-length inequality was associated with high odds of having prevalent knee osteoarthritis (KO) in the shorter leg.
Leg-length inequality, in all possible value ranges, did not enlarge the odds of incident knee osteoarthritis (radiographic) in the shorter leg. However, the inequality of at least 1 cm enlarged the odds of incident symptomatic osteoarthritis of the knee in the inferior leg. In addition, the inequality was associated with an increased risk of incident symptomatic osteoarthritis of the knee.
Inequality of the leg length of at least 1 cm was linked with higher odds of progression in the inferior leg over the study period than in inequality of smaller value. The relationship in the superior leg of leg-inequality and progression of the osteoarthritis of the knee was insignificant. Moreover, inequality in the length of the leg of 2 cm or more was not associated with increased odds of progressive knee osteoarthritis in either leg. Nevertheless, of the 26 knees that indicated leg-length inequality of 2 cm or more, just 6 indicated progressive knee osteoarthritis.
This study is a longitudinal analysis of the relationship of inequality in leg length and radiographs of knee osteoarthritis in a large cohort (Harvey et al. 2010). A longitudinal evaluation, like a cohort study, targets a specific subpopulation. However, a cohort study focuses on a common trait whereas a longitudinal targets a specific geographic area or community. These slight differences influence the extent to which the results of the study could be generalized to other regions of a country and the level to which professionals can make inferences. Furthermore, the greater breadth and depth of data collected in cohort studies strengthen their analysis options (National Research Council, 2002), as is the case in Harvey’s et al study.
The researchers employed various analysis options to explain various aspects of the outcome and exposure of interest. For instance, researchers performed the analysis for leg-inequality of less than 1 cm, 1 cm or more, and 2 cm or more, which yielded different inferences for prevalent or incident radiographic or symptomatic osteoarthritis of the knee. Although Harvey et al. (2010) argue that their study is a longitudinal evaluation of the exposure against the outcome of interest; the study seems a cohort study. This is because the people enrolled in MOST may not necessarily be residents of the area where it is situated and may instead hail from various regions across the country and perhaps other regions of the world. Thus, this perception helps validate the results of the study, as the subjects may be representatives of the population of different areas.
Various factors supported the trustworthiness of the study on the relationship between leg-inequality and knee osteoarthritis. High accuracy and precision of radiographic data enhance validity to the results and enable analysis with a smaller cut-off measure for inequality on leg-length (Harvey et al. 2010). Controlling for alignment added to the validity of the findings since misalignment that is prominent in one leg could cause leg-length inequality.
A confounding factor is one that does not mediate the effect of exposure but which may muffle its effects, and blur observation. For any factor to produce confounding, the relationship between the confounder and the exposure must be in the study cohorts. This relationship exists among the participants that comprise the cohort. Therefore, researchers can determine the exposure-confounder relationship from the data (Rothman, Greenland, & Lash, 2008) to be able to control for the confounding factors, such as misalignment as in the study by Harvey and colleagues.
Although the results of the knee-based analysis affirmed that the exposure poses greater risks of knee osteoarthritis to the shorter leg, the analysis of the risks the longer leg posed were inconsistent. Despite further analysis with logistic regression, Harvey et al (2010) could not affirm the conclusion as the statistical tool was deficient.
A Randomized Control Trial to determine the efficacy of physiotherapy on Knee Osteoarthritis by Bennell et al. (2005).
The major objective of the study by Bennell et al. (2005) was to test the efficacy of physiotherapy on the consequences of the symptoms of knee osteoarthritis, such as pain and disability. The research had another objective, establishing whether the self-management strategies had the capacity to maintain any of the physiotherapy benefits.
The researchers set to conduct the study with the hypothesis that physiotherapy provides a positive effect on symptoms of knee osteoarthritis compared to the absence of physiotherapy represented by placebo treatment. The hypothesis of this study was one-tailed. In essence, a randomized controlled trial is a quantitative, controlled experiment that evaluates two or more interventions administered to groups of people assigned, randomly, to receive each intervention, and compares the outcomes between the two groups (Stolberg et al. 2004).
The setting of the study was in the Melbourne community in Australia. The inclusion and exclusion criteria used by the researchers would be expected to guarantee the validity of the study with other factors being equal. This question would be answered in the proceeding paragraphs. Bennell et al. (2005) screened the participant initially over the phone and via medical screening. This situation reduces the chances of enrolling subjects who do not suit the objective of the study. Some subjects may be too inconvenient to meet the study protocols, which increases the rate of dropouts or missing data and subsequent bias of study results. Thus, one would expect the screening to add to the validity of the study. However, screening alone does not guarantee the validity of the results of any study, including a randomized controlled study.
According to Stolberg, Norman, & Trop (2004), randomized control trials are susceptible to various forms of biases at all phases of their course. However, the main attractiveness of random controlled trials attaches to its ability to minimize allocation bias (Jadad, 1998). Allocation bias refers to discrimination in assigning subjects to the two treatment groups for comparison of the outcome of clinical intervention.
After the researchers finished doing a baseline assessment of the subjects, they randomly assigned them to the different treatment groups; namely, physiotherapy or placebo group. Bennell and his colleagues (2005) used a computer-generated table of random numbers to assign the subject to either of the two treatment groups. The computer uses two key approaches to generate random numbers, including True Random Number Generators (TNG’s) and Pseudo-Random Numbers Generators (PRNGs). These approaches are algorithms that employ pre-calculated tables or mathematical formulae to produce random numbers (Chaitin, 2001). Once the investigators develop a table of random numbers they need to assign the subjects to the study groups.
The initial step in the treatment assignment is to design a link between the random numbers and the treatments groups. Assuming the odd numbers correspond to the intervention group and even numbers to the placebo group, the next step is to define a way to read the table, for example, to read across the rows or down the column (Silva 1999). An investigator chooses starting point from which the numbers are read following the defined way of reading the table. The numbers are written down as they follow each other. For example, 8, 0, 9, 5, 3, 5, 9, 5, 2, 1. The fourth step involves making the treatment assignments as defined in the table below.
Source: Silva (1999)
This ensured that different traits were distributed equally amongst the physiotherapy and placebo treatment groups, as randomization eliminates allocation bias. Further, an independent researcher kept custody of the assignment scheme, which was concealed. The researchers disclosed the assignment of subjects to treating physiotherapists via telephone when the subjects reported for treatment. All these strategies served to add validity to the results of the study.
Randomization is the major characteristic of randomized controlled trials. This procedure confers RCT its strength (Stolberg et al. 2004). Random assignment implies all subjects have equal opportunity to be allotted to both groups (Altman 1991). The researchers, the clinicians or the subjects did not interfere with the allocations.
Highest possible chance of being identical amongst the study groups at the baseline. If this step is done properly, it minimizes the chance of serious inequality between identified and unidentified factors that could affect the clinical course of the subjects.
The use of competent professionals to administer the physiotherapy and placebo guarantees the validity of the results. The sessions were about 30 to 40, where the expert physiotherapists administered treatment respectively. The sessions were conducted once a week for 4 weeks and then consistently for 8 weeks. The physiotherapist and subjects did not maintain any contact during follow-up, while the investigators recorded the use of other interventions.
Arguably, the approach the researchers used to test the hypothesis was not appropriate. First, physiotherapy does not produce outcomes with small intensity as manifested in the scheme of administering treatments. A 30-minute session of physiotherapy once a week is not sufficient to elicit significant responses from subjects. Second, lack of contact between physiotherapy and subjects meant that the subject could wrongly self-administer some of the physiotherapy interventions, which translates to failure of user intervention. Consequently, there will be no significant difference between the groups. On top of that, subjects were permitted to use medications and other interventions outside of the study protocols. Hence, Bennell, et al. (2005) performed a poor study as its dismal findings at the end testify. They needed to extend the treatment session and period to one or two hours, and a year or two respectively.
The researchers used fake ultrasound and non-therapeutic gel for the placebo, and they did not administer any intervention for the placebo group during the follow-up (Bennell et al. 2005). Use of placebo relates to RCTs in which one or more outcome measures are subjective, in which case it is important to designate the trial as a placebo-controlled study; a subset of RCT.
Placebo refers to a substance or an exercise that mimics a treatment, which does not cause the healing effect on a subject. Participants and investigators cannot distinguish the actual therapy from a placebo. It is challenging to design a placebo for some physiotherapy programs, such as exercise and massage. Although Bennell and colleagues designed some form of placebo to administer during the study, the subjects were able to recognize the treatments they were receiving a placebo, which could have promoted bias as they could adjust their response to suit the expectation of the study or choose to give the opposite expectation because of annoyance with the study. The possibility of bias is greatly reduced when the subjects and the researchers are not aware of the treatment allocation in the study. Thus, they cannot manipulate to attain their own desired outcomes during these trials.
The significance of blinding broadens to all clinical trials evaluations. Therefore, Bennell et al. (2005) could have validated the results of the study by employing an autonomous blinded end-point committee (Stanley 2007). The principles of masking involve various dimensions. The researchers used a blinded examiner to conduct evaluations of all outcomes. The use of placebo and preventing concurrent attendance for treatments or evaluations by subjects was intended to maximize blinding, although this strategy was not successful as some subjects allocated to the placebo group recognize their allocation sometimes through the study. It is difficult to design a placebo version of most physiotherapy programs administered to physiotherapy groups. For example, there could be no placebo version of exercise, taping and massage. Hence, there is a high chance for participants to recognize a fake physiotherapy treatment as they may have experienced the program prior to joining the study.
The statistician and the person in charge of managing data were blinded to treatment assignment up to the end of the analyses (Bennell et al. 2005). This strategy would have added validity to the results if the blinding of the placebo group were successful.
Bennell et al. (2005) employed various tools to measure the outcomes of the study. They gauged pain on movement using a Visual Analogue Scale (VAS).
The investigators recorded patients’ overall change in pain from baseline on a Likert scale, which allowed classification of the subjects as improvers into those with Likert score of 4 or 5; and those who recorded a change in pain of 1. 75 cm or more on the VAS. The study had secondary outcomes. The researchers measured the pain, disability and life quality of the subjects using various tools. The secondary outcomes were measured using a questionnaire tool called the Western Ontario and McMaster Universities osteoarthritis index (WOMAC), 11 points numerical VAS and the knee pain scale, SF-36, and AQoL index. In addition, they used other tools such as step test and KinCon dynamometer.
Multiple outcomes as shown in the above example is a major feature of a randomized controlled trial. Peduzzi, Henderson, Hartigan, and Lavori (2002) argue that a response of the subject to treatment is multifaceted, necessitating the need to measure as many of these dimensions as possible in a controlled trial. The aim of this approach is to gain a holistic insight into the benefits or limitations of intervention(s). In the study to assess the efficacy of various physiotherapy programs on the treatment of knee osteoarthritis, Bennell et al. (2005) targeted three outcomes pain, disability and quality of life.
Bennell et al. (2005) performed statistical analyses of the data using SPSS software. They did the analysis with an intention to treat the basis typical of RCTs, replacing the missing data using a conservative method. The missing data in the respective groups were replaced by the mean of the alternative group. This involved replacing missing data in the placebo group with the mean physiotherapy group.
Intention to treat means that the researchers analyze all the randomized participants based on the first treatment assigned, and counting of total events done against the assigned treatment. Leaving out participants from the analysis can create bias. There are various ways by which participants may be excluded. For instance, participants who do not get the allotted treatment, those who get the incorrect treatment allotment, those who pass away before the treatment is given, those who do not comply with the protocols of the study, or those who drop out. This situation manifested in the study to test the efficacy of physiotherapy in the management of knee osteoarthritis. This study reported the dropout of participants before study completion. 12 participants dropped out in the first 12 weeks of the study, while additional 5 participants dropped out at the follow-up phase (Bennell et al. 2005).
Limitations of Study Designs
The intention to treat principles employed by the RCTs does not give a true test of the efficacy of a treatment in those subjects who followed the study protocol but of the effectiveness of treatment given to all subjects (Fisher, Dixon and Herson 341) regardless of dropouts. Therefore, Bennell et al. (2005) would have strengthened the validity of the results by excluding the analysis of data from subjects who dropped out pants and the events that arise from it (Goetghebeur & Loeys 2002). This approach is termed analysis per protocol.
Rather, Bennell et al. (2005) would have achieved real results using cohort study design. In this case, they would divide the participants into exposed and unexposed groups, with the physiotherapy programs as the exposure. The exposure group will be enrolled for a month or two physiotherapy facilities where they would be trained to self-administer some physiotherapy interventions such as exercise and taping. The study course will include a one-year follow-up period for both groups to evaluate the change in clinical outcomes, such as disability, pain and quality of life associated with knee osteoarthritis.
Conclusion
RCT may not be relevant for assessing interventions with uncommon outcomes or effects that require an extended time to show, in which case other study designs including cohort studies are more proper. In other instances, high risks of low compliance to study protocols or dropout rates, or financial constraints undermine the feasibility of performing an RCT.
A cohort study is good for identifying adverse or rare effects of treatments, or for evaluating dissimilar approaches or changes in service delivery. A cohort study is suitable for evaluating treatment outcomes, or finding out the disease etiology when an RCT is not possible because of ethical issues (Harner & Collinson 2005). For example, it is unethical to include a therapy arm, which is inferior to any other intervention (Silva 2012), as was the case with the study by Bennell et al. (2005).
RCTs are the best possible approach to assess an intervention, of comparing interventions and analyzing a cause-effect relationship (Harner & Collinson 2005). Substituting RCT with a cohort study means that the observers will not be able to evaluate the relationship between the mentioned elements that RCT can achieve.
RCT cannot answer Harvey et al.’s (2010) study because the research question intends to evaluate the relationship of leg-length inequality with prevalence, the incidence of knee osteoarthritis. The study is based on a natural setting rather than the experimental setting. This means that the exposure, leg-inequality, is an inherent phenomenon in the participants and, thus, its relationship to knee osteoarthritis is purely natural and does is not determined by participants or researchers. Ethics hinders researchers from exposing people to factors that cause adverse health conditions. Therefore, the cohort study design is the most appropriate design for investigating the relationship between leg-length inequality and knee osteoarthritis. Similarly, a cohort study design could not answer Bennell et al.’s (2005) study, as physiotherapy is an artificial phenomenon whose effects depend on the researcher and participants. Unlike a natural exposure, such as leg-length, physiotherapy is not consistent and its relationship with improved consequences of knee osteoarthritis is not certain. Hence, the researchers must compare the consequences of physiotherapy and placebo. A cohort study compares baseline conditions against conditions at the end of a study. In essence, a cohort study evaluates the exposure-outcome relationship in a natural setting. It is not effective in evaluating an artificial phenomenon.
References
Altman, D 1991, Practical Statistics for Medical Research, Chapman & Hall, London England.
Bennell, K, Hinman, R, Metcalf, B, Buchbinder, R, McConnell, J, McColl, G, et al. 2005, ‘Efficacy of Physiotherapy Management of Knee Joint Osteoarthritis: A Randomised, Double Blind, Placebo-Controlled Trial’, Annals of Rheumatoid Disability, vol. 64, pp. 906-12.
Chaitin, G 2001, Exploring Randomness, Springer London.
Fisher, L, Dixon, D, & Herson, J 1990, “Intention to Treat in Clinical Trials”, In K. Peace, Statistical Issues in Drug Research and Development, Mercel Dekker, New York, pp. 291-312.
Goetghebeur, E, & Loeys, T 2002, ‘Beyond Intention to Treat’, Epidemiol Review, vol. 24, pp. 85-90.
Harner, S, & Collinson, G 2005, Achieving Evidence-Based Practice: A Handbook for Practitioners, Elseiver, Philadelphia.
Harvey, W, Yang, M, Theodore, C, Segal, N, Lane, N, Lewis, C, et al. 2010, ‘Association of Leg-Length Inequality With Knee Osteoarthritis’, Annals of Internal Medicine, vol. 152, pp. 287-95.
Jadad, 1998, Randomized Controlled Trials: A User’s Guide, BMJ Books, London.
Peduzzi, P, Henderson, W, Hartigan, P, & Lavori, P 2002, ‘Analysis of Randomized Controlled Trials’, Epidemiologic Reviews, vol. 24 no. 1, pp. 26-38.
National Research Council, 2002, Leveraging Longitudinal Data in Developing Countries: Report of a Workshop, National Academy Press, Washington DC.
Reeves, B 2003, ‘Principles of Research: Limitations of Non-Randomized Studies’, Surgery, vol. 21 no. 6, pp. 129-133.
Rothman, K, Greenland, S & Lash, T 2008, Modern Epidemiology, Lippincott Williams & Watkins, San Francisco.
Schoenbach, V. 2011, Principles of Epidemiology for Public Health: Study Designs – Cohort Studies, (Powerpoint), Department of Epidemiology, Gillings School of Global Public Health, University of North Carolina, Chapel Hill.
Silva, S 1999, Cancer Epidemiology: Principles and Methods, International Agency for Research on Cancer – World Health Organization.
Stanley, K 2007, ‘Design of Randomized Controlled Trials’, Journal of the American Heart Association, vol. 115, pp. 1164-1169.
Stolberg, H, Norman, G & Trop, I 2004, ‘Fundamentals of Clinical Research for Radiologist: Randomized Controlled Trial’, American Journal of Radiology, vol. 183, pp. 1539-1544.
Vandenbroucke, J, von Elm, E, Altman, D, Gotzsche, P, Mulrow, C, & Pocock, S 2007, ‘SROBE Initiative. Strengthening the Reporting of Observational Studies in Epidemiology (STROBE): Explanation and Elaboration’, Annals International Medicine, vol. 147, pp. 163-94.