For part of the RHS presentation

Research Designs Dr Pradeep Paul George Health Services & Outcomes Research.

Objectives To know the purpose of good study design Types of errors Types of study designs Advantages & disadvantages How to choose a suitable design to answer the research questions.

Importance of study design Most problems in studies are due to poor design (not poor analysis) “There are only a handful ways to do a study properly but a thousand ways to do it wrong” Sackett (1986) The best statistical analysis is only as good as the data it’s based on.

[Audio] What is a study design? Logical plan of action for data collection and analysis Protection against Bias Random error Research design is the logical plan of action for the error free data collection (Valid & Precise) and analysis. When we are saying error free we mean unbiased and precise data. A Good research design offers protection against bias and random error.

[Audio] What makes a good study design? The purpose of good study design is Answer the question quantitatively Ensure external generalizability Minimize bias Optimize precision What makes a good study design…….A good study design should help us answer the research question clearly with minimal bias and high precision. A good study design would also helps us collect accurate data which can be generalized to the population.

[Audio] Error’s in epidemiological studies Worst Case Archer 2 Systematic error (Bias) Archer 1 Biased but Precise Biased & Imprecise Random error (Imprecision) Archer 3 Archer 4 Unbiased but Imprecise Unbiased & Precise Best Case Imagine a archery competition in which four archer’s take aim at the bulls eye and shoot their arrows. The first archers shots were biased and impecise, they were far away from the truth (bulls eye) and haphazardly distributed around the target. The second archers shots biased but they are precise, that is they are far away from the bulls eye (truth) but they are precise they are close to each other, There is some systematic error in the way, this archer is firing his shots. The third archer shots are close to the truth (unbiased) but not precise, haphazardly distributed around the target, there is some random error that operating with the way this archer is firing his shots, the more number of tries would help him increase his precision. The fourth archers shots are close to the truth and are very precise. Lets try to understand more about random and systematic error and how they operate.

[Audio] Random Error Symmetrical divergence, due to chance alone Also known as Imprecision Inconsistency Irreproducibility Sources of Random Error Sampling error Measurement error Random error is the divergence of an observation from the true population value due to chance alone. Random error leads to lack of precision in measuring associations. Random error can be reduced with careful measurement of exposure and outcome. We can never completely eliminate random error because we are usually only studying a sample of the population. We can however minimise it. Sampling error usually occurs as part of the process of selecting study participants who are always a sample of a larger population. The best way to reduce sampling error is to increase the sample size. How does random error affect your sample…… And, individuals do differ, and no measurement is ever completely accurate. Measurement errors due to instruments can be reduced by pilot testing and calibarating them before the start of the study.

[Audio] Systematic Error Bias Consistent tendency to produce results that differ from the true values Also known as Inaccuracy Sources of Bias Selection of study subjects Confounding Measurement Systematic error is the tendency to produce result that is systematically away from the true values. Systematic error is inherent in most epidemiological designs. There are about 30 specific types of bias that can result from systematic error in research design. The two main ones are selection and measurement biases. Confounding is not a bias, per se, it is a result of the non random distribution of risk factors in the source and study populations. Lets look at how systematic error affects the data…...

[Audio] Selection Bias When the people selected for the study are not representative of the target population – poor generalizability Population Higher Health Problems Better Compliance Volunteers Sample Occurs when there is systematic difference between the characteristics of people selected for the study and characteristics of those who are not selected. For eg: if we conduct a survey to estimate the prevalence of HT by taking a sample from the target population, after enumeration of the sample some people volunteer to participate in the study and some do not. If the health characteristics of the people who volunteered are different from the non participants then it may bias the result. Say if people who volunteered had higher levels of health problems when compared to Non participants. This type of bias is known as selection bias.

[Audio] Confounding Occurs when a variable exists in the study population and is associated with both the disease and the exposure being studied May create the appearance of a cause effect relationship that really does not exist Age and social class are common confounders Alcohol consumption Lung Cancer Smoking Confounding factor Confounding is a major problem in most epidemiological designs. Confounding results when the unknown or unmeasured variable is associated with both the disease and the exposure, confounding may result in spurious association. For eg: High body mass index/ obesity is associated with Type 2 Diabetes, but the hidden confounder there might be a genotype which is predisposing people to both obesity and Diabetes. So how to control the confounding menace…….

[Audio] Measurement Bias Occurs when individual measurements or classifications of disease or exposure are inaccurate If occurs equally in all groups being compared (non differential bias) – results in underestimate of the true strength of the relationship Sources: Example Uncalibrated instruments Unvalidated questionnaires Recall of exposure bias Ascertainment of outcome bias Measurement bias occurs when there is inaccurate measurements of classification of disease or exposure or individual measurements. Measurement bias is quite common in studies which ascertain objective or subjective measurements Examples – bias as a result of uncalibrated instruments, Recall bias leading to wrong information on the exposure status..

[Audio] Choosing a Study Design Factors Study objectives resources Ethical issues Disease area What are the factors that influence the study design? There are four major factors which influences the study design, those are Research questions – study objectives, therapeutic area of the study how is the Occurrence of the disease – is it Rare/Common, Ethical issues – is it ethical to use this design, Resources – How much budget is permissible for this study, how many resources /Manpower would I need to conduct this study.These are some of the factors which may influence the study design.

[Audio] Descriptive vs Comparative aims Descriptive aims (1 group) Descriptive studies Case series Comparative aims (≥ 2 Groups) Experimental R-C-T Non R-C-T Observational Cohort Case control Cross sectional Research Designs can be broadly classified in two types, designs which helps us answer single group questions and comparative aims Descriptive studies are used predominantly used for studying single group characteristics. They are used to study variation in frequency by demographic characteristics, place & time. Questions with comparative aims can be addressed by either experimental or observational designs investigator depending on whether the investigator assigned the exposure. Experimental Studies are conducted under controlled conditions where researcher controls the exposure. Observational studies do not involve intervention. Observe natural course of events where changes in one characteristic is studied in association with changes in other characteristics. The three common observational designs are cohort, Case control and Cross sectional designs.

[Audio] Randomized controlled trial Evaluate new preventive or therapeutic regimen Strongest design to study causation randomization, minimize confounding bias ( but not completely) As we have already seen, R-C-T provides the highest evidence of causality. Randomization ensures that the control and treatment groups will be comparable. Its useful in evaluating new treatment regimens. The weak links are the ethical uses which needs to be addressed when the intervention is harmful or has poor clinical outcome. R-C-T is not the design of choice for rare diseases and rare outcomes. Subject participation is quite crucial for RCT’s.

[Audio] Randomized controlled trial Design Blinding Treatment Outcome measure Eligible subjects Follow up Measure Randomize Mean Difference Outcome measure Recruit Control Population Intervention & Control Outcome measures pico The key elements of RCT’s are the population studied, the intervention, the control group and the outcomes (P-I-C-O-) Randomisation ensures that the groups remain comparable except the treatment that is being administered During administration of study treatment the process of allocation is done without the knowledge of the patient (single blinding), physician nor the study statistician (Double blinding). This makes sure that preferential bias does not creep in. Outcome are assessed at the end of the study and the measures are compared statistically.

[Audio] R-C-T Example Blinding Treatment PAL’s Progression of Myopia Eligible 6 11yrs with myopia Follow up 3 years Randomize Mean Diff SVL’s Recruit Progression of Myopia Population Intervention & Control Outcome measures pico Gwiazda J, Hyman L and othersA randomized clinical trial of progressive addition lenses versus single vision lenses on the progression of myopia in children.Invest Ophthalmol Vis Sci. 2003 Apr;44(4):1492-500. The eligible study subjects for this study are children in the age group of 6-11 years with Myopia. The interventions Progressive addition lenses and single vision lenses were randomly assigned to the study subjects. The treatment allocation procedure was blinded, the patients, optometrists were not aware of the interventions. Both the treatment group were followed up for 3 years. Outcomes were progression of myopia in the two groups as observed by the mean increase in myopia and mean difference between the treatment and control group.

[Audio] How do we Randomize … Genuine randomization of large numbers computer generated coin toss, dice roll, draws from a bag The paradigm of the R-C-T is the randomization. Randomization is a process where study participants are assigned to one of two or more treatment options by chance. How do we randomize?...... The various methods of randomization are Computer generated random numbers, Flipping of coin/dice..

[Audio] Allocation Concealment… Allocation concealment 3rd party sealed opaque serially numbered envelopes real time generation Purpose is to protect the randomization process from subversion by PI The other important step in R-C-T is the allocation concealment. Allocation concealment means to conceal the process of random allocation. Allocation concealment concentrates on preventing selection and confounding biases. Allocation concealment can be done by a third party or by sending the allocation sequence in a sealed opaque envelopes and allocation can be done in real time as and when as participant is recruited.

[Audio] Blinding protects … Blinding of Caregivers protects against confounding bias due to differential care given Patients preserves Placebo effect Outcome assessors protects against measurement bias The term blinding refers to keeping trial participants, investigators (usually health care providers), or assessors (those collecting outcome data) unaware of the assigned intervention, so that they will not be influenced by that knowledge. Blinding of caregivers protects against confounding bias due to differential care given, for eg Patients in the treatment group may receive better care than the control group if their intervention status is known Blinding of patients make sure that both the groups respond in similar way to the treatment, if the intervention status is known to the patient, the person in the control group may be less compliant when compared to those in the treatment group. To avoid this blinding of patients is mandatory Blinding of outcome assessors protects against the measurement bias..

[Audio] R-C-T Pros & Cons Advantages Design with minimal bias, Randomization & Blinding helps Disadvantages Expensive Patients hard to recruit May suffer from selection bias (lack of generalizability) as study subjects are volunteers Not ethical for harmful interventions Not practical for rare outcomes Generalizability of results of the study is a problem because of the stringent inclusion and exclusion criteria’s. R-C-T are pretty expensive to conduct and patient recruitment is a difficult task. R-C-T is not the ideal design for harmful interventional and exposure because of the ethical issues.

[Audio] Descriptive study What proportion of children with musculoskeletal pain given N-S-A-I-D achieve pain relief within 1 hour? Relieved (no. = ?) Children with musculoskeletal pain given NSAID Not relieved (no. = ?).

[Audio] Cross sectional study (2) How accurate is the Wong Baker Faces Pain Rating Scale in assessing pain among children? Children with various degrees of pain Assessed using the WBFPRS Assessed using standard PS.

[Audio] Cross sectional study Analysis (Validity) Gold standard plus plus A B A plus B C D C plus D A plus C B plus D N Test Sensitivity = A/A plus C Specificity = D/B plus D Positive Predictive Value = A/A plus B Negative Predictive Value = D/C plus D.

[Audio] Cross sectional study Uses: For estimating prevalence rate For measuring validity Key features: Unable to establish temporal sequence of events Provides the weakest measure of association.

[Audio] Case control study Outcome status Exposure status Yes Cases No Controls Yes No time.

[Audio] Case control study Requirements Single source for cases and controls Cases and controls representative of their respective populations Advantages Disadvantages Useful for rare outcomes Impractical for rare exposures Cheap Provides indirect estimate of risk Quick results Weak evidence for causality Secondary data on exposure often not available Primary collection of data on exposure prone to recall bias More prone to misclassification bias.

[Audio] Case control study Analysis Odds ratio = odds of exposure among cases compared to odds of exposure among controls Outcome plus plus A B A plus B C D C plus D A plus C B plus D N A x D B x C Exposure.

[Audio] Case control study (3) Is N-S-A-I-D effective for the relief of musculoskeletal pain in children? Outcome status Exposure status Nsaid Pain Not NSAID Nsaid  Pain Not NSAID time.

[Audio] Interpretation of result Pain Yes No Yes A B A plus B No C D C plus D A plus C B plus D N Nsaid OR = 1 NSAID has no effect on pain OR > 1 Children who are in pain are more likely to have taken NSAID OR < 1 Children who are in pain are less likely to have taken NSAID.

[Audio] Challenges Are data on analgesic intake readily available? If data on analgesic intake are not available, can patients/caregivers accurately and reliably provide the data? If there is a difference in exposure between cases and controls, is this not due to dissimilarity in relevant characteristics, for example case mix, disease severity, patient demographics, doctor characteristics.

[Audio] Cohort study Outcome status Exposure status Cases Yes Controls Cases No Controls time.

[Audio] Analysis Relative risk = Incidence in the exposed compared to incidence in the unexposed Outcome plus Exposure plus A B A plus B C D C plus D A plus C B plus D N A / A plus B C / C plus D.

[Audio] Requirements Similarity in exposed and unexposed groups None of the subjects have the outcome at the start of study Sufficient duration of follow up Advantages Disadvantages Useful for rare exposures Expensive Provides direct estimate of risk More time consuming than other non experimental designs Preserves temporality of events Weak evidence for causality Attrition Compared to other designs, requires larger sample.

[Audio] Quantitative Research Designs – Cohort study Exposure status Outcome status Pain Nsaid  Pain Pain Not NSAID  Pain time.

[Audio] Cohort study Challenges Do we have a basis for sample size calculation? If we have an estimate of the minimum required sample size, do we have the time/resources to complete the study?.

[Audio] Cohort study Interpretation of result Pain Yes No Paracetamol A B A plus B Not paracetamol C D C plus D A plus C B plus D N Analgesic RR = 1 NSAID has no effect on pain RR > 1 Children who took N-S-A-I-D are more likely to get no pain relief RR < 1 Children who took N-S-A-I-D are more likely to get pain relief.

[Audio] Match the Study design to purpose Study designs Study Objective Observational Experimental Prevalence Incidence or natural history To identify causes risk factors or people at high risk To prevent diseases (causality) To treat disease (causality) Cross sectional Cohort Cohort, Case Control, Cross sectional R-C-T R-C-T R-C-T Here are some of the research questions and their corresponding research designs To estimate the Prevalence – Crossectional design is ideal To understand the natural history – Cohort design is the best choice To identify risk factors and people at high risk – we could use either cohort, case control, cross sectional or the R-C-T designs To prevent/ cure diseases or to alter the course of the disease – R-C-T is the design of choice.

[Audio] Practical Exercise. PRACTICAL EXERCISE.

[Audio] Identify the most suitable research design for the following questions How strong is the association between duration of hospital stay and the risk of developing nosocomial M-D-R G-N-B-? What is the in hospital mortality rate for patients who are transferred from the ED to the general ward and subsequently to the I-C-U-? 3. Can setting up an observation unit in the hospital reduce the length of stay of patients who consult at the ED? 4. Which demographic and psychosocial characteristics are predictive of treatment default among substance abuse patients? 5. Is there a relationship between turnover rates of nurses and patient safety event rates? 6. How accurate are estimates of medication errors based on voluntary reports? 7. Is the community based falls prevention program effective in reducing the incidence of falls among elderly individuals 60 plus years old?.