What are the most reliable types of research?
The internet and social media have revolutionised accessibility of information. Gone are the days when we needed to subscribe to a journal that arrived monthly through our letterbox and when we had to enter a library in order to access research. We now have journals and research available to us within seconds via the internet and there are many fitness professionals out there, like myself, who constantly engage with research and share the findings with their followers.
There are also, however, many websites, publications and individuals sharing fitness information based purely on anecdotal evidence, their own beliefs, outdated research and, to be frank, completely made-up stuff that is usually intentionally controversial or evocative because it will create more attention or it’s the current trend.
All the information I share is research-based and admittedly, it has been a learning curve for me to understand what makes for a reliable study, but I can now confidently share information with conviction, as I know it’s the output of carefully planned, thorough, well-reasoned and peer-reviewed research with its findings based on strong evidence. I’d like to share that knowledge with you, so you can sort the fact from the fiction and have confidence in the information you engage with.
Research Methods
There are several common methods at the disposal of a researcher, each with differing levels of bias risk, rigour and accuracy. It’s important we understand these, so we can select the most reliable, rigorous source when comparing studies.
What is a Meta-analysis?
This is the research method I seek out as a priority. Meta-analysis is a quantitative, formal, epidemiological method whereby multiple research studies are critically assessed in order to reach a conclusion on a particular area of research.
As an example, a researcher may critically assess the findings of 20 studies relating to the impact of strength training on lean body mass. They would assess all the studies and based on the large body of information they have, draw a final conclusion.
Outcomes from this example may be a more precise estimate of the impact on lean body mass than any individual study contributing to the pooled analysis.
Meta-analysis benefits include a consolidated and quantitative review of a large, and often complex, sometimes conflicting, body of research. It’s the gold standard of research, so should be top of your list when researching.
What is a Systematic Review?
A systematic review is similar to a meta-analysis, in that it uses existing research, but provides a meticulous summary in answer to a predefined research question rather than using statistical methods. They are known to be one of the most reliable sources of evidence and often include a meta-analysis in order to synthesize the data from the body of research being reviewed.
As data from other sources are being used, there is a lesser chance of bias, making the research high-quality.
What are Randomised Controlled Trials?
Randomised Controlled Trials (RCT) are the gold standard of trials, RCTs are considered the most stringent way of determining whether a direct cause-effect relationship exists between an intervention (an action) and the outcome. For this test, subjects (individuals) are randomly assigned to one of two groups:
Group 1 - Experimental group
Group 2 - The control group
The experimental group receives the intervention that is being tested, for example, during a study looking at the efficacy of a certain exercise for a certain result, group one would carry out that exercise during the trials. The control group would carry out an alternative so the researchers have a comparable.
The data collected from the groups is then studied and compared to see what results the intervention elicits.
A well designed and executed RCT can provide strong evidence of cause and effect, where one exists, however, poorly designed and executed trials can be borne of flawed methodology and lead to significant errors in its findings — which can, of course, mislead the audience.
What are Quasi-Experimental Studies?
Similar to RCTs, these trials also sometimes evaluate efficacy using an intervention and a control group, but using a non-randomised method and can be conducted with no control group.
As the researcher can, in essence, control who is in each group (or if there is even a control group), there is an enate risk with these trials that the groups are not equal when the trial starts which brings doubt on the direct cause of the results as non-randomised trials generally overestimate the strength of the relationship. Also, if conducted without a control group, there is little for comparison which would be used in randomised studies to assess the strength of cause and effect. There is, therefore, less confidence that the intervention has truly had the effect being observed.
These studies are generally conducted when there is a barrier to randomisation, such as, ethical reasons or to save costs, as these are cheaper to conduct than randomised studies. They do have their place, where randomised trials are not possible, but the audience must be satisfied that they have been conducted with the right level of vigour.
What is a Cohort Study?
This is a type of observational study, so there is no intervention, the researcher only observes. They are usually conducted where a trial would not be ethical, for example, something that would require changes to a human body – like plastic surgery. Individuals, or participants, are followed over a period of time, this could be up to several years in order to gather adequate statistical strength of conclusion.
The participants share a common characteristic or risk factor, and researchers will follow up at intervals to collect, for example, lifestyle and health-related data to assess possible correlations between lifestyle and a particular risk factor – like obesity and lifestyle correlations. It can also be conducted retrospectively, where researchers assess passed lifestyle when a cohort has a disease, to again find correlations / cause.
Cohort studies are regarded as the most effective of all the observational studies, they can be very effective where trials are not suitable, and the long length of the study makes for long-term outcome assessment. However, due to their length, they can be expensive, there is a higher risk that participants could withdraw from the study before it ends, and there can be selection bias.
What are Case-Control Studies?
Another observational study, similar to randomised controlled trials, where there is both an experiment and a control group, a case-control study compares two groups, one with a condition and the other without. The participants are then compared, usually in terms of lifestyle, for a health and fitness study to see if there are any correlations to hypothesis ‘cause’ or contributing factors.
These are relatively inexpensive and can be conducted out by small teams, and are very useful when there is little understood regarding an outcome and it’s risk factors. The biggest disadvantage to this type of study is that it can be challenging to obtain reliable information regarding a participant’s past and have been found to produce the opposite conclusion of more robust studies.
What is a Case Study?
The final, least reliable of the observational studies family, case studies are an investigation of a group or an individual. Data is usually gathered via observation and interviews relating to the participants past and present life.
There are several benefits of using this approach, it allows the researcher to delve into much more detail as the number of participants is generally low, this can make case studies a cost-effective approach. It also provides insight for further research and allows research in areas where it may be unethical to run a trial, where there is an intervention needed.
It is, however, at the bottom of the research hierarchy pyramid due to a lack of rigour, and propensity for bias, and where there is a very small number of subjects, it cannot be concluded that the results are respective of a larger body of people.
Parting shot
For anyone that has managed to get to the end of this article, I commend you. Research is an area that fascinates me, but I appreciate it can be on the heavy side for many. That being said, if you really want to have confidence in research, this information is invaluable and you are now armed to seek out the truth! Ensure that where you follow individuals on social media or a blog, the author is referencing recent research and that you take time to read, check the type of study and understand the research being cited. It’s easy to misinterpret information or make assumptions, so ensure you are satisfied that the reference is relevant and the study is high-quality, sound information – you’ll ace that now!
Much love, Andy.