Critical appraisal simply put is the process of systematically looking at research papers to assess three important things: trustworthiness, value and relevance. When critically appraising a research paper the first step is to examine the study for any bias.
Bias can occur in the design or methodology of the study and this can distort the study's findings so that they do not accurately reflect the truth. It should be noted that no study is totally free from bias and for this reason it is necessary to systematically check that the researchers have done all they can to minimise bias.
A study which is sufficiently free from bias is said to have internal validity. A study will be said to have external validity when it can be generalised to the clinical (or wider population) context.
Critical appraisal checklists provide a framework for interpreting and determining the reliability of the evidence. Checklists are designed to help you answer the questions - is the study unbiased, are the findings reliable, and are the findings valid?
AMSTAR 2
An updated version of AMSTAR that appraises systematic reviews, including ones based on non-randomised studies of healthcare interventions. Includes additional criteria such as inclusion of PICO, risk of bias in the evidence synthesis stage, causes and significance of heterogeneity, and justification of chosen study design. 'Yes' answers to questions denote positive results.
JBI’s critical appraisal tools assist in assessing the trustworthiness, relevance and results of published papers.
This set of eight critical appraisal tools are designed to be used when reading research, these include tools for Systematic Reviews, Randomised Controlled Trials, Cohort Studies, Case Control Studies, Economic Evaluations, Diagnostic Studies, Qualitative studies and Clinical Prediction Rule.
These are free to download and can be used by anyone under the Creative Commons License.
Johns Hopkins Research Evidence Appraisal Tool
Contains questions used to evaluate an article's study design and level of evidence. This tool contains three questions that allow the review to determine a study's methodology. Uses a 16 item checklist for research studies and a 12 item checklist for systematic reviews and meta-analyses.
Bias may result from systematic errors in the research methodology. This table from the Cochrane Handbook summarises the different types of bias.
| Type of bias | Description | Collaboration’s ‘Risk of bias’ tool |
| Selection bias | Systematic differences between baseline characteristics of the groups that are compared |
|
| Performance bias | Systematic differences between groups in the care that is provided, or in exposure to factors other than the interventions of interest |
|
| Attrition bias | Systematic differences between groups in withdrawals from a study. Incomplete outcome data |
|
| Detection bias | Systematic differences between groups in how outcomes are determined |
|
| Reporting bias | Systematic differences between reported and unreported findings |
|
The Cochrane risk of bias tool is now recommended for use within all Cochrane reviews, and is widely used by non-Cochrane reviews of randomized controlled trials.
