Literature Review: Purpose of a Literature Review
- Literature Review
- Purpose of a Literature Review
- Work in Progress
- Compiling & Writing
- Books, Articles, & Web Pages
- Types of Literature Reviews
- Departmental Differences
- Citation Styles & Plagiarism
- Know the Difference! Systematic Review vs. Literature Review
The purpose of a literature review is to:
- Provide a foundation of knowledge on a topic
- Identify areas of prior scholarship to prevent duplication and give credit to other researchers
- Identify inconstancies: gaps in research, conflicts in previous studies, open questions left from other research
- Identify the need for additional research (justifying your research)
- Identify the relationship of works in the context of their contribution to the topic and other works
- Place your own research within the context of existing literature, making a case for why further study is needed.
Videos & Tutorials
VIDEO: What is the role of a literature review in research? What's it mean to "review" the literature? Get the big picture of what to expect as part of the process. This video is published under a Creative Commons 3.0 BY-NC-SA US license. License, credits, and contact information can be found here: https://www.lib.ncsu.edu/tutorials/litreview/
Elements in a Literature Review
- Elements in a Literature Review txt of infographic
- << Previous: Literature Review
- Next: Searching >>
- Last Updated: Oct 19, 2023 12:07 PM
- URL: https://uscupstate.libguides.com/Literature_Review
Why is it important to do a literature review in research?
The importance of scientific communication in the healthcare industry
The Importance and Role of Biostatistics in Clinical Research
“A substantive, thorough, sophisticated literature review is a precondition for doing substantive, thorough, sophisticated research”. Boote and Baile 2005
Authors of manuscripts treat writing a literature review as a routine work or a mere formality. But a seasoned one knows the purpose and importance of a well-written literature review. Since it is one of the basic needs for researches at any level, they have to be done vigilantly. Only then the reader will know that the basics of research have not been neglected.
The aim of any literature review is to summarize and synthesize the arguments and ideas of existing knowledge in a particular field without adding any new contributions. Being built on existing knowledge they help the researcher to even turn the wheels of the topic of research. It is possible only with profound knowledge of what is wrong in the existing findings in detail to overpower them. For other researches, the literature review gives the direction to be headed for its success.
The common perception of literature review and reality:
As per the common belief, literature reviews are only a summary of the sources related to the research. And many authors of scientific manuscripts believe that they are only surveys of what are the researches are done on the chosen topic. But on the contrary, it uses published information from pertinent and relevant sources like
- Scholarly books
- Scientific papers
- Latest studies in the field
- Established school of thoughts
- Relevant articles from renowned scientific journals
and many more for a field of study or theory or a particular problem to do the following:
- Summarize into a brief account of all information
- Synthesize the information by restructuring and reorganizing
- Critical evaluation of a concept or a school of thought or ideas
- Familiarize the authors to the extent of knowledge in the particular field
- Compare & contrast
By doing the above on the relevant information, it provides the reader of the scientific manuscript with the following for a better understanding of it:
- It establishes the authors’ in-depth understanding and knowledge of their field subject
- It gives the background of the research
- Portrays the scientific manuscript plan of examining the research result
- Illuminates on how the knowledge has changed within the field
- Highlights what has already been done in a particular field
- Information of the generally accepted facts, emerging and current state of the topic of research
- Identifies the research gap that is still unexplored or under-researched fields
- Demonstrates how the research fits within a larger field of study
- Provides an overview of the sources explored during the research of a particular topic
Importance of literature review in research:
The importance of literature review in scientific manuscripts can be condensed into an analytical feature to enable the multifold reach of its significance. It adds value to the legitimacy of the research in many ways:
- Provides the interpretation of existing literature in light of updated developments in the field to help in establishing the consistency in knowledge and relevancy of existing materials
- It helps in calculating the impact of the latest information in the field by mapping their progress of knowledge.
- It brings out the dialects of contradictions between various thoughts within the field to establish facts
- The research gaps scrutinized initially are further explored to establish the latest facts of theories to add value to the field
- Indicates the current research place in the schema of a particular field
- Provides information for relevancy and coherency to check the research
- Apart from elucidating the continuance of knowledge, it also points out areas that require further investigation and thus aid as a starting point of any future research
- Justifies the research and sets up the research question
- Sets up a theoretical framework comprising the concepts and theories of the research upon which its success can be judged
- Helps to adopt a more appropriate methodology for the research by examining the strengths and weaknesses of existing research in the same field
- Increases the significance of the results by comparing it with the existing literature
- Provides a point of reference by writing the findings in the scientific manuscript
- Helps to get the due credit from the audience for having done the fact-finding and fact-checking mission in the scientific manuscripts
- The more the reference of relevant sources of it could increase more of its trustworthiness with the readers
- Helps to prevent plagiarism by tailoring and uniquely tweaking the scientific manuscript not to repeat other’s original idea
- By preventing plagiarism , it saves the scientific manuscript from rejection and thus also saves a lot of time and money
- Helps to evaluate, condense and synthesize gist in the author’s own words to sharpen the research focus
- Helps to compare and contrast to show the originality and uniqueness of the research than that of the existing other researches
- Rationalizes the need for conducting the particular research in a specified field
- Helps to collect data accurately for allowing any new methodology of research than the existing ones
- Enables the readers of the manuscript to answer the following questions of its readers for its better chances for publication
- What do the researchers know?
- What do they not know?
- Is the scientific manuscript reliable and trustworthy?
- What are the knowledge gaps of the researcher?
22. It helps the readers to identify the following for further reading of the scientific manuscript:
- What has been already established, discredited and accepted in the particular field of research
- Areas of controversy and conflicts among different schools of thought
- Unsolved problems and issues in the connected field of research
- The emerging trends and approaches
- How the research extends, builds upon and leaves behind from the previous research
A profound literature review with many relevant sources of reference will enhance the chances of the scientific manuscript publication in renowned and reputed scientific journals .
journal Publishing services | Scientific Editing Services | Medical Writing Services | scientific research writing service | Scientific communication services
Scientific Research Paper Writing
Medical Research Paper Writing
Scientific Communication in healthcare
Statistical analyses of case-control studies
PUB - Selecting material (e.g. excipient, active pharmaceutical ingredient) for drug development
Selecting material (e.g. excipient, active pharmaceutical ingredient, packaging material) for drug development
PUB - Health Economics of Data Modeling
Health economics in clinical trials
Comments are closed.
An official website of the United States government
The .gov means it's official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you're on a federal government site.
The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.
- Account settings
- Browse Titles
NCBI Bookshelf. A service of the National Library of Medicine, National Institutes of Health.
Lau F, Kuziemsky C, editors. Handbook of eHealth Evaluation: An Evidence-based Approach [Internet]. Victoria (BC): University of Victoria; 2017 Feb 27.
Handbook of eHealth Evaluation: An Evidence-based Approach [Internet].
Chapter 9 methods for literature reviews.
Guy Paré and Spyros Kitsiou .
Literature reviews play a critical role in scholarship because science remains, first and foremost, a cumulative endeavour ( vom Brocke et al., 2009 ). As in any academic discipline, rigorous knowledge syntheses are becoming indispensable in keeping up with an exponentially growing eHealth literature, assisting practitioners, academics, and graduate students in finding, evaluating, and synthesizing the contents of many empirical and conceptual papers. Among other methods, literature reviews are essential for: (a) identifying what has been written on a subject or topic; (b) determining the extent to which a specific research area reveals any interpretable trends or patterns; (c) aggregating empirical findings related to a narrow research question to support evidence-based practice; (d) generating new frameworks and theories; and (e) identifying topics or questions requiring more investigation ( Paré, Trudel, Jaana, & Kitsiou, 2015 ).
Literature reviews can take two major forms. The most prevalent one is the “literature review” or “background” section within a journal paper or a chapter in a graduate thesis. This section synthesizes the extant literature and usually identifies the gaps in knowledge that the empirical study addresses ( Sylvester, Tate, & Johnstone, 2013 ). It may also provide a theoretical foundation for the proposed study, substantiate the presence of the research problem, justify the research as one that contributes something new to the cumulated knowledge, or validate the methods and approaches for the proposed study ( Hart, 1998 ; Levy & Ellis, 2006 ).
The second form of literature review, which is the focus of this chapter, constitutes an original and valuable work of research in and of itself ( Paré et al., 2015 ). Rather than providing a base for a researcher’s own work, it creates a solid starting point for all members of the community interested in a particular area or topic ( Mulrow, 1987 ). The so-called “review article” is a journal-length paper which has an overarching purpose to synthesize the literature in a field, without collecting or analyzing any primary data ( Green, Johnson, & Adams, 2006 ).
When appropriately conducted, review articles represent powerful information sources for practitioners looking for state-of-the art evidence to guide their decision-making and work practices ( Paré et al., 2015 ). Further, high-quality reviews become frequently cited pieces of work which researchers seek out as a first clear outline of the literature when undertaking empirical studies ( Cooper, 1988 ; Rowe, 2014 ). Scholars who track and gauge the impact of articles have found that review papers are cited and downloaded more often than any other type of published article ( Cronin, Ryan, & Coughlan, 2008 ; Montori, Wilczynski, Morgan, Haynes, & Hedges, 2003 ; Patsopoulos, Analatos, & Ioannidis, 2005 ). The reason for their popularity may be the fact that reading the review enables one to have an overview, if not a detailed knowledge of the area in question, as well as references to the most useful primary sources ( Cronin et al., 2008 ). Although they are not easy to conduct, the commitment to complete a review article provides a tremendous service to one’s academic community ( Paré et al., 2015 ; Petticrew & Roberts, 2006 ). Most, if not all, peer-reviewed journals in the fields of medical informatics publish review articles of some type.
The main objectives of this chapter are fourfold: (a) to provide an overview of the major steps and activities involved in conducting a stand-alone literature review; (b) to describe and contrast the different types of review articles that can contribute to the eHealth knowledge base; (c) to illustrate each review type with one or two examples from the eHealth literature; and (d) to provide a series of recommendations for prospective authors of review articles in this domain.
9.2. Overview of the Literature Review Process and Steps
As explained in Templier and Paré (2015) , there are six generic steps involved in conducting a review article:
- formulating the research question(s) and objective(s),
- searching the extant literature,
- screening for inclusion,
- assessing the quality of primary studies,
- extracting data, and
- analyzing data.
Although these steps are presented here in sequential order, one must keep in mind that the review process can be iterative and that many activities can be initiated during the planning stage and later refined during subsequent phases ( Finfgeld-Connett & Johnson, 2013 ; Kitchenham & Charters, 2007 ).
Formulating the research question(s) and objective(s): As a first step, members of the review team must appropriately justify the need for the review itself ( Petticrew & Roberts, 2006 ), identify the review’s main objective(s) ( Okoli & Schabram, 2010 ), and define the concepts or variables at the heart of their synthesis ( Cooper & Hedges, 2009 ; Webster & Watson, 2002 ). Importantly, they also need to articulate the research question(s) they propose to investigate ( Kitchenham & Charters, 2007 ). In this regard, we concur with Jesson, Matheson, and Lacey (2011) that clearly articulated research questions are key ingredients that guide the entire review methodology; they underscore the type of information that is needed, inform the search for and selection of relevant literature, and guide or orient the subsequent analysis. Searching the extant literature: The next step consists of searching the literature and making decisions about the suitability of material to be considered in the review ( Cooper, 1988 ). There exist three main coverage strategies. First, exhaustive coverage means an effort is made to be as comprehensive as possible in order to ensure that all relevant studies, published and unpublished, are included in the review and, thus, conclusions are based on this all-inclusive knowledge base. The second type of coverage consists of presenting materials that are representative of most other works in a given field or area. Often authors who adopt this strategy will search for relevant articles in a small number of top-tier journals in a field ( Paré et al., 2015 ). In the third strategy, the review team concentrates on prior works that have been central or pivotal to a particular topic. This may include empirical studies or conceptual papers that initiated a line of investigation, changed how problems or questions were framed, introduced new methods or concepts, or engendered important debate ( Cooper, 1988 ). Screening for inclusion: The following step consists of evaluating the applicability of the material identified in the preceding step ( Levy & Ellis, 2006 ; vom Brocke et al., 2009 ). Once a group of potential studies has been identified, members of the review team must screen them to determine their relevance ( Petticrew & Roberts, 2006 ). A set of predetermined rules provides a basis for including or excluding certain studies. This exercise requires a significant investment on the part of researchers, who must ensure enhanced objectivity and avoid biases or mistakes. As discussed later in this chapter, for certain types of reviews there must be at least two independent reviewers involved in the screening process and a procedure to resolve disagreements must also be in place ( Liberati et al., 2009 ; Shea et al., 2009 ). Assessing the quality of primary studies: In addition to screening material for inclusion, members of the review team may need to assess the scientific quality of the selected studies, that is, appraise the rigour of the research design and methods. Such formal assessment, which is usually conducted independently by at least two coders, helps members of the review team refine which studies to include in the final sample, determine whether or not the differences in quality may affect their conclusions, or guide how they analyze the data and interpret the findings ( Petticrew & Roberts, 2006 ). Ascribing quality scores to each primary study or considering through domain-based evaluations which study components have or have not been designed and executed appropriately makes it possible to reflect on the extent to which the selected study addresses possible biases and maximizes validity ( Shea et al., 2009 ). Extracting data: The following step involves gathering or extracting applicable information from each primary study included in the sample and deciding what is relevant to the problem of interest ( Cooper & Hedges, 2009 ). Indeed, the type of data that should be recorded mainly depends on the initial research questions ( Okoli & Schabram, 2010 ). However, important information may also be gathered about how, when, where and by whom the primary study was conducted, the research design and methods, or qualitative/quantitative results ( Cooper & Hedges, 2009 ). Analyzing and synthesizing data : As a final step, members of the review team must collate, summarize, aggregate, organize, and compare the evidence extracted from the included studies. The extracted data must be presented in a meaningful way that suggests a new contribution to the extant literature ( Jesson et al., 2011 ). Webster and Watson (2002) warn researchers that literature reviews should be much more than lists of papers and should provide a coherent lens to make sense of extant knowledge on a given topic. There exist several methods and techniques for synthesizing quantitative (e.g., frequency analysis, meta-analysis) and qualitative (e.g., grounded theory, narrative analysis, meta-ethnography) evidence ( Dixon-Woods, Agarwal, Jones, Young, & Sutton, 2005 ; Thomas & Harden, 2008 ).
9.3. Types of Review Articles and Brief Illustrations
EHealth researchers have at their disposal a number of approaches and methods for making sense out of existing literature, all with the purpose of casting current research findings into historical contexts or explaining contradictions that might exist among a set of primary research studies conducted on a particular topic. Our classification scheme is largely inspired from Paré and colleagues’ (2015) typology. Below we present and illustrate those review types that we feel are central to the growth and development of the eHealth domain.
9.3.1. Narrative Reviews
The narrative review is the “traditional” way of reviewing the extant literature and is skewed towards a qualitative interpretation of prior knowledge ( Sylvester et al., 2013 ). Put simply, a narrative review attempts to summarize or synthesize what has been written on a particular topic but does not seek generalization or cumulative knowledge from what is reviewed ( Davies, 2000 ; Green et al., 2006 ). Instead, the review team often undertakes the task of accumulating and synthesizing the literature to demonstrate the value of a particular point of view ( Baumeister & Leary, 1997 ). As such, reviewers may selectively ignore or limit the attention paid to certain studies in order to make a point. In this rather unsystematic approach, the selection of information from primary articles is subjective, lacks explicit criteria for inclusion and can lead to biased interpretations or inferences ( Green et al., 2006 ). There are several narrative reviews in the particular eHealth domain, as in all fields, which follow such an unstructured approach ( Silva et al., 2015 ; Paul et al., 2015 ).
Despite these criticisms, this type of review can be very useful in gathering together a volume of literature in a specific subject area and synthesizing it. As mentioned above, its primary purpose is to provide the reader with a comprehensive background for understanding current knowledge and highlighting the significance of new research ( Cronin et al., 2008 ). Faculty like to use narrative reviews in the classroom because they are often more up to date than textbooks, provide a single source for students to reference, and expose students to peer-reviewed literature ( Green et al., 2006 ). For researchers, narrative reviews can inspire research ideas by identifying gaps or inconsistencies in a body of knowledge, thus helping researchers to determine research questions or formulate hypotheses. Importantly, narrative reviews can also be used as educational articles to bring practitioners up to date with certain topics of issues ( Green et al., 2006 ).
Recently, there have been several efforts to introduce more rigour in narrative reviews that will elucidate common pitfalls and bring changes into their publication standards. Information systems researchers, among others, have contributed to advancing knowledge on how to structure a “traditional” review. For instance, Levy and Ellis (2006) proposed a generic framework for conducting such reviews. Their model follows the systematic data processing approach comprised of three steps, namely: (a) literature search and screening; (b) data extraction and analysis; and (c) writing the literature review. They provide detailed and very helpful instructions on how to conduct each step of the review process. As another methodological contribution, vom Brocke et al. (2009) offered a series of guidelines for conducting literature reviews, with a particular focus on how to search and extract the relevant body of knowledge. Last, Bandara, Miskon, and Fielt (2011) proposed a structured, predefined and tool-supported method to identify primary studies within a feasible scope, extract relevant content from identified articles, synthesize and analyze the findings, and effectively write and present the results of the literature review. We highly recommend that prospective authors of narrative reviews consult these useful sources before embarking on their work.
Darlow and Wen (2015) provide a good example of a highly structured narrative review in the eHealth field. These authors synthesized published articles that describe the development process of mobile health ( m-health ) interventions for patients’ cancer care self-management. As in most narrative reviews, the scope of the research questions being investigated is broad: (a) how development of these systems are carried out; (b) which methods are used to investigate these systems; and (c) what conclusions can be drawn as a result of the development of these systems. To provide clear answers to these questions, a literature search was conducted on six electronic databases and Google Scholar . The search was performed using several terms and free text words, combining them in an appropriate manner. Four inclusion and three exclusion criteria were utilized during the screening process. Both authors independently reviewed each of the identified articles to determine eligibility and extract study information. A flow diagram shows the number of studies identified, screened, and included or excluded at each stage of study selection. In terms of contributions, this review provides a series of practical recommendations for m-health intervention development.
9.3.2. Descriptive or Mapping Reviews
The primary goal of a descriptive review is to determine the extent to which a body of knowledge in a particular research topic reveals any interpretable pattern or trend with respect to pre-existing propositions, theories, methodologies or findings ( King & He, 2005 ; Paré et al., 2015 ). In contrast with narrative reviews, descriptive reviews follow a systematic and transparent procedure, including searching, screening and classifying studies ( Petersen, Vakkalanka, & Kuzniarz, 2015 ). Indeed, structured search methods are used to form a representative sample of a larger group of published works ( Paré et al., 2015 ). Further, authors of descriptive reviews extract from each study certain characteristics of interest, such as publication year, research methods, data collection techniques, and direction or strength of research outcomes (e.g., positive, negative, or non-significant) in the form of frequency analysis to produce quantitative results ( Sylvester et al., 2013 ). In essence, each study included in a descriptive review is treated as the unit of analysis and the published literature as a whole provides a database from which the authors attempt to identify any interpretable trends or draw overall conclusions about the merits of existing conceptualizations, propositions, methods or findings ( Paré et al., 2015 ). In doing so, a descriptive review may claim that its findings represent the state of the art in a particular domain ( King & He, 2005 ).
In the fields of health sciences and medical informatics, reviews that focus on examining the range, nature and evolution of a topic area are described by Anderson, Allen, Peckham, and Goodwin (2008) as mapping reviews . Like descriptive reviews, the research questions are generic and usually relate to publication patterns and trends. There is no preconceived plan to systematically review all of the literature although this can be done. Instead, researchers often present studies that are representative of most works published in a particular area and they consider a specific time frame to be mapped.
An example of this approach in the eHealth domain is offered by DeShazo, Lavallie, and Wolf (2009). The purpose of this descriptive or mapping review was to characterize publication trends in the medical informatics literature over a 20-year period (1987 to 2006). To achieve this ambitious objective, the authors performed a bibliometric analysis of medical informatics citations indexed in medline using publication trends, journal frequencies, impact factors, Medical Subject Headings (MeSH) term frequencies, and characteristics of citations. Findings revealed that there were over 77,000 medical informatics articles published during the covered period in numerous journals and that the average annual growth rate was 12%. The MeSH term analysis also suggested a strong interdisciplinary trend. Finally, average impact scores increased over time with two notable growth periods. Overall, patterns in research outputs that seem to characterize the historic trends and current components of the field of medical informatics suggest it may be a maturing discipline (DeShazo et al., 2009).
9.3.3. Scoping Reviews
Scoping reviews attempt to provide an initial indication of the potential size and nature of the extant literature on an emergent topic (Arksey & O’Malley, 2005; Daudt, van Mossel, & Scott, 2013 ; Levac, Colquhoun, & O’Brien, 2010). A scoping review may be conducted to examine the extent, range and nature of research activities in a particular area, determine the value of undertaking a full systematic review (discussed next), or identify research gaps in the extant literature ( Paré et al., 2015 ). In line with their main objective, scoping reviews usually conclude with the presentation of a detailed research agenda for future works along with potential implications for both practice and research.
Unlike narrative and descriptive reviews, the whole point of scoping the field is to be as comprehensive as possible, including grey literature (Arksey & O’Malley, 2005). Inclusion and exclusion criteria must be established to help researchers eliminate studies that are not aligned with the research questions. It is also recommended that at least two independent coders review abstracts yielded from the search strategy and then the full articles for study selection ( Daudt et al., 2013 ). The synthesized evidence from content or thematic analysis is relatively easy to present in tabular form (Arksey & O’Malley, 2005; Thomas & Harden, 2008 ).
One of the most highly cited scoping reviews in the eHealth domain was published by Archer, Fevrier-Thomas, Lokker, McKibbon, and Straus (2011) . These authors reviewed the existing literature on personal health record ( phr ) systems including design, functionality, implementation, applications, outcomes, and benefits. Seven databases were searched from 1985 to March 2010. Several search terms relating to phr s were used during this process. Two authors independently screened titles and abstracts to determine inclusion status. A second screen of full-text articles, again by two independent members of the research team, ensured that the studies described phr s. All in all, 130 articles met the criteria and their data were extracted manually into a database. The authors concluded that although there is a large amount of survey, observational, cohort/panel, and anecdotal evidence of phr benefits and satisfaction for patients, more research is needed to evaluate the results of phr implementations. Their in-depth analysis of the literature signalled that there is little solid evidence from randomized controlled trials or other studies through the use of phr s. Hence, they suggested that more research is needed that addresses the current lack of understanding of optimal functionality and usability of these systems, and how they can play a beneficial role in supporting patient self-management ( Archer et al., 2011 ).
9.3.4. Forms of Aggregative Reviews
Healthcare providers, practitioners, and policy-makers are nowadays overwhelmed with large volumes of information, including research-based evidence from numerous clinical trials and evaluation studies, assessing the effectiveness of health information technologies and interventions ( Ammenwerth & de Keizer, 2004 ; Deshazo et al., 2009 ). It is unrealistic to expect that all these disparate actors will have the time, skills, and necessary resources to identify the available evidence in the area of their expertise and consider it when making decisions. Systematic reviews that involve the rigorous application of scientific strategies aimed at limiting subjectivity and bias (i.e., systematic and random errors) can respond to this challenge.
Systematic reviews attempt to aggregate, appraise, and synthesize in a single source all empirical evidence that meet a set of previously specified eligibility criteria in order to answer a clearly formulated and often narrow research question on a particular topic of interest to support evidence-based practice ( Liberati et al., 2009 ). They adhere closely to explicit scientific principles ( Liberati et al., 2009 ) and rigorous methodological guidelines (Higgins & Green, 2008) aimed at reducing random and systematic errors that can lead to deviations from the truth in results or inferences. The use of explicit methods allows systematic reviews to aggregate a large body of research evidence, assess whether effects or relationships are in the same direction and of the same general magnitude, explain possible inconsistencies between study results, and determine the strength of the overall evidence for every outcome of interest based on the quality of included studies and the general consistency among them ( Cook, Mulrow, & Haynes, 1997 ). The main procedures of a systematic review involve:
- Formulating a review question and developing a search strategy based on explicit inclusion criteria for the identification of eligible studies (usually described in the context of a detailed review protocol).
- Searching for eligible studies using multiple databases and information sources, including grey literature sources, without any language restrictions.
- Selecting studies, extracting data, and assessing risk of bias in a duplicate manner using two independent reviewers to avoid random or systematic errors in the process.
- Analyzing data using quantitative or qualitative methods.
- Presenting results in summary of findings tables.
- Interpreting results and drawing conclusions.
Many systematic reviews, but not all, use statistical methods to combine the results of independent studies into a single quantitative estimate or summary effect size. Known as meta-analyses , these reviews use specific data extraction and statistical techniques (e.g., network, frequentist, or Bayesian meta-analyses) to calculate from each study by outcome of interest an effect size along with a confidence interval that reflects the degree of uncertainty behind the point estimate of effect ( Borenstein, Hedges, Higgins, & Rothstein, 2009 ; Deeks, Higgins, & Altman, 2008 ). Subsequently, they use fixed or random-effects analysis models to combine the results of the included studies, assess statistical heterogeneity, and calculate a weighted average of the effect estimates from the different studies, taking into account their sample sizes. The summary effect size is a value that reflects the average magnitude of the intervention effect for a particular outcome of interest or, more generally, the strength of a relationship between two variables across all studies included in the systematic review. By statistically combining data from multiple studies, meta-analyses can create more precise and reliable estimates of intervention effects than those derived from individual studies alone, when these are examined independently as discrete sources of information.
The review by Gurol-Urganci, de Jongh, Vodopivec-Jamsek, Atun, and Car (2013) on the effects of mobile phone messaging reminders for attendance at healthcare appointments is an illustrative example of a high-quality systematic review with meta-analysis. Missed appointments are a major cause of inefficiency in healthcare delivery with substantial monetary costs to health systems. These authors sought to assess whether mobile phone-based appointment reminders delivered through Short Message Service ( sms ) or Multimedia Messaging Service ( mms ) are effective in improving rates of patient attendance and reducing overall costs. To this end, they conducted a comprehensive search on multiple databases using highly sensitive search strategies without language or publication-type restrictions to identify all rct s that are eligible for inclusion. In order to minimize the risk of omitting eligible studies not captured by the original search, they supplemented all electronic searches with manual screening of trial registers and references contained in the included studies. Study selection, data extraction, and risk of bias assessments were performed independently by two coders using standardized methods to ensure consistency and to eliminate potential errors. Findings from eight rct s involving 6,615 participants were pooled into meta-analyses to calculate the magnitude of effects that mobile text message reminders have on the rate of attendance at healthcare appointments compared to no reminders and phone call reminders.
Meta-analyses are regarded as powerful tools for deriving meaningful conclusions. However, there are situations in which it is neither reasonable nor appropriate to pool studies together using meta-analytic methods simply because there is extensive clinical heterogeneity between the included studies or variation in measurement tools, comparisons, or outcomes of interest. In these cases, systematic reviews can use qualitative synthesis methods such as vote counting, content analysis, classification schemes and tabulations, as an alternative approach to narratively synthesize the results of the independent studies included in the review. This form of review is known as qualitative systematic review.
A rigorous example of one such review in the eHealth domain is presented by Mickan, Atherton, Roberts, Heneghan, and Tilson (2014) on the use of handheld computers by healthcare professionals and their impact on access to information and clinical decision-making. In line with the methodological guidelines for systematic reviews, these authors: (a) developed and registered with prospero ( www.crd.york.ac.uk/ prospero / ) an a priori review protocol; (b) conducted comprehensive searches for eligible studies using multiple databases and other supplementary strategies (e.g., forward searches); and (c) subsequently carried out study selection, data extraction, and risk of bias assessments in a duplicate manner to eliminate potential errors in the review process. Heterogeneity between the included studies in terms of reported outcomes and measures precluded the use of meta-analytic methods. To this end, the authors resorted to using narrative analysis and synthesis to describe the effectiveness of handheld computers on accessing information for clinical knowledge, adherence to safety and clinical quality guidelines, and diagnostic decision-making.
In recent years, the number of systematic reviews in the field of health informatics has increased considerably. Systematic reviews with discordant findings can cause great confusion and make it difficult for decision-makers to interpret the review-level evidence ( Moher, 2013 ). Therefore, there is a growing need for appraisal and synthesis of prior systematic reviews to ensure that decision-making is constantly informed by the best available accumulated evidence. Umbrella reviews , also known as overviews of systematic reviews, are tertiary types of evidence synthesis that aim to accomplish this; that is, they aim to compare and contrast findings from multiple systematic reviews and meta-analyses ( Becker & Oxman, 2008 ). Umbrella reviews generally adhere to the same principles and rigorous methodological guidelines used in systematic reviews. However, the unit of analysis in umbrella reviews is the systematic review rather than the primary study ( Becker & Oxman, 2008 ). Unlike systematic reviews that have a narrow focus of inquiry, umbrella reviews focus on broader research topics for which there are several potential interventions ( Smith, Devane, Begley, & Clarke, 2011 ). A recent umbrella review on the effects of home telemonitoring interventions for patients with heart failure critically appraised, compared, and synthesized evidence from 15 systematic reviews to investigate which types of home telemonitoring technologies and forms of interventions are more effective in reducing mortality and hospital admissions ( Kitsiou, Paré, & Jaana, 2015 ).
9.3.5. Realist Reviews
Realist reviews are theory-driven interpretative reviews developed to inform, enhance, or supplement conventional systematic reviews by making sense of heterogeneous evidence about complex interventions applied in diverse contexts in a way that informs policy decision-making ( Greenhalgh, Wong, Westhorp, & Pawson, 2011 ). They originated from criticisms of positivist systematic reviews which centre on their “simplistic” underlying assumptions ( Oates, 2011 ). As explained above, systematic reviews seek to identify causation. Such logic is appropriate for fields like medicine and education where findings of randomized controlled trials can be aggregated to see whether a new treatment or intervention does improve outcomes. However, many argue that it is not possible to establish such direct causal links between interventions and outcomes in fields such as social policy, management, and information systems where for any intervention there is unlikely to be a regular or consistent outcome ( Oates, 2011 ; Pawson, 2006 ; Rousseau, Manning, & Denyer, 2008 ).
To circumvent these limitations, Pawson, Greenhalgh, Harvey, and Walshe (2005) have proposed a new approach for synthesizing knowledge that seeks to unpack the mechanism of how “complex interventions” work in particular contexts. The basic research question — what works? — which is usually associated with systematic reviews changes to: what is it about this intervention that works, for whom, in what circumstances, in what respects and why? Realist reviews have no particular preference for either quantitative or qualitative evidence. As a theory-building approach, a realist review usually starts by articulating likely underlying mechanisms and then scrutinizes available evidence to find out whether and where these mechanisms are applicable ( Shepperd et al., 2009 ). Primary studies found in the extant literature are viewed as case studies which can test and modify the initial theories ( Rousseau et al., 2008 ).
The main objective pursued in the realist review conducted by Otte-Trojel, de Bont, Rundall, and van de Klundert (2014) was to examine how patient portals contribute to health service delivery and patient outcomes. The specific goals were to investigate how outcomes are produced and, most importantly, how variations in outcomes can be explained. The research team started with an exploratory review of background documents and research studies to identify ways in which patient portals may contribute to health service delivery and patient outcomes. The authors identified six main ways which represent “educated guesses” to be tested against the data in the evaluation studies. These studies were identified through a formal and systematic search in four databases between 2003 and 2013. Two members of the research team selected the articles using a pre-established list of inclusion and exclusion criteria and following a two-step procedure. The authors then extracted data from the selected articles and created several tables, one for each outcome category. They organized information to bring forward those mechanisms where patient portals contribute to outcomes and the variation in outcomes across different contexts.
9.3.6. Critical Reviews
Lastly, critical reviews aim to provide a critical evaluation and interpretive analysis of existing literature on a particular topic of interest to reveal strengths, weaknesses, contradictions, controversies, inconsistencies, and/or other important issues with respect to theories, hypotheses, research methods or results ( Baumeister & Leary, 1997 ; Kirkevold, 1997 ). Unlike other review types, critical reviews attempt to take a reflective account of the research that has been done in a particular area of interest, and assess its credibility by using appraisal instruments or critical interpretive methods. In this way, critical reviews attempt to constructively inform other scholars about the weaknesses of prior research and strengthen knowledge development by giving focus and direction to studies for further improvement ( Kirkevold, 1997 ).
Kitsiou, Paré, and Jaana (2013) provide an example of a critical review that assessed the methodological quality of prior systematic reviews of home telemonitoring studies for chronic patients. The authors conducted a comprehensive search on multiple databases to identify eligible reviews and subsequently used a validated instrument to conduct an in-depth quality appraisal. Results indicate that the majority of systematic reviews in this particular area suffer from important methodological flaws and biases that impair their internal validity and limit their usefulness for clinical and decision-making purposes. To this end, they provide a number of recommendations to strengthen knowledge development towards improving the design and execution of future reviews on home telemonitoring.
Table 9.1 outlines the main types of literature reviews that were described in the previous sub-sections and summarizes the main characteristics that distinguish one review type from another. It also includes key references to methodological guidelines and useful sources that can be used by eHealth scholars and researchers for planning and developing reviews.
Typology of Literature Reviews (adapted from Paré et al., 2015).
As shown in Table 9.1 , each review type addresses different kinds of research questions or objectives, which subsequently define and dictate the methods and approaches that need to be used to achieve the overarching goal(s) of the review. For example, in the case of narrative reviews, there is greater flexibility in searching and synthesizing articles ( Green et al., 2006 ). Researchers are often relatively free to use a diversity of approaches to search, identify, and select relevant scientific articles, describe their operational characteristics, present how the individual studies fit together, and formulate conclusions. On the other hand, systematic reviews are characterized by their high level of systematicity, rigour, and use of explicit methods, based on an “a priori” review plan that aims to minimize bias in the analysis and synthesis process (Higgins & Green, 2008). Some reviews are exploratory in nature (e.g., scoping/mapping reviews), whereas others may be conducted to discover patterns (e.g., descriptive reviews) or involve a synthesis approach that may include the critical analysis of prior research ( Paré et al., 2015 ). Hence, in order to select the most appropriate type of review, it is critical to know before embarking on a review project, why the research synthesis is conducted and what type of methods are best aligned with the pursued goals.
9.5. Concluding Remarks
In light of the increased use of evidence-based practice and research generating stronger evidence ( Grady et al., 2011 ; Lyden et al., 2013 ), review articles have become essential tools for summarizing, synthesizing, integrating or critically appraising prior knowledge in the eHealth field. As mentioned earlier, when rigorously conducted review articles represent powerful information sources for eHealth scholars and practitioners looking for state-of-the-art evidence. The typology of literature reviews we used herein will allow eHealth researchers, graduate students and practitioners to gain a better understanding of the similarities and differences between review types.
We must stress that this classification scheme does not privilege any specific type of review as being of higher quality than another ( Paré et al., 2015 ). As explained above, each type of review has its own strengths and limitations. Having said that, we realize that the methodological rigour of any review — be it qualitative, quantitative or mixed — is a critical aspect that should be considered seriously by prospective authors. In the present context, the notion of rigour refers to the reliability and validity of the review process described in section 9.2. For one thing, reliability is related to the reproducibility of the review process and steps, which is facilitated by a comprehensive documentation of the literature search process, extraction, coding and analysis performed in the review. Whether the search is comprehensive or not, whether it involves a methodical approach for data extraction and synthesis or not, it is important that the review documents in an explicit and transparent manner the steps and approach that were used in the process of its development. Next, validity characterizes the degree to which the review process was conducted appropriately. It goes beyond documentation and reflects decisions related to the selection of the sources, the search terms used, the period of time covered, the articles selected in the search, and the application of backward and forward searches ( vom Brocke et al., 2009 ). In short, the rigour of any review article is reflected by the explicitness of its methods (i.e., transparency) and the soundness of the approach used. We refer those interested in the concepts of rigour and quality to the work of Templier and Paré (2015) which offers a detailed set of methodological guidelines for conducting and evaluating various types of review articles.
To conclude, our main objective in this chapter was to demystify the various types of literature reviews that are central to the continuous development of the eHealth field. It is our hope that our descriptive account will serve as a valuable source for those conducting, evaluating or using reviews in this important and growing domain.
- Ammenwerth E., de Keizer N. An inventory of evaluation studies of information technology in health care. Trends in evaluation research, 1982-2002. International Journal of Medical Informatics. 2004; 44 (1):44–56. [ PubMed : 15778794 ]
- Anderson S., Allen P., Peckham S., Goodwin N. Asking the right questions: scoping studies in the commissioning of research on the organisation and delivery of health services. Health Research Policy and Systems. 2008; 6 (7):1–12. [ PMC free article : PMC2500008 ] [ PubMed : 18613961 ] [ CrossRef ]
- Archer N., Fevrier-Thomas U., Lokker C., McKibbon K. A., Straus S.E. Personal health records: a scoping review. Journal of American Medical Informatics Association. 2011; 18 (4):515–522. [ PMC free article : PMC3128401 ] [ PubMed : 21672914 ]
- Arksey H., O’Malley L. Scoping studies: towards a methodological framework. International Journal of Social Research Methodology. 2005; 8 (1):19–32.
- A systematic, tool-supported method for conducting literature reviews in information systems. Paper presented at the Proceedings of the 19th European Conference on Information Systems ( ecis 2011); June 9 to 11; Helsinki, Finland. 2011.
- Baumeister R. F., Leary M.R. Writing narrative literature reviews. Review of General Psychology. 1997; 1 (3):311–320.
- Becker L. A., Oxman A.D. In: Cochrane handbook for systematic reviews of interventions. Higgins J. P. T., Green S., editors. Hoboken, nj : John Wiley & Sons, Ltd; 2008. Overviews of reviews; pp. 607–631.
- Borenstein M., Hedges L., Higgins J., Rothstein H. Introduction to meta-analysis. Hoboken, nj : John Wiley & Sons Inc; 2009.
- Cook D. J., Mulrow C. D., Haynes B. Systematic reviews: Synthesis of best evidence for clinical decisions. Annals of Internal Medicine. 1997; 126 (5):376–380. [ PubMed : 9054282 ]
- Cooper H., Hedges L.V. In: The handbook of research synthesis and meta-analysis. 2nd ed. Cooper H., Hedges L. V., Valentine J. C., editors. New York: Russell Sage Foundation; 2009. Research synthesis as a scientific process; pp. 3–17.
- Cooper H. M. Organizing knowledge syntheses: A taxonomy of literature reviews. Knowledge in Society. 1988; 1 (1):104–126.
- Cronin P., Ryan F., Coughlan M. Undertaking a literature review: a step-by-step approach. British Journal of Nursing. 2008; 17 (1):38–43. [ PubMed : 18399395 ]
- Darlow S., Wen K.Y. Development testing of mobile health interventions for cancer patient self-management: A review. Health Informatics Journal. 2015 (online before print). [ PubMed : 25916831 ] [ CrossRef ]
- Daudt H. M., van Mossel C., Scott S.J. Enhancing the scoping study methodology: a large, inter-professional team’s experience with Arksey and O’Malley’s framework. bmc Medical Research Methodology. 2013; 13 :48. [ PMC free article : PMC3614526 ] [ PubMed : 23522333 ] [ CrossRef ]
- Davies P. The relevance of systematic reviews to educational policy and practice. Oxford Review of Education. 2000; 26 (3-4):365–378.
- Deeks J. J., Higgins J. P. T., Altman D.G. In: Cochrane handbook for systematic reviews of interventions. Higgins J. P. T., Green S., editors. Hoboken, nj : John Wiley & Sons, Ltd; 2008. Analysing data and undertaking meta-analyses; pp. 243–296.
- Deshazo J. P., Lavallie D. L., Wolf F.M. Publication trends in the medical informatics literature: 20 years of “Medical Informatics” in mesh . bmc Medical Informatics and Decision Making. 2009; 9 :7. [ PMC free article : PMC2652453 ] [ PubMed : 19159472 ] [ CrossRef ]
- Dixon-Woods M., Agarwal S., Jones D., Young B., Sutton A. Synthesising qualitative and quantitative evidence: a review of possible methods. Journal of Health Services Research and Policy. 2005; 10 (1):45–53. [ PubMed : 15667704 ]
- Finfgeld-Connett D., Johnson E.D. Literature search strategies for conducting knowledge-building and theory-generating qualitative systematic reviews. Journal of Advanced Nursing. 2013; 69 (1):194–204. [ PMC free article : PMC3424349 ] [ PubMed : 22591030 ]
- Grady B., Myers K. M., Nelson E. L., Belz N., Bennett L., Carnahan L. … Guidelines Working Group. Evidence-based practice for telemental health. Telemedicine Journal and E Health. 2011; 17 (2):131–148. [ PubMed : 21385026 ]
- Green B. N., Johnson C. D., Adams A. Writing narrative literature reviews for peer-reviewed journals: secrets of the trade. Journal of Chiropractic Medicine. 2006; 5 (3):101–117. [ PMC free article : PMC2647067 ] [ PubMed : 19674681 ]
- Greenhalgh T., Wong G., Westhorp G., Pawson R. Protocol–realist and meta-narrative evidence synthesis: evolving standards ( rameses ). bmc Medical Research Methodology. 2011; 11 :115. [ PMC free article : PMC3173389 ] [ PubMed : 21843376 ]
- Gurol-Urganci I., de Jongh T., Vodopivec-Jamsek V., Atun R., Car J. Mobile phone messaging reminders for attendance at healthcare appointments. Cochrane Database System Review. 2013; 12 cd 007458. [ PMC free article : PMC6485985 ] [ PubMed : 24310741 ] [ CrossRef ]
- Hart C. Doing a literature review: Releasing the social science research imagination. London: SAGE Publications; 1998.
- Higgins J. P. T., Green S., editors. Cochrane handbook for systematic reviews of interventions: Cochrane book series. Hoboken, nj : Wiley-Blackwell; 2008.
- Jesson J., Matheson L., Lacey F.M. Doing your literature review: traditional and systematic techniques. Los Angeles & London: SAGE Publications; 2011.
- King W. R., He J. Understanding the role and methods of meta-analysis in IS research. Communications of the Association for Information Systems. 2005; 16 :1.
- Kirkevold M. Integrative nursing research — an important strategy to further the development of nursing science and nursing practice. Journal of Advanced Nursing. 1997; 25 (5):977–984. [ PubMed : 9147203 ]
- Kitchenham B., Charters S. ebse Technical Report Version 2.3. Keele & Durham. uk : Keele University & University of Durham; 2007. Guidelines for performing systematic literature reviews in software engineering.
- Kitsiou S., Paré G., Jaana M. Systematic reviews and meta-analyses of home telemonitoring interventions for patients with chronic diseases: a critical assessment of their methodological quality. Journal of Medical Internet Research. 2013; 15 (7):e150. [ PMC free article : PMC3785977 ] [ PubMed : 23880072 ]
- Kitsiou S., Paré G., Jaana M. Effects of home telemonitoring interventions on patients with chronic heart failure: an overview of systematic reviews. Journal of Medical Internet Research. 2015; 17 (3):e63. [ PMC free article : PMC4376138 ] [ PubMed : 25768664 ]
- Levac D., Colquhoun H., O’Brien K. K. Scoping studies: advancing the methodology. Implementation Science. 2010; 5 (1):69. [ PMC free article : PMC2954944 ] [ PubMed : 20854677 ]
- Levy Y., Ellis T.J. A systems approach to conduct an effective literature review in support of information systems research. Informing Science. 2006; 9 :181–211.
- Liberati A., Altman D. G., Tetzlaff J., Mulrow C., Gøtzsche P. C., Ioannidis J. P. A. et al. Moher D. The prisma statement for reporting systematic reviews and meta-analyses of studies that evaluate health care interventions: Explanation and elaboration. Annals of Internal Medicine. 2009; 151 (4):W-65. [ PubMed : 19622512 ]
- Lyden J. R., Zickmund S. L., Bhargava T. D., Bryce C. L., Conroy M. B., Fischer G. S. et al. McTigue K. M. Implementing health information technology in a patient-centered manner: Patient experiences with an online evidence-based lifestyle intervention. Journal for Healthcare Quality. 2013; 35 (5):47–57. [ PubMed : 24004039 ]
- Mickan S., Atherton H., Roberts N. W., Heneghan C., Tilson J.K. Use of handheld computers in clinical practice: a systematic review. bmc Medical Informatics and Decision Making. 2014; 14 :56. [ PMC free article : PMC4099138 ] [ PubMed : 24998515 ]
- Moher D. The problem of duplicate systematic reviews. British Medical Journal. 2013; 347 (5040) [ PubMed : 23945367 ] [ CrossRef ]
- Montori V. M., Wilczynski N. L., Morgan D., Haynes R. B., Hedges T. Systematic reviews: a cross-sectional study of location and citation counts. bmc Medicine. 2003; 1 :2. [ PMC free article : PMC281591 ] [ PubMed : 14633274 ]
- Mulrow C. D. The medical review article: state of the science. Annals of Internal Medicine. 1987; 106 (3):485–488. [ PubMed : 3813259 ] [ CrossRef ]
- Evidence-based information systems: A decade later. Proceedings of the European Conference on Information Systems ; 2011. Retrieved from http://aisel .aisnet.org/cgi/viewcontent .cgi?article =1221&context =ecis2011 .
- Okoli C., Schabram K. A guide to conducting a systematic literature review of information systems research. ssrn Electronic Journal. 2010
- Otte-Trojel T., de Bont A., Rundall T. G., van de Klundert J. How outcomes are achieved through patient portals: a realist review. Journal of American Medical Informatics Association. 2014; 21 (4):751–757. [ PMC free article : PMC4078283 ] [ PubMed : 24503882 ]
- Paré G., Trudel M.-C., Jaana M., Kitsiou S. Synthesizing information systems knowledge: A typology of literature reviews. Information & Management. 2015; 52 (2):183–199.
- Patsopoulos N. A., Analatos A. A., Ioannidis J.P. A. Relative citation impact of various study designs in the health sciences. Journal of the American Medical Association. 2005; 293 (19):2362–2366. [ PubMed : 15900006 ]
- Paul M. M., Greene C. M., Newton-Dame R., Thorpe L. E., Perlman S. E., McVeigh K. H., Gourevitch M.N. The state of population health surveillance using electronic health records: A narrative review. Population Health Management. 2015; 18 (3):209–216. [ PubMed : 25608033 ]
- Pawson R. Evidence-based policy: a realist perspective. London: SAGE Publications; 2006.
- Pawson R., Greenhalgh T., Harvey G., Walshe K. Realist review—a new method of systematic review designed for complex policy interventions. Journal of Health Services Research & Policy. 2005; 10 (Suppl 1):21–34. [ PubMed : 16053581 ]
- Petersen K., Vakkalanka S., Kuzniarz L. Guidelines for conducting systematic mapping studies in software engineering: An update. Information and Software Technology. 2015; 64 :1–18.
- Petticrew M., Roberts H. Systematic reviews in the social sciences: A practical guide. Malden, ma : Blackwell Publishing Co; 2006.
- Rousseau D. M., Manning J., Denyer D. Evidence in management and organizational science: Assembling the field’s full weight of scientific knowledge through syntheses. The Academy of Management Annals. 2008; 2 (1):475–515.
- Rowe F. What literature review is not: diversity, boundaries and recommendations. European Journal of Information Systems. 2014; 23 (3):241–255.
- Shea B. J., Hamel C., Wells G. A., Bouter L. M., Kristjansson E., Grimshaw J. et al. Boers M. amstar is a reliable and valid measurement tool to assess the methodological quality of systematic reviews. Journal of Clinical Epidemiology. 2009; 62 (10):1013–1020. [ PubMed : 19230606 ]
- Shepperd S., Lewin S., Straus S., Clarke M., Eccles M. P., Fitzpatrick R. et al. Sheikh A. Can we systematically review studies that evaluate complex interventions? PLoS Medicine. 2009; 6 (8):e1000086. [ PMC free article : PMC2717209 ] [ PubMed : 19668360 ]
- Silva B. M., Rodrigues J. J., de la Torre Díez I., López-Coronado M., Saleem K. Mobile-health: A review of current state in 2015. Journal of Biomedical Informatics. 2015; 56 :265–272. [ PubMed : 26071682 ]
- Smith V., Devane D., Begley C., Clarke M. Methodology in conducting a systematic review of systematic reviews of healthcare interventions. bmc Medical Research Methodology. 2011; 11 (1):15. [ PMC free article : PMC3039637 ] [ PubMed : 21291558 ]
- Sylvester A., Tate M., Johnstone D. Beyond synthesis: re-presenting heterogeneous research literature. Behaviour & Information Technology. 2013; 32 (12):1199–1215.
- Templier M., Paré G. A framework for guiding and evaluating literature reviews. Communications of the Association for Information Systems. 2015; 37 (6):112–137.
- Thomas J., Harden A. Methods for the thematic synthesis of qualitative research in systematic reviews. bmc Medical Research Methodology. 2008; 8 (1):45. [ PMC free article : PMC2478656 ] [ PubMed : 18616818 ]
- Reconstructing the giant: on the importance of rigour in documenting the literature search process. Paper presented at the Proceedings of the 17th European Conference on Information Systems ( ecis 2009); Verona, Italy. 2009.
- Webster J., Watson R.T. Analyzing the past to prepare for the future: Writing a literature review. Management Information Systems Quarterly. 2002; 26 (2):11.
- Whitlock E. P., Lin J. S., Chou R., Shekelle P., Robinson K.A. Using existing systematic reviews in complex systematic reviews. Annals of Internal Medicine. 2008; 148 (10):776–782. [ PubMed : 18490690 ]
This publication is licensed under a Creative Commons License, Attribution-Noncommercial 4.0 International License (CC BY-NC 4.0): see https://creativecommons.org/licenses/by-nc/4.0/
- Cite this Page Paré G, Kitsiou S. Chapter 9 Methods for Literature Reviews. In: Lau F, Kuziemsky C, editors. Handbook of eHealth Evaluation: An Evidence-based Approach [Internet]. Victoria (BC): University of Victoria; 2017 Feb 27.
- PDF version of this title (4.5M)
- Disable Glossary Links
In this Page
- Overview of the Literature Review Process and Steps
- Types of Review Articles and Brief Illustrations
- Concluding Remarks
- PMC PubMed Central citations
- PubMed Links to PubMed
- Chapter 9 Methods for Literature Reviews - Handbook of eHealth Evaluation: An Ev... Chapter 9 Methods for Literature Reviews - Handbook of eHealth Evaluation: An Evidence-based Approach
Your browsing activity is empty.
Activity recording is turned off.
Turn recording back on
Connect with NLM
National Library of Medicine 8600 Rockville Pike Bethesda, MD 20894
Web Policies FOIA HHS Vulnerability Disclosure
Help Accessibility Careers
- Knox College Library
- Research Guides
- What Is It?
- Finding Literature Reviews
A literature review is both a process and a product. As a process, it involves searching for information related to your topic, to familiarize yourself with the relevant research and to identify issues and gaps in the research. In most cases you're seeking to identify the key authors and key arguments that are relevant to your topic, not to exhaustively read everything written on the subject.
Types of Literature Reviews
A stand alone literature review can be a single work in its own right. Examples include:
- A class assignment
- A review article
Literature reviews can also be component parts of larger bodies of work. Examples include:
- A thesis / dissertation
- An academic journal article introduction
What is a Literature Review?
A literature review is the writing process of summarizing, synthesizing and/or critiquing the literature found as a result of a literature search. It may be used as background or context for a primary research project.
There are several reasons to review the literature :
- Identify the developments in the field of study
- Learn about the information sources and the research methodologies
- Find gaps in the literature that can become research questions
- Validate the originality of a research project
- Evaluate the methods
- Identify errors to avoid
- Highlight the strengths, weaknesses and controversies in the field of study
- Identify the subject experts
When writing your review, there are objectives you should keep in mind :
- Inform the audience of the developments in the field
- Establish your credibility
- Discuss the relevance and significance of your question(s)
- Provide the context for your methodological approach
- Discuss the relevance and appropriateness of your approach.
The level of detail or comprehensiveness of your literature review may depend on many things, but especially the purpose and audience of your review. For example, if you're writing a literature review that will aid you in writing a thesis or dissertation, you may want to have a very comprehensive lit review that reviews all relevant literature on a topic, as well as relevant sources beyond what is immediately and freely available (e.g. foundational scholarly articles not available through library collections).
Purpose of a Literature Review
Watch this YouTube video to understand the purpose of a literature review.
- Next: Finding Literature Reviews >>
Research Methods: Literature Reviews
- Annotated Bibliographies
- Literature Reviews
- Scoping Reviews
- Systematic Reviews
- Scholarship of Teaching and Learning
- Persuasive Arguments
- Subject Specific Methodology
A literature review involves researching, reading, analyzing, evaluating, and summarizing scholarly literature (typically journals and articles) about a specific topic. The results of a literature review may be an entire report or article OR may be part of a article, thesis, dissertation, or grant proposal. A literature review helps the author learn about the history and nature of their topic, and identify research gaps and problems.
Steps & Elements
- Determine your topic and its components by asking a question
- Research: locate literature related to your topic to identify the gap(s) that can be addressed
- Read: read the articles or other sources of information
- Analyze: assess the findings for relevancy
- Evaluating: determine how the article are relevant to your research and what are the key findings
- Synthesis: write about the key findings and how it is relevant to your research
Elements of a Literature Review
- Summarize subject, issue or theory under consideration, along with objectives of the review
- Divide works under review into categories (e.g. those in support of a particular position, those against, those offering alternative theories entirely)
- Explain how each work is similar to and how it varies from the others
- Conclude which pieces are best considered in their argument, are most convincing of their opinions, and make the greatest contribution to the understanding and development of an area of research
Writing a Literature Review Resources
- How to Write a Literature Review From the Wesleyan University Library
- Write a Literature Review From the University of California Santa Cruz Library. A Brief overview of a literature review, includes a list of stages for writing a lit review.
- Literature Reviews From the University of North Carolina Writing Center. Detailed information about writing a literature review.
- Undertaking a literature review: a step-by-step approach Cronin, P., Ryan, F., & Coughan, M. (2008). Undertaking a literature review: A step-by-step approach. British Journal of Nursing, 17(1), p.38-43
Literature Review Tutorial
- << Previous: Annotated Bibliographies
- Next: Scoping Reviews >>
- Last Updated: Sep 18, 2023 9:39 AM
- URL: https://guides.auraria.edu/researchmethods
1100 Lawrence Street Denver, CO 80204 303-315-7700 Ask Us Directions
Take the first step and invest in your future.
Offering flexibility & convenience in 51 online degrees & programs.
Featuring 15 intercollegiate NCAA Div II athletic teams.
Find your Fit
UIS has over 85 student and 10 greek life organizations, and many volunteer opportunities.
Arts & Culture
Celebrating the arts to create rich cultural experiences on campus.
Give Like a Star
Your generosity helps fuel fundraising for scholarships, programs and new initiatives.
UIS was listed No. 1 in Illinois and No. 3 in the Midwest in 2023 rankings.
- Quick links Applicants & Students Important Apps & Links Alumni Faculty and Staff Community Admissions How to Apply Cost & Aid Tuition Calculator Registrar Orientation Visit Campus Academics Register for Class Programs of Study Online Degrees & Programs Graduate Education International Student Services Study Away Student Support UIS Life Dining Diversity & Inclusion Get Involved Health & Wellness COVID-19 United in Safety Residence Life Student Life Programs UIS Connection Important Apps UIS Mobile App Advise U Canvas myUIS i-card Balance Pay My Bill - UIS Bursar Self-Service Registration Email Resources Bookstore Box Information Technology Services Library Orbit Policies Webtools Get Connected Area Information Calendar Campus Recreation Departments & Programs (A-Z) Parking UIS Newsroom Connect & Get Involved Update your Info Alumni Events Alumni Networks & Groups Volunteer Opportunities Alumni Board News & Publications Featured Alumni Alumni News UIS Alumni Magazine Resources Order your Transcripts Give Back Alumni Programs Career Development Services & Support Accessibility Services Campus Services Campus Police Facilities & Services Registrar Faculty & Staff Resources Website Project Request Web Services Training & Tools Academic Impressions Career Connect CSA Reporting Cybersecurity Training Faculty Research FERPA Training Website Login Campus Resources Newsroom Campus Calendar Campus Maps i-Card Human Resources Public Relations Webtools Arts & Events UIS Performing Arts Center Visual Arts Gallery Event Calendar Sangamon Experience Center for Lincoln Studies ECCE Speaker Series Community Engagement Center for State Policy and Leadership Illinois Innocence Project Innovate Springfield Central IL Nonprofit Resource Center NPR Illinois Community Resources Child Protection Training Academy Office of Electronic Media University Archives/IRAD Institute for Illinois Public Finance
- Request Info Request info for.... Undergraduate/Graduate Online Study Away Continuing & Professional Education International Student Services General Inquiries
The purpose of a literature review is to collect relevant, timely research on your chosen topic, and synthesize it into a cohesive summary of existing knowledge in the field. This then prepares you for making your own argument on that topic, or for conducting your own original research.
Depending on your field of study, literature reviews can take different forms. Some disciplines require that you synthesize your sources topically, organizing your paragraphs according to how your different sources discuss similar topics. Other disciplines require that you discuss each source in individual paragraphs, covering various aspects in that single article, chapter, or book.
Within your review of a given source, you can cover many different aspects, including (if a research study) the purpose, scope, methods, results, any discussion points, limitations, and implications for future research. Make sure you know which model your professor expects you to follow when writing your own literature reviews.
Tip : Literature reviews may or may not be a graded component of your class or major assignment, but even if it is not, it is a good idea to draft one so that you know the current conversations taking place on your chosen topic. It can better prepare you to write your own, unique argument.
Benefits of Literature Reviews
- Literature reviews allow you to gain familiarity with the current knowledge in your chosen field, as well as the boundaries and limitations of that field.
- Literature reviews also help you to gain an understanding of the theory(ies) driving the field, allowing you to place your research question into context.
- Literature reviews provide an opportunity for you to see and even evaluate successful and unsuccessful assessment and research methods in your field.
- Literature reviews prevent you from duplicating the same information as others writing in your field, allowing you to find your own, unique approach to your topic.
- Literature reviews give you familiarity with the knowledge in your field, giving you the chance to analyze the significance of your additional research.
Choosing Your Sources
When selecting your sources to compile your literature review, make sure you follow these guidelines to ensure you are working with the strongest, most appropriate sources possible.
Find sources within the scope of your topic
Find sources that are not too old for your assignment
Find sources whose authors have authority on your topic
Find sources that meet your instructor’s guidelines (academic, professional, print, etc.)
Tip: Treat your professors and librarians as experts you can turn to for advice on how to locate sources. They are a valuable asset to you, so take advantage of them!
Organizing Your Literature Review
Some assignments require discussing your sources together, in paragraphs organized according to shared topics between them.
For example, in a literature review covering current conversations on Alison Bechdel’s Fun Home , authors may discuss various topics including:
- her graphic style
- her allusions to various literary texts
- her story’s implications regarding LGBT experiences in 20 th century America.
In this case, you would cluster your sources on these three topics. One paragraph would cover how the sources you collected dealt with Bechdel’s graphic style. Another, her allusions. A third, her implications.
Each of these paragraphs would discuss how the sources you found treated these topics in connection to one another. Basically, you compare and contrast how your sources discuss similar issues and points.
To determine these shared topics, examine aspects including:
- Definition of terms
- Common ground
- Issues that divide
- Rhetorical context
Depending on the assignment, your professor may prefer that you discuss each source in your literature review individually (in their own, separate paragraphs or sections). Your professor may give you specific guidelines as far as what to cover in these paragraphs/sections.
If, for instance, your sources are all primary research studies, here are some aspects to consider covering:
Each section of your literature review, in this case, will identify all of these elements for each individual article.
You may or may not need to separate your information into multiple paragraphs for each source. If you do, using proper headings in the appropriate citation style (APA, MLA, etc.) will help keep you organized.
If you are writing a literature review as part of a larger assignment, you generally do not need an introduction and/or conclusion, because it is embedded within the context of your larger paper.
If, however, your literature review is a standalone assignment, it is a good idea to include some sort of introduction and conclusion to provide your reader with context regarding your topic, purpose, and any relevant implications or further questions. Make sure you know what your professor is expecting for your literature review’s content.
Typically, a literature review concludes with a full bibliography of your included sources. Make sure you use the style guide required by your professor for this assignment.
Want to create or adapt books like this? Learn more about how Pressbooks supports open publishing practices.
The Literature Review
28 The Literature Review
A literature review is a survey of everything that has been written about a particular topic, theory, or research question. The word “literature” means “sources of information”. The literature will inform you about the research that has already been conducted on your chosen subject. This is important because we do not want to repeat research that has already been done unless there is a good reason for doing so (i.e. there has been a new development in this area or testing a theory with a new population, or even just to see if the research can be reproduced). Literature reviews usually serve as a background for a larger work (e.g. as part of a research proposal), or it may stand on its own. Much more than a simple list of sources, an effective literature review analyzes and synthesizes information about key themes or issues.
Purpose of a literature review
The literature review involves an extensive study of research publications, books and other documents related to the defined problem. The study is important because it advises you, as a researcher, whether the problem you identified has already been solved by other researchers. It also advises you as to the status of the problem, techniques that have been used by other researchers to investigate the problem, and other related details.
A literature review goes beyond the search for information and includes the identification and articulation of relationships between existing literature and your field of research. The literature review enables the researcher to discover what has been already been written about a topic and to understand the relationship between the various contributions. This will enable the researcher to determine the contributions of each sources (books, article, etc.) to the topic. Literature reviews also enable the researcher to identify and (if possible) resolve contradictions, and also determine research gaps and/or unanswered questions.
Even though the nature of the literature review may vary with different types of studies, the basic purposes remain constant and could be summarized as follows:
- Provide a context for your research;
- Justify the research you are proposing;
- Ensure that your proposed research has not been carried out by another person (and if you find it has, then your literature review should specify why replication is necessary);
- Show where your proposed research fits into the existing body of knowledge;
- Enable the researcher to learn from previous theory on the subject;
- Illustrate how the subject has been studied previously;
- Highlight flaws in previous research;
- Outline gaps in previous research;
- Show how your proposed research can add to the understanding and knowledge of the field;
- Help refine, refocus, or even move the topic in a new direction
What is involved in writing a literature review?
- Research – to discover what has been written about the topic;
- Critical Appraisal – to evaluate the literature, determine the relationship between the sources and ascertain what has been done already and what still needs to be done;
- Writing – to explain what you have found
Generally speaking, it is helpful to think of the literature review as a funnel. One starts with a broad examination of the research related to the issue, working down to look at more specific aspects of the issue, which leads to the gap or the specific issue that your research will address.
How to undertake a literature review
The first step in undertaking a literature review is to conduct a library search of academic research that has been done on your topic. This can be done electronically, or if you are within close vicinity to a library, you can go in and use their computers to find electronic and print holdings. You can also use Google Scholar for your search. In some cases, research conducted outside academia can serve as an important research source for your literature review. Indeed, such research can have important practical implications, as opposed to academic research which usually (although not always) tends toward theoretical applications.
However, it is important to understand who funded the research you review, in addition to the perspective and the purpose of the research. This is becoming an issue in Canada as universities and colleges increasingly turn to industry for research funding grants (see How TransAlta used a university-sanctioned research project to lobby for the coal industry ).
As part of this first step there are a few more some things to be thinking about as you review the literature, as follows:
- Who are the various researchers who have studied this topic? Who are the most prolific researchers/writers on this topic? Has a specific researcher or teams of researchers been identified as pioneers or leaders in this field of study?
- How have the various researchers defined key terms that are relevant to your topic? Have the definitions of any of the key terms evolved over time?
- What are the different theories that have been examined and applied to this topic? How, if at all, have the various theories applied to this topic over time evolved?
- What methodologies have been used to study this topic? Have the methodologies evolved over time?
In addition to thinking about these questions, you should be taking notes during this process. It can be helpful to keep these notes in an Excel file. For example, your notes should include the following information:
- If the article is empirical, write down the results of the research study in one or two sentences of your own words. e.g. “people who are between ages 18 – 35 are more likely to own a smart phone than those above or below.” It is also a good idea to make note of the methods, the research design, the number of participants and details on the sample used in the study. Sometimes, you may even want to write down the names of the statistical procedures used to analyze the data or even some of the statistics, depending on your assignment.
- If the article is a review of previous research, look for the main points. It may be helpful to read or skim the whole article, look away, and ask yourself what you felt was the main idea.
- Write down any limitations or gaps you notice, anything that seems to contradict something you read elsewhere, or just anything that you think is important or interesting (Adjei, n.d.)
When reading through your sources, remember that you are looking for the “big picture,” not a collection of random, separate articles (an annotated bibliography). You are also not trying to prove a point (an essay). You are looking for common themes and patterns in the research as a whole. You are also looking to see how the various pieces of research are linked, if at all. As part of this process, you also want to identify research gaps or areas that require further research related your topic (Adjei, n.d.). In this regard, you cannot be expected to be an expert on your topic. A suggestion for finding gaps is to read the conclusion section of the academic journal articles and conference proceedings your search has uncovered. Researchers often identify gaps in the research in their conclusion. They may even suggest areas for future research. However, remember, if a researcher suggested a gap 10 years ago, it is likely that the gap has now been addressed. To find a gap, look at the most recent research your literature review has uncovered (within 2-3 years of the current date). At this point in your search of the literature, you may realize that your research question needs to change or adapt. This is a fairly common occurrence, as when you first develop a research question, you cannot be sure what the status of the research area is, until you undertake your review of the literature related to this topic. Finally, it is worth mentioning that it is very likely you will not include all of the resources you have read in your literature review. If you are asked to include 20 resources in your literature review, for example, expect to read approximately 30.
How to write a literature review
There are three parts to the literature review: the introduction, the body, and the conclusion. In the following paragraphs we outline what to include in each of these sections. This section concludes with a variety of resources for you to check out.
- The introduction must identify the topic by briefly discussing the significance of the topic including a statement that outlines the conclusion to be drawn from the literature review.
- If your literature review is part of a larger work, explain the importance of the review to your research question.
- Defend the importance of the topic by giving a broad overview of the scope of the work you are reviewing. For example, if you are interested in post traumatic stress disorder (PTSD) in paramedics, you might provide some stats to prove how much work time is lost by those suffering from PTSD.
- Clarify whether you are looking at the entire history of the field, or just a particular period of time.
- Discuss and assess the research according to specific organizational principles (see examples below), rather than addressing each source separately. Most, if not all, paragraphs should discuss more than one source. Avoid addressing your sources alphabetically as this does not assist in developing the themes or key issues central to your review.
- Compare, contrast, and connect the various pieces of research. Much of the research you are reading should be connected. Or you may notice various themes within the research (i.e. effects of PTSD on sick time, effects of PTSD on families of paramedics, effects of PTSD on overall paramedic wellness, etc.). If you have undertaken a thorough review of the literature, you should start to see the bigger picture of how the research on this topic has evolved over time, who the main researchers are on this topic, how the methods and theories related to this topic have changed (if at all), over time.
- Summarize the works you are reviewing. Just as in any written assignment, use logical organization and clear transitions. Spend more time on the researchers and bodies of research that are considered most important in the field and/or that are most relevant.
Based upon your research, suggest where the research in the field will or should go next. If you are proposing your own research study, show how you will contribute to the field and fill in any gaps. The conclusion would also be a good place to defend the importance of the topic, now that you have demonstrated the current state of thinking in the field.
Other resources to help you write a literature review
In conclusion, there is a plethora of resources, both here and online, that provide information on how to write a literature review. For example, check out these three, very helpful YouTube videos prepared by a professor at the University of Maryland, in the U.S.A:
- The Literature Review, Part 1
- The Literature Review, Part 2
- The Literature Review, Part 3
Table 5.1 also provides some suggested organizational techniques, as well as instances when you might use these various techniques. The table also provides a writing sample to demonstrate the writing technique.
And remember, most university and college libraries also have valuable information on literature reviews. Here is the link to one such website: JIBC Literature Review information [PDF]
Acceptable sources for literature reviews
There are a few acceptable sources for literature reviews, and I will list these in order from what will be considered most acceptable to less acceptable sources for your literature review assignments, as follows:
- Peer reviewed journal articles;
- Edited academic books;
- Articles in professional journals;
- Statistical data from government websites;
- Website material from professional associations (use sparingly and carefully);
The following sections will explain and provide examples of these various sources.
Peer reviewed journal articles (papers)
A peer reviewed journal article is a paper that has been submitted to a scholarly journal, accepted, and published. Peer review journal papers go through a rigorous, blind review process of peer review. What this means is that two to three experts in the area of research featured in the paper have reviewed and accepted the paper for publication. The names of the author(s) who are seeking to publish the research have been removed (blind review), so as to minimize any bias towards the authors of the research. Albeit, sometimes a savvy reviewer can discern who has done the research based upon previous publications, etc. This blind review process can be long (often 12 to 18 months) that may involve many back and forth edits on the behalf of the researchers, as they work to address the edits and concerns of the peers who reviewed their paper. Often, reviewers will reject the paper for a variety of reasons, such as unclear or questionable methods, lack of contribution to the field, etc. Because peer reviewed journal articles have gone through a rigorous process of review, they are considered to be the premier source for research. Peer reviewed journal articles should serve as the foundation for your literature review.
The following website will provide more information on peer-reviewed journal articles: Evaluating Information Sources: What Is A Peer-Reviewed Article? Make sure you watch the little video on the upper left-hand side of your screen, in addition to reading the material.
Edited academic books
An edited academic book is a collection of scholarly scientific papers written by different authors. The works are original papers, not published elsewhere (Wikipedia, 2018). The papers within the text also go through a process of review; however, the review is often not a blind review because the authors have been invited to contribute to the book. Consequently, edited academic books are fine to use for your literature review, but you also want to ensure that your literature review contains mostly peer reviewed journal papers.
Articles in professional journals.
Articles from professional journals should be used with caution, as far as it relates to a source for your literature review. This is because articles in trade journals are not usually peer reviewed, even though they may appear as such. A good way to find out is to read the “About us” section of the professional journal. They should state there if the papers are peer reviewed. You can also google the name of the journal and add peer reviewed to the search and you should be able to find out that way.
Statistical data from governmental websites.
Governmental websites can be excellent sources for statistical data. For examples, Statistics Canada collects and publishes data related to the economy, society, and the environment.
Website material from professional associations
Material from other websites can also serve as a source for statistics that you may need for your literature review. As you want to justify the value of the research you are interested in, you might make use of a professional association´s website to learn how many members they have, for example. As a hypothetical example, you might want to demonstrate, as part of the introduction to your literature review, why more research on the topic of PTSD in police officers is important. You could use peer reviewed journal articles to determine the prevalence of PTSD in police officers in Canada in the last ten years and then use the Ontario Police Officers´ Association website to determine the approximate number of police officers employed in the Province of Ontario over the last ten years. This might help you create an approximation of how many police officers could be suffering with PTSD in Ontario. That number could potentially help to justify a research grand down the road. But again, this type of website-based material should be used with caution and sparingly.
- This chapter has been adapted from Unit 2: Literature Review in Research Methods by Joseph K. Adjei. © Creative Commons Attribution-ShareAlike 3.0 License .
The Literature Review by Valerie A. Sheppard is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License , except where otherwise noted.
Share This Book
- Become Involved |
- Give to the Library |
- Staff Directory |
- UNF Library
- Thomas G. Carpenter Library
Conducting a Literature Review
Benefits of conducting a literature review.
- Steps in Conducting a Literature Review
- Summary of the Process
- Additional Resources
- Literature Review Tutorial by American University Library
- The Literature Review: A Few Tips On Conducting It by University of Toronto
- Write a Literature Review by UC Santa Cruz University Library
While there might be many reasons for conducting a literature review, following are four key outcomes of doing the review.
Assessment of the current state of research on a topic . This is probably the most obvious value of the literature review. Once a researcher has determined an area to work with for a research project, a search of relevant information sources will help determine what is already known about the topic and how extensively the topic has already been researched.
Identification of the experts on a particular topic . One of the additional benefits derived from doing the literature review is that it will quickly reveal which researchers have written the most on a particular topic and are, therefore, probably the experts on the topic. Someone who has written twenty articles on a topic or on related topics is more than likely more knowledgeable than someone who has written a single article. This same writer will likely turn up as a reference in most of the other articles written on the same topic. From the number of articles written by the author and the number of times the writer has been cited by other authors, a researcher will be able to assume that the particular author is an expert in the area and, thus, a key resource for consultation in the current research to be undertaken.
Identification of key questions about a topic that need further research . In many cases a researcher may discover new angles that need further exploration by reviewing what has already been written on a topic. For example, research may suggest that listening to music while studying might lead to better retention of ideas, but the research might not have assessed whether a particular style of music is more beneficial than another. A researcher who is interested in pursuing this topic would then do well to follow up existing studies with a new study, based on previous research, that tries to identify which styles of music are most beneficial to retention.
Determination of methodologies used in past studies of the same or similar topics. It is often useful to review the types of studies that previous researchers have launched as a means of determining what approaches might be of most benefit in further developing a topic. By the same token, a review of previously conducted studies might lend itself to researchers determining a new angle for approaching research.
Upon completion of the literature review, a researcher should have a solid foundation of knowledge in the area and a good feel for the direction any new research should take. Should any additional questions arise during the course of the research, the researcher will know which experts to consult in order to quickly clear up those questions.
- << Previous: Home
- Next: Steps in Conducting a Literature Review >>
- Last Updated: Aug 29, 2022 8:54 AM
- URL: https://libguides.unf.edu/litreview