What is a systematic review? What are they good for?


This is a question often posed by our science and social science students, so I like to refer to Cochrane Library for definitions:

“A systematic review is a high-level overview of primary research on a particular research question that tries to identify, select, synthesize and appraise all high quality research evidence relevant to that question in order to answer it” (Cochrane, 2014)

Another definition they give is:

“A systematic review attempts to identify, appraise and synthesize all the empirical evidence that meets pre-specified eligibility criteria to answer a given research question. Researchers conducting systematic reviews use explicit methods aimed at minimizing bias, in order to produce more reliable findings that can be used to inform decision making” (Cochrane, 2014).

Systematic reviews are also a type of journal article, published alongside primary research articles in scholarly journals.

 

How do you know if a journal article is a systematic review?

Systematic reviews will identify themselves  as such in the abstract, introduction, and/or the methods (sometimes even in the title). For example, you could find language like “the purpose of this systematic review…”  like the example below

Systematic Review Abstract

Systematic reviews will also often include a methods section: the authors will list what databases they searched, what terms or keywords they used, the kinds of articles they included and excluded, etc. I’ve included an example below from the open-access Journal of Neurosciences in Renal Practice:

Systematic Review - Methods

Reviews that are not systematic don’t have these distinguishing features. If you’re not sure what kind of article something is, make sure to ask a librarian or your professor.

 

When would you want to use a systematic review?

Even though systematic reviews are secondary sources they are often considered more credible than regular literature reviews: this is because the authors have systematically found every article that falls into their research parameters. In a regular literature review, we can’t be sure if the authors have missed important research.

Systematic reviews are a great way to start off your research: if you’re able to find one on your exact topic, you can thank your lucky stars! We recommend students use the reference lists from systematic reviews to help identify primary research on their topics. Rather than digging through databases to get an overview on your topic, systematic reviews provide a starting list of primary research articles that you can then go find. Then you can just fill in the gaps!

That said, it’s important to pay attention to the methods of the review: what dates did they examine? Have more studies been published since the systematic review was written? Did they exclude any studies that you should probably consider? Reading a systematic review carefully will help you with determining what research you need to do next.

 

When should you not use a systematic review?

Systematic reviews are still secondary sources: the authors are not contributing new information or understanding to the field, but rather summarizing and combining the results of others’ research. Even though the authors take efforts to eliminate bias, there is no guarantee they have analyzed the studies correctly or from your perspective on the topic (they’re still humans, after all). Make sure to read them with a critical eye. If your professor has asked you to use primary research only, make sure to ask before using a systematic review in your reference list.

How do I determine if a source is ‘primary’?


Thanks to our weekend librarian Josh Klar for this helpful post! – HC

In science, a primary source article reports on an empirical research study conducted by the authors of the article. Secondary sources, usually reviews, are often summarizing research that was done by other people.

A few good ways to identify a primary source article include looking for the following in the abstract or in the body of the article:

  • A reference to the “study” that was conducted by the authors:Abstract
  • A reference to the “research method” used to conduct the study (Sometimes this will not be discussed in the abstract and will be found in the article under “Method” or “Methodology”):Methods
    Secondary sources in science – again, usually reviews – sometimes include methods, too, so make sure to read this section carefully. Primary articles will refer to the kind of study that was conducted: clinical trial, randomized controlled trial, observational study, etc.
  • Primary articles reference the “results” discovered through the study, too – See Example 1 above again. This is where the authors list their contribution to the overall understanding of this topic in the literature.

 

In social sciences, humanities or the arts, sometimes it can be a bit trickier to determine if material is a primary source. In these disciplines, a primary source is any material that was produced by eyewitnesses to or participants in an event, and typically the material was written or created during the time of study or observation. Think of a photograph or a page from a diary. These are records of a moment in time and are examples of primary sources. Other examples could be: Memoirs, speeches, manuscripts, letters, interviews, news film footage, autobiographies, ethnographies or research data from a study. These primary sources all serve as raw material to interpret social, cultural, or historical events.

And just remember… any source that arrives at conclusions based on research from other studies are NOT primary sources. A few examples would include

  • Literature reviews
  • Encyclopedia articles
  • Many textbooks

 

How many resources should be in an academic paper?


Note: this post was written by guest editor Alison Foster. Thanks Alison! – HC

If we received this question at the library desk, our first response is always: “ask your prof.”  They’re the ones grading your assignments and are the ultimate authority. That said, students will often ask us for our “professional opinion,” so I’ll give a few tips below.

In my experience, there is no set number of resources required for a paper. Often, scholars will include a large volume of resources, in order to address the major points made in their research paper and support their argument/thesis. In the case of student assignments, you will often be asked to include a minimum number of assignments, but rarely asked to limit yourself (always check your assignment guidelines!). Overall, my response is: include as many resources as you need to support your ideas!

A great way to determine if you have included an appropriate amount of resources is to ask yourself these questions:

  1. Have I backed up all of my key points with relevant and credible resources? Are there any gaps or unsupported statements in my essay?
  2. Am I relying too heavily on a source? Do I have 2-3 sources to back up each of my key points?
  3. If I have some unsupported points in my essay, could I back them up using resources that I have already cited/read?
  4. If my paper addresses a controversial issue or topic, have I included resources that address both sides of the issue?

Looking over your work for information gaps is a valuable skill that will help you in your present and future research!

How do you write an annotated bibliography?


The theory is the same behind all annotated bibliographies, but the practice may be different for each professor.  Above all: read your assignment instructions!  They may be slightly different from my rough guidelines below. 

In general, annotated bibliographies contain two things:

  1. The full citation, in proper citation format, of each resource you are annotating (e.g. books, articles, websites, etc).
  2. One paragraph (or “annotation”) of each source, underneath the full citation.

What should be included in your annotations?  This is where your professors may disagree.  But commonly students are asked to include:

  1. A VERY brief description of the resource’s content.  This is often just 1-2 sentences summarizing the main argument of the source.
  2. A critique of the source.  Things you could look at:
    • Currency
    • Author credentials
    • Author Bias/Perspective/Holes in their argument
    • Bibliography/References used by the author
    • Comparison between this source and the other ones in your annotated bibliography
    • Overall assessment of strengths and weaknesses
  3. A description of how your source is useful for your assignment.  This could include how this resource contributes to your overall understanding of your topic

Again, the above three points are usually done in just one paragraph.

Here are some super helpful links and handouts on writing annotated bibliographies:

Need help practicing your annotations?  Here is a worksheet from a past Brescia Foods and Nutrition class:

Some sample annotated bibliographies to get you started:

How do I read a journal article? I don’t want to read the whole thing every time!


A lot of students feel overwhelmed by research because they have to read what feels like a million articles before they find a few good ones. Fear not!  You won’t need to read every article word-for-word (although you will have to read some).  My response is catered to primary research articles, but the theory could easily be applied to articles from other disciplines, like history or political science.

  1. Read the title and the abstract: does it apply to your research?  If yes, move on to the next step.  If no, don’t bother reading this article.
  2. Read the introduction and conclusion: do you still think you can use it in your assignment?  If yes, move on to the next step.  If no, get rid of this article so it doesn’t confuse you in the future.
  3. Read the discussion ( the section right before the conclusion):  is there anything of value here?  Do you still think you can use this article in your assignment?   If no, get rid of this article.  If yes, you can probably use this article in your bibliography.  The next few steps will just give you more information.
  4. Read the results: start by reading the first sentence in each paragraph to identify which sections are valuable for you.   The results, though, will give you details on the “discussion” portion of the article and will likely give you some good content.  That said, start by reading only the first sentence in each paragraph of the results to
  5. Check the literature review or scan the bibliography: if you’ve made it this far, you will likely find other articles worth reading in the lit review or bibliography.  This is called “citation mining” and is an efficient way of finding useful resources.
  6. Read whatever is left: you may have missed something good! You probably haven’t read the methodology yet, which is helpful if you’re looking to repeat the study or identify how this article is similar to others you’ve read.

This order is only my personal opinion: make sure to check out more reliable sources, too!

Can I trust the books and articles I find from Western’s websites? Are they reliable?


This is a tricky question to answer.  This was posed to me by a history student, so my answer is geared to their needs:

Western is an academic institution: we are not trying to collect every book on the planet but rather the materials that support the learning and research of our students, staff and faculty.  That said, it is possible that there are books in our collection that may not be “credible” for your essay, depending on your topic.  Part of academia is keeping a record of past interpretations of different topics, so things will not be thrown out just because they’re not contemporary thought.  And, as my husband says, anybody can write a book.

So, if you’re worried about the credibility of books, make sure to look at them with a critical eye.  This doesn’t have to be a long, complicated process, but just glancing over a few things (i.e. who wrote it, when was it published, etc).  I have another blog post about simple ways to evaluate resources.

As for journal articles, it’s easier to make sure that you’re finding good quality stuff while you’re doing your research.  If you’re searching in an academic database, usually there’s a “peer-reviewed” option that you can check.  Choosing this option will only look at scholarly material that has gone through the peer-review process, rather than having other stuff mixed in (i.e. book reviews, trade magazines, etc). I’ve included a screenshot of Historical Abstracts so you can see what I mean.

HistoricalAbstracts_peerreviewed_cleverresearcher

Summon has a similar option, but it’s called “Limit to articles from scholarly publications.”

Regardless of what type of resource you’re researching, though, make sure to use a critical eye and read what you’ve found!

Can I use popular periodicals (like magazines) in my essay?


This one-hundred-percent depends on your professor.  If you haven’t asked them yet, or if you haven’t read your assignment instructions, do that first before reading any further.

If you would prefer to take my advice, my answer to this question would be: no.  If you have any doubt whether it’s appropriate, it’s better to play it safe.

What is a “popular periodical” some of you may ask?  These would be things like popular magazines (Macleans, Time, People), trade magazines (Foodservice and Hospitality) or newspapers.  Academic periodicals include journal articles.

Consider your assignment, too: if you’re writing a thesis-based research essay, citing from US weekly probably isn’t the best idea.  But for some assignments – i.e. especially for MOS students – it’s advisable to look at trade magazines.

So, make sure to ask your prof!