What is a systematic review? What are they good for?

This is a question often posed by our science and social science students, so I like to refer to Cochrane Library for definitions:

“A systematic review is a high-level overview of primary research on a particular research question that tries to identify, select, synthesize and appraise all high quality research evidence relevant to that question in order to answer it” (Cochrane, 2014)

Another definition they give is:

“A systematic review attempts to identify, appraise and synthesize all the empirical evidence that meets pre-specified eligibility criteria to answer a given research question. Researchers conducting systematic reviews use explicit methods aimed at minimizing bias, in order to produce more reliable findings that can be used to inform decision making” (Cochrane, 2014).

Systematic reviews are also a type of journal article, published alongside primary research articles in scholarly journals.


How do you know if a journal article is a systematic review?

Systematic reviews will identify themselves  as such in the abstract, introduction, and/or the methods (sometimes even in the title). For example, you could find language like “the purpose of this systematic review…”  like the example below

Systematic Review Abstract

Systematic reviews will also often include a methods section: the authors will list what databases they searched, what terms or keywords they used, the kinds of articles they included and excluded, etc. I’ve included an example below from the open-access Journal of Neurosciences in Renal Practice:

Systematic Review - Methods

Reviews that are not systematic don’t have these distinguishing features. If you’re not sure what kind of article something is, make sure to ask a librarian or your professor.


When would you want to use a systematic review?

Even though systematic reviews are secondary sources they are often considered more credible than regular literature reviews: this is because the authors have systematically found every article that falls into their research parameters. In a regular literature review, we can’t be sure if the authors have missed important research.

Systematic reviews are a great way to start off your research: if you’re able to find one on your exact topic, you can thank your lucky stars! We recommend students use the reference lists from systematic reviews to help identify primary research on their topics. Rather than digging through databases to get an overview on your topic, systematic reviews provide a starting list of primary research articles that you can then go find. Then you can just fill in the gaps!

That said, it’s important to pay attention to the methods of the review: what dates did they examine? Have more studies been published since the systematic review was written? Did they exclude any studies that you should probably consider? Reading a systematic review carefully will help you with determining what research you need to do next.


When should you not use a systematic review?

Systematic reviews are still secondary sources: the authors are not contributing new information or understanding to the field, but rather summarizing and combining the results of others’ research. Even though the authors take efforts to eliminate bias, there is no guarantee they have analyzed the studies correctly or from your perspective on the topic (they’re still humans, after all). Make sure to read them with a critical eye. If your professor has asked you to use primary research only, make sure to ask before using a systematic review in your reference list.

How do I determine if a source is ‘primary’?

Thanks to our weekend librarian Josh Klar for this helpful post! – HC

In science, a primary source article reports on an empirical research study conducted by the authors of the article. Secondary sources, usually reviews, are often summarizing research that was done by other people.

A few good ways to identify a primary source article include looking for the following in the abstract or in the body of the article:

  • A reference to the “study” that was conducted by the authors:Abstract
  • A reference to the “research method” used to conduct the study (Sometimes this will not be discussed in the abstract and will be found in the article under “Method” or “Methodology”):Methods
    Secondary sources in science – again, usually reviews – sometimes include methods, too, so make sure to read this section carefully. Primary articles will refer to the kind of study that was conducted: clinical trial, randomized controlled trial, observational study, etc.
  • Primary articles reference the “results” discovered through the study, too – See Example 1 above again. This is where the authors list their contribution to the overall understanding of this topic in the literature.


In social sciences, humanities or the arts, sometimes it can be a bit trickier to determine if material is a primary source. In these disciplines, a primary source is any material that was produced by eyewitnesses to or participants in an event, and typically the material was written or created during the time of study or observation. Think of a photograph or a page from a diary. These are records of a moment in time and are examples of primary sources. Other examples could be: Memoirs, speeches, manuscripts, letters, interviews, news film footage, autobiographies, ethnographies or research data from a study. These primary sources all serve as raw material to interpret social, cultural, or historical events.

And just remember… any source that arrives at conclusions based on research from other studies are NOT primary sources. A few examples would include

  • Literature reviews
  • Encyclopedia articles
  • Many textbooks


How do I find a journal abbreviation?

Thanks to our Saturday librarian Marg Baltzer for this helpful post! – HC

The citation style that I have to use abbreviates the name of the journal. Where can I find the correct abbreviation for the journals that I cite in my paper?

There are four main sources you can use to determine the correct abbreviation. If one source does not list the abbreviation you are looking for, try the next one on the list! (If you are off campus, make sure you login first)

If you try all four sources and still can’t find your abbreviated journal title, don’t make it up! Some journal titles aren’t abbreviated, including some that are just one word titles. Talk with your professor to know how you should proceed!

  1. Ulrich’s Periodical Directory. When you login to Ulrich’s all you need to do is search for the journal name, click on the title and then scroll down to, and click on ‘additional title details’. It may or may not have the abbreviation listed.
  2. Web of Science. In Web of Science, click on the arrow right beside ‘basic search’
    Basic search arrowFrom the drop down list, choose ‘cited reference search’. Under the second search box you will see “view abbreviation list”. From this page, you can navigate to the required journal title
  3. NLM Catalog. On the search page of the National Library of Medicine (PubMed), find and click on ‘Journals in NCBI Databases”. Enter your journal title and the results will give you the correct abbreviation.
    4. Journal Seek. In JournalSeek , simply enter your journal title and if the journal is in their database, the search results will give you the correct abbreviation

My prof suggested I search for articles in a specific journal: what does that mean?

Note: this post is jargon-heavy and aimed at students who have a little bit of research experience. Make sure to check out our other posts on journals and finding articles if you’re not sure what some of the terminology means. HC.

When you get to know a topic or a discipline, the same journal titles will keep popping up. You may notice this if you do a few research assignments on the same topic, or once you reach fourth year and have studied the same discipline for awhile. For experts in the field, like your professors, they’ll know which journal titles are likely to cover different topics within your discipline. This is often why they’ll suggest to you “look in Leadership Quarterly” or “check the Journal of Social and Clinical Psychology,” because they know the kinds of titles each journal will publish.

So: knowing how to access articles just from a specific journal title can be an important skill. Luckily the steps aren’t all that different from searching a database, you just need to know where to start. Off campus users: make sure to log in before you start!

  1. Start at the catalogue. There are lots of other ways to start, but this way will always connect you.
  2. Search by Journal Title. So, type in “leadership quarterly” or whatever journal you’re searching. Make sure to click the “Journal Title” radio button on the catalogue before hitting Search.
  3. CaptureLook for the “Click here for online access links.” There may be more than one, depending whether we pay for an online subscription for this journal from multiple places. Choose the link you think is most suitable for your needs. A warning, though: the date ranges next to the links aren’t always accurate!Capture 2
  4. Search for articles! Make sure the option “Limit Search to this Journal” or “Only this Journal” is checked.Capture 3Alternatively, you can browse by date or issue to see what’s been published recently. The best part: you don’t have to go searching for full-text articles!

How do I cite court decisions or case law?

This question came to me from students in Psychology 3313, but I’ve tried to make my answers as general as possible! – HC

Citing legal information can feel like a really un-fun treasure hunt: our APA citation guide, for example, refers you to the Publication Manual, which refers you to Appendix 7.1 which refers to you to the Bluebook. Then when you look at the Bluebook the examples are American. Annoying.

So, first thing’s first:

  1. What country is your court decision/case from? If it’s American use the Bluebook: A Uniform System of Citation. If it’s Canadian, use the Canadian Guide to Uniform Legal Citation instead. Either way, these resources are located in the library’s reference section.  If it’s from outside Canada or the US I would suggest asking a librarian or emailing the Law Library’s reference desk.
  2. Build your citation. If you find the books difficult to understand (e.g. if you’re inexperienced with legal terminology, like me), I recommend checking out one of the following helpful resources for actually creating your citation. These guides are helpful as they break down the instructions in the Canadian Guide and the Bluebook.

Canadian guides:

    1. Queen’s University Library – Guide to Canadian Legal Citation
    2. Carlton University Library – Citing Legal Sources (see p. 2 “Citing Jurisprudence”)

American guides:

    1. Cornell University – Introduction to Basic Legal Citation (browse menu on the left)
    2. Georgetown – Citing Cases

If you’d like some additional assistance from a librarian, feel free to contact the Law Library (or come visit us, of course!)

Getting relevant results: Improving your topic

Often when students are struggling with irrelevant research results it’s actually their topic that needs work. For most, their topics are either too broad or too narrow (see examples below).  There are a few things you can do, though, to help improve your topic.

Is your topic SMART?

  1. Specific – Is your topic too broad for the scope of your assignment (e.g. How can you be an ethical leader)? Too narrow (Steve Jobs and the ethical leadership of Apple)? Or just right (e.g. Is being ethical an essential quality of leadership)?
  2. Measurable – What will you be discussing or measuring throughout your paper? (In what ways is X affected by Y? How is X related to Y? What is the impact of X? How are X and Y related?)
  3. Attainable – Can you answer your research question within the limits of your paper? (e.g. You have 7-9 pages; at least 2 pages will be taken up by your introduction, background information, and conclusions. So you have 5-7 pages to actually answer your research question).
  4. Relevant – Is your topic related to the course? Does it meet the expectations of your professor? Are you interested in it?
  5. Timely – Can you answer the question, get all the research you need, and write the assignment before the due date? Also, is your topic too recent? Can there feasibly be academic research available on it?

Try turning your topic idea into a researchable question:

  1. Ask the 5W’s of your topic
    Who – what population are you talking about? (e.g. university students)
    What – what about that population? (that they’re stressed and don’t eat well as a result)
    Where – what geographical area are you focusing on? (e.g. Canada)
    When – what time period will you focus on? (e.g. last twenty years)
    Why – why does this topic matter? What does it have to do with your assignment/course?  (e.g. if you can’t answer all 5W’s you will need to do more background research first!
  2. Then ask yourself ‘How’?
    This is the “measurable” part of your SMART question (How are X and Y related; How does X influence Y)
    Asking the question HOW about your 5W’s can often help you develop a great research question!How does stress affect the eating habits of university students in Canada? (Note: I didn’t use “when” but that’s okay I think!)

And, just in case they’re helpful, here are some common types of research questions (Note: these ideas aren’t mine so please cite the original author, reference below)!

  • Definitions (While many people think X is a Y, can it better thought of as a Z?)
  • Evaluation (Can you argue that your person/activity/quality/thing is good, better or best (or bad, worse, or worst) compared to its peers?
  • Compare/contrast (Are there things/elements/people who are similar or dissimilar to the one you are thinking about?)
  • Explanation (Obvious causes of X; How Y would not have happened without X; Alternative causes to X than what is currently assumed)
  • Counter argument (Argue against what is currently assumed to be true)
  • Justification (Why does something matter?  Argue for something people haven’t previously considered that you think matters a great deal)

Source: Hacker, Diane. (2009). A Canadian writer’s reference, 4th ed. Boston: Bedford.

What is a DOI?

DOI is an acronym for “digital object identifier”, meaning a “digital identifier of an object” rather than an “identifier of a digital object”.

The DOI system is designed to work over the Internet. A DOI name is permanently assigned to an object (e.g. a journal article) to provide a persistent link to current information about that item, including where the object (or information about it) can be found on the Internet. While information about an object can change over time, its DOI name will not change.

For more information about DOI’s, visit the DOI website: http://www.doi.org/.

We’re sometimes asked: are DOI’s permanent or can they be changed? (e.g. when a journal is made available in pre-publication and then gets published in a journal).  The IDF does not have any specific rules on this (we could only find reference that “Individual Registering Agencies (RAs) adopt appropriate rules for their community and application”). In general, if substantial changes are made to a document or it is necessary to identify both the original and the changed material, a new DOI is assigned.  It’s safe to assume, then, that *most* of the time DOI’s will stay the same throughout the life of a document