Wenonah Campbell, Justine Hamilton, Michelle Phoenix, Ashwini Namasivayam- MacDonald, and Lyn Turkstra Speech-Language Pathology Program, School of Rehabilitation Science McMaster University
The position statement by the American Speech-Language-Hearing Association (ASHA) Joint Coordinating Committee on Evidence-Based Practice (EBP) indicates that speechlanguage pathologists are expected to “incorporate the principles of evidence-based practice in clinical decision making to provide high quality clinical care” where “the term evidence-based practice refers to an approach in which current, high-quality research evidence is integrated with practitioner expertise and client preferences and values into the process of making clinical decisions” (ASHA, 2005). Among the skills required to engage in EBP, clinicians must be able to evaluate the quality of all sources of evidence informing clinical care, critically appraise sources of information related to the use of assessments and the provision of interventions, and continually identify and appraise new research evidence that may impact clinical services and client care (ASHA, 2005).
In recognition of the vital role that professional graduate programs play in preparing future clinicians with a strong foundation in the skills required to engage in EBP in clinical practice, the inaugural ASHA Faculty Development Institute explored how faculty could adjust their curricular content and pedagogy to better support students—so that those students can acquire the knowledge and skills that they will need to truly embed EBP into their routine practice after graduation. In the narrative that follows, we share how our problem-based professional master’s degree program in speech-language pathology uniquely infuses the teaching and learning of EBP into our curriculum within and across courses as well as units of study.
What Does McMaster University’s Problem-Based Program in Speech-Language Pathology Look Like?
Our graduate program is offered full time over 23 months and provides students with the knowledge, skills, and professional behaviors for practice as entry-level speechlanguage pathologists. Like all programs, the students complete coursework and clinical practica in a variety of settings during their 2 study years. The program is organized into five thematic units (shown in Figure 1), with clinical placements following each academic unit. Placements vary in length from 2 weeks following the first unit of study to 8–9 weeks following the final unit.
Unique to McMaster University, our program uses a problem-based, self-directed learning philosophy (Hamilton et al., 2019). In problem-based learning, students focus on a health care problem in a small group that is guided by a instructor, who facilitates discussion and supports students to think rationally and critically. Health care problems are scenarios drawn from real-world clinical contexts that are carefully designed and written to stimulate student learning. Instructors are community-based speech-language pathologists who receive training in problem-based learning and ongoing mentoring from faculty to support their group’s exploration of the problem. Students bring their previous knowledge and experience as well as their current course content to the examination of the problem. As reported by Hamilton et al. (2019), the problem-based tutorial (PBT) process unfolds in four steps: (1) brainstorming, (2) identification of learning objectives, (3) discussion, and (4) feedback. When students first receive the health care problem, they brainstorm to identify key concepts, relevant knowledge or experience that they already have, and gaps in their knowledge. With their instructor’s guidance, students select specific ideas to explore from their brainstorming results. They reformulate these ideas into specific learner outcomes that guide the independent research that students will complete prior to their next tutorial. At the subsequent tutorial meeting, students share their research, evaluate what they have learned individually and from each other, and synthesize and apply their collective knowledge to a comprehensive and critical analysis of the health care problem. Each tutorial closes with group members and the instructor providing positive and constructive feedback on group process, professional behavior, EBP, and clinical reasoning.
The PBT course is the cornerstone of the program’s curriculum, with students spending 5 hours each week in their small groups in addition to approximately 10 hours of independent work to prepare for those sessions. The remaining required courses in McMaster University’s speech-language pathology program occur in every unit of study and include a clinical skills course (6 hours per week), a foundational knowledge course (3 hours per week), and a seminar or EBP course (3 hours per week). The content in each course is organized by overall thematic unit and then is delivered in coordination with each health care problem. Thus, when students begin a new health care problem in their PBT, the content in their other courses for that week is related to the concepts and issues being explored in that problem. In this way, students are supported to integrate and apply their learning in ways that are meaningful to clinical practice. In addition to the coordination of content across courses, also unique to a problem-based learning approach is the idea that students are introduced to concepts and content iteratively, with each exposure building on previously learned content. Thus, for example, students do not complete courses focused on anatomy and physiology or specific types of communication and swallowing disorders; instead, they are exposed to core content multiple times over their 2 years of study. For a description of how content in specific practice areas (e.g., voice, fluency, articulation, acquired language disorders) is addressed across the five units of study, interested readers may consult the handout available on our program website.
How Is Evidence-Based Practice Integrated Within Our Speech-Language Pathology Program?
Within our curriculum, students receive a solid foundation in knowledge and skills essential to EBP as part of their seminar course in the first unit of study. Advanced knowledge and skills related to co-producing clinical practice research are developed in a full-year EBP course that occurs across the fourth and fifth units of study. Throughout the program, EBP concepts are reinforced in other courses as well as on clinical placements. Finally, EBP is integral to the self-directed study in all PBTs across the five units of study. For example, the Appendix at the end of this document contains an excerpt from the evaluation form that instructors complete for each PBT across the five units of study. The excerpt shows the array of EBP skills that each tutor must evaluate for each student, with skills color-coded to indicate the corresponding unit of study by which mastery is expected. To ensure that tutors are prepared to evaluate students’ EBP skills, we provide information sessions for the tutors in each unit. These information sessions describe what EBP skills students are developing in that unit as well as what they will have carried forward from prior units. Our health science librarians also provide a session for tutors in each unit—with the goal of advising them about how to use the resources that students will be accessing during their independent research. Finally, we provide questions and prompts that instructors can use to support students in applying their EBP skills in tutorial sessions.
Establishing Foundational Knowledge and Skills in EBP
In the Unit 1 seminar course, students acquire and apply foundational skills in EBP— including asking focused clinical research questions about assessment and intervention, searching for and locating relevant literature, and appraising individual studies related to clinical practice. The course itself is scheduled for 1.5 hours twice each week and includes a mixture of lecture, small-group discussion and analysis, and student-led presentations. Our program has adopted the definition of EBP put forth by Dollaghan (2007), which states that EBP is “the conscientious, explicit, and judicious integration of (1) best available external evidence from systematic research, (2) best available evidence internal to clinical practice, and (3) best available evidence concerning the preferences of a fully informed patient” (p. 2). Although Dollaghan’s article is more than 10 years old now, her definition of EBP—and the guidance provided to students on how to conduct critical appraisal—is just as relevant and applicable to contemporary practice as when it was first published. Moreover, the use of examples specific to speechlanguage pathology practice and the provision of critical appraisal forms with clear guidance for implementation make this an ideal text for those new to the critical appraisal process.
With respect to the seminar course curriculum, seven classes are delivered across the first 5 weeks of study to establish the basic knowledge that students need to search for, locate, read, and critically appraise research evidence. Table 1 describes the topics, learning objectives, and teaching resources for these seven lectures. One class is delivered by a health sciences librarian, who provides an overview of how to develop and execute a search strategy in an electronic bibliographic database for a focused clinical research question related to speech-language pathology. That overview is accompanied by a real-time demonstration.
In the first week of the program, following the session with the librarian, students also receive their first assignment, which is to locate literature relevant to a clinical research question by conducting an iterative search for evidence in a bibliographic database. Specifically, students are provided a choice of clinical research questions written using the Population-Intervention-Comparison- Outcome (PICO) format (Dollaghan, 2007). They are asked to choose one question, separate it into its PICO components, generate a list of potential search terms for each component, and then iteratively develop a search strategy for their question in one electronic database. As part of the assignment submission, students
• complete an instructor-supplied template that documents their original brainstorm for search terms;
• make their final selection of terms to enter into their chosen database;
• provide instructors with a copy of their search strategy;
• produce citations for five articles they located that are relevant to their PICO question; and
• write a 500-word reflection on the process that they followed and the resources that they accessed while completing the assignment—as well as what they learned from this experience.
Because students are required to conduct independent research for their PBTs each week, this assignment is strategically placed very early in the program so that students are exposed to library resources and learn how to use them from the outset. Student feedback collected informally and on course evaluations suggests that students appreciate and value learning how to locate research evidence efficiently and effectively during the initial phases of the program.
Following the completion of the first assignment and the provision of the 5 weeks of foundational content, each student is required to give five student-led group presentations on appraisal skills required for five types of empirical research evidence (i.e., intervention studies using group comparison designs, intervention studies using single-subject experimental designs, diagnostic accuracy studies, systematic reviews and meta-analyses, and qualitative studies). Each presentation is scheduled for one class session and includes the following three components: (1) a presentation that addresses key concepts and information about the research design, (2) an interactive activity to give peers an opportunity to apply their knowledge, and (3) an opportunity for peers to ask questions. Each group is expected to teach fundamental material about how to critically appraise their respective research design by drawing from the instructor-assigned reading and critical appraisal form (e.g., the accompanying chapter and critical appraisal form in the Dollaghan [2007] textbook). In consultation with the instructor, groups also select an exemplar article that uses their selected research design, which they critically appraise in advance and use as part of their interactive activity to teach their peers about the critical appraisal process. Consultation with the instructor is also available to all groups to assist them with clarifying key concepts and reviewing their plan for their presentation. This ensures that the content of group presentations is accurately conveyed and that the learning experience is optimized for all. All students in the class are expected to have read the assigned reading and exemplar article in advance of each group presentation. To facilitate their peers’ learning, groups are expected to post their presentation materials prior to class as well as share their completed critical appraisal forms following the class. Student feedback on course evaluations suggests that although students report that the presentations require considerable work and effort, they appreciate the deeper learning that occurs because of their involvement in teaching this course content. Student feedback also suggests that they appreciate learning from and with their peers.
For the final EBP-related assignment in this course, students are provided with a set of articles selected by the instructor; these articles showcase the different research designs addressed via the group presentations. Each student is required to select one article, which must use a research design that is different than the one they reviewed for their group presentation, and prepare a written paper that (1) summarizes the article in their own words; (2) includes an appendix with the corresponding critical appraisal form completed by the student; (3) synthesizes the main strengths and limitations of the article in the body of their paper; (4) comments on the validity and clinical importance of the findings; and (5) makes a recommendation as to whether the article should be used to inform clinical practice. Completed independently, this assignment supports students
in building their knowledge and skills in critical appraisal of a second type of research design. Furthermore, students are required to demonstrate an ability to paraphrase a piece of research in writing, examine strengths and limitations within a research study in relation to one another to arrive at a conclusion about the overall strength of the evidence, and synthesize their critical analysis through scholarly writing. The main way in which the task is constrained is through the careful selection of the exemplar articles that students can choose to critique. Exemplar articles are selected to show a variation in quality and a range of strengths and limitations. This ensures that students have sufficient material to critique. More important, though, it is important to choose articles that are not too long (e.g., 5–10 pages), have a single focused research question, and have a clear research design. Without these attributes, it can be overwhelming for students who often are undertaking independent critical appraisal for the first time. Constraining the article choices is also helpful to the instructor because they also must read and critique each article to evaluate students’ work. To discourage plagiarism, once a set of articles is utilized for this assignment, they are not offered again until the corresponding student cohort has matriculated from the program.
Developing Advanced Knowledge and Skills in EBP
In Units 4 and 5, students complete a full-year course to enable them to critically analyze the literature and collaborate on a clinical research project relevant to speechlanguage pathology. The research projects that students complete are invited from faculty and the local clinical community. Interested supervisors complete an application form and a checklist to consider their readiness to supervise a project (e.g., expected Integrating Evidence-Based Practice 11 time commitment and availability of resources to support their project, if applicable). Faculty members review the applications to determine fit with the course and with the speech-language pathology program. They also meet with potential supervisors to discuss and refine proposed projects. Once projects are selected, faculty match students to projects based on several factors (e.g., number of students required to complete the work, students’ interests and skill sets, availability to travel if required by the project). Student teams and supervisors meet throughout Unit 4 to develop the project proposal and again in Unit 5 to complete the research project. Students have specific class time set aside in their course schedule to ensure that they have time to meet and conduct the work involved.
Across the two units of study, students achieve the following learning outcomes:
• Construct relevant, appropriate, and clearly defined clinical questions.
• Know where to search for the most appropriate evidence.
• Appraise and synthesize the evidence and provide a clear rationale for application.
• Communicate the process and results of these efforts orally and in writing to a variety of audiences.
• Consider theoretical, evidence-based, and practical application issues.
• Lead a discussion among peers related to EBP process and content.
• Present information to peers in a manner that demonstrates understanding of the topic.
In Unit 4, students meet with their team members, project supervisor, and faculty mentor (as needed) to define their project scope, identify the research question, conduct a literature review, and prepare their research proposal. At their first meeting, students complete learning contracts and project agreements with their supervisors to formalize their agreed-upon project and to set a schedule of work that they will need to complete. To prepare students for undertaking project activities, class instruction and experiential learning is provided about research design, data acquisition and analysis, and translation of knowledge into practice. Students also build on their knowledge and skills from previous units in searching and synthesizing the literature. Table 2 describes session topics, learning objectives, and teaching resources for Unit 4 classes. During Unit 5, students use their course time to conduct their proposed research, prepare and analyze data, and write a final project report. Project supervisors and teams can access their faculty mentor during this time whenever specific guidance or resources are needed. The course culminates with an EBP Symposium—hosted by the program and course instructors for the students, project supervisors, and the clinical community—at which each student team presents a poster or presentation describing their work.
With respect to evaluation, students complete a series of sequenced assignments that structure the process—from reviewing the literature to producing a final project report and an oral or poster presentation. The sequence includes a schedule for creating outlines and drafts of each proposal component (i.e., introduction and methods) as well as the final report (i.e., results and discussion). Students receive feedback on their outlines and drafts and can iteratively incorporate that feedback into their final report and presentation. Project supervisors review the students’ products at each step and provide their feedback for the students to incorporate prior to the course submission date. Formal evaluation is completed by the course instructors, who also provide their feedback to the student teams. Assigning responsibility for grading to the course instructors has many benefits: It reduces the workload of the community clinicians who have volunteered their time; it ensures that students receive opportunities for feedback from multiple sources, including faculty with research training; and it provides an equitable structure for evaluating students’ work that is consistent across projects and teams.
One unique assignment that we use in this course is designed to scaffold students’ review and synthesis of the research literature. Specifically, we adapted a set of literature review charts organized in an Excel file from the Johns Hopkins University Libraries (see https://guides.library.jhu.edu/lit-review/synthesize). The charts have three separate “tabs” that include columns with prompts to guide students in (a) succinctly summarizing information from individual articles, (b) consistently evaluating the credibility and validity of each article’s content, and (c) identifying similarities and “themes” across individual pieces of research so that it is easier to see how they relate to one another. Although students have told us that these charts are time-consuming to complete, they are effective in helping students elucidate patterns in the literature, which is an essential prerequisite to writing a synthesized literature review.
Conclusions and Lessons Learned
Over the course of developing and refining EBP content within our curriculum, one lesson that has been reinforced continuously is the importance of building a strong relationship with our university librarians. They have been an invaluable resource in helping teach our students the skills necessary to search the literature effectively and efficiently. We routinely invite our health science librarians to deliver sessions to our students as well as to the community clinicians who are instructors in our problembased program. Our librarians have even reviewed and provided feedback on the assignment we use in Unit 1 to teach students how to conduct an effective database search. We also have cultivated partnerships with our local clinical community by working with them to answer research questions that are meaningful to them and that arise from their own everyday practice. Through these partnerships, not only have we built capacity for EBP in our student clinicians but we also are building capacity for clinical practice research in the local clinical community over the long term. Further, we have learned the importance of integrating EBP into all courses in the program— including evaluating students on their ability to critically appraise, synthesize, and apply evidence to the discussion of health care problems in PBTs. Finally, although integrating EBP content across a full curriculum is not a simple or easy process, we have found that curriculum planning meetings prior to each unit of study help us organize and integrate content. Our faculty uses a shared document to organize our curricular units, a practice that supports shared understanding of the relation between content across our courses. Our curriculum development is ongoing, and we continually make refinements at the end of each unit based on our experiences as faculty and feedback from our students. What is reported in this article at present will likely evolve. However, in sharing our program’s current approach, we hope that we have sparked at least one new idea for your own program!
References
American Speech-Language-Hearing Association. (2005). Evidence-based practice in communication disorders [Position Statement]. www.asha.org/policy
Cypress, B. S. (2015). Qualitative research: The ‘‘what,’’ ‘‘why,’’ ‘‘who,’’ and ‘‘how’’! Dimensions of Critical Care Nursing, 34(6), 356–361. https://doi.org/10.1097/dcc.0000000000000150
Dollaghan, C. A. (2007). The handbook for evidence-based practice in communication disorders. Paul H. Brookes.
Graham, I. D., Logan, J., Harrison, M. B., Straus, S. E., Tetroe, J., Caswell, W., & Robinson, N. (2006). Lost in knowledge translation: Time for a map? Journal of Continuing Education in the Health Professions, 26(1), 13–24. https://doi.org/10.1002/chp.47
Greenhalgh, T. (2014). How to read a paper: The basics of evidence-based medicine (6th ed.). Wiley Blackwell.
Hamilton, J., & Phoenix, M. (2021). Tutorial performance evaluation form [Unpublished manuscript]. Speech-Language Program, McMaster University.
Hamilton, J., Phoenix, M., Campbell, W., & Turkstra, L. (2019). Problem-based learning in speech-language pathology graduate education. ASHA Access Academics and Research Newsletter. https://academy.pubs.asha.org/2019/06/problem-based-learning-in-speech-language-pathology-graduate-education/
Keegan, L. C. (2012, Fall). Review of research methods in communication disorders. Contemporary Issues in Communication Science and Disorders, 39, 98–104. https://doi.org/10.1044/cicsd_39_F_98
Korstjens, I., & Moser, A. (2018). Series: Practical guidance to qualitative research. Part 4: Trustworthiness and publishing. European Journal of General Practice, 24(1), 120–124. https://doi.org/10.1080/13814788.2017.1375092
Paré, G., & Kitsiou, S. (2017). Methods for literature reviews. In F. Lau & C. Kuziemsky (Eds.), Handbook of eHealth evaluation: An evidence-based approach. University of Victoria.
Rowland, D. R. (n.d.). Reviewing the literature: A short guide for research students. The Learning Hub, Student Services, The University of Queensland.
White, S., Raghavendra, P., & McAllister, S. (2017). Letting the CAT out of the bag: Contribution of critically appraised topics to evidence-based practice. Evidence- Based Communication Assessment and Intervention, 11(1–2), 27–37.
Table 1
Session Topics, Learning Objectives, and Teaching Resources for Establishing Foundational Knowledge in Evidence- Based Practice (EBP)
| Topic | Learning Objectives | Teaching Resources |
| Introduction to evidence-based practice | To define the term evidence-based practice (EBP)To describe the four components of aclinical research question | Dollaghan (2007) – Chapters 1 and 2 |
| Visit from a health science librarian | To describe research resources available through the libraryTo describe and demonstrate how to construct and execute an effective search strategy in a bibliographicdatabase | Library resources for the Speech Language Pathology Program at McMaster |
| How to read a research article | To apply strategies for critically readingresearch | How to Read and Comprehend ScientificResearch Articles |
| To share lessons learned about being“critical consumers” of research | Critically Reading Journal Articles | |
| Building quantitative research knowledge: Reliability andvalidity | To define the terms internal validity andexternal validity To identify factors that impact the extent to which evidence is considered internally and/or externally valid | Dollaghan (2007) – Chapter 4Reliability vs. Validity Internal Validity |
| Building quantitative research knowledge: Statistics | To describe and explain foundational statistical concepts (e.g., levels of measurement, types of statistical tests, p values, statistical error, confidence intervals) | Greenhalgh (2019) – Chapter 5Levels of Measurement Statistical Significance, the Null Hypothesis, and p Values Understanding a Confidence Interval Confidence Interval Interactive Visualization |
| Building quantitative research knowledge: Clinical importance | To define the terms precision and effect sizeTo describe common measures of effect size | Dollaghan (2007) – Chapter 5Cohen’s d Interactive Visualization r2 Interactive Visualization Understanding the Odds Ratio |
| Building qualitative research knowledge | To define qualitative research and identify its underpinning assumptionsTo describe five different types of qualitative researchTo identify sources of data in qualitative studiesTo explain the concepts of rigor andtrustworthiness in qualitative research | Cypress, B. S. (2015). Korstjens, I., & Moser, A. (2018). |
Table 2
Session Topics, Learning Objectives, and Teaching Resources for Developing Advanced Knowledge in EBP
| Topic | Learning Objectives | Teaching Resources |
| Visit from a health science librarian | To describe literature review resources available through the libraryTo demonstrate how to construct and execute systematic searches acrossdatabases | Detailed tutorial on the literature search process |
| Types of literature reviews and how to synthesize literature | To identify the reasons for doing literature reviewsTo identify strategies for reviewing, organizing, synthesizing, and critically analyzing literatureTo recognize and differentiate between different types of literature reviews | Paré, G., & Kitsiou, S. (2017). Chapter 9 –Rowland, D. R. (n.d.).White, S., Raghavendra, P., & McAllister, S. (2017).Writing the General Literature Review Section of a Proposal Tips and Strategies for Doing a General Literature Review |
| To describe the types of literature reviews relevant to students’ projectsTo engage students in applying knowledge about literature reviews in abrainstorming activity | ||
| How to do quantitative and qualitative research | To apply quantitative and qualitative research methods in an experiential activityTo obtain experience analyzing and interpreting quantitative and qualitative dataTo identify how different research methods can be used to address the same research questionTo reflect on the experience and relate to students’ projects | Keegan, L. C. (2012, Fall). |
| How to prepare an impact statement | To develop and deliver a 1-minute “elevator pitch” that would compel a funding source to support students’projects | |
| Introduction to knowledge translation | To define and describe knowledge translation and how it is related to clinical practiceTo explain the Knowledge-to-Action framework for moving researchevidence into clinical practice | Graham, I. D., Logan, J., Harrison, M. B., Straus, S. E., Tetroe, J., Caswell, W., & Robinson, N. (2006). |
Figure 1
Description of Thematic Units of Study in McMaster University’s Speech-Language Pathology Program
Appendix
Evaluation Criteria Related to Evidence-Based Practice
McMaster University MSc(SLP)
TUTORIAL PERFORMANCE EVALUATION-BY TUTOR
Source: Hamilton, J., & Phoenix, M. (2021). Tutorial Performance Evaluation Form. Speech-Language Program, McMaster University. Excerpt provided with permission.
