Inter-rater reliability of a reflective rubric to assess pharmacy students’ reflective thinking

Lucas, C; Bosnic-Anticevich, S; Schneider, CR; Bartimote-Aufflick, K; McEntee, M; Smith, L

Inter-rater reliability of a reflective rubric to assess pharmacy students’ reflective thinking

Lucas, C

Bosnic-Anticevich, S Schneider, CR Bartimote-Aufflick, K McEntee, M Smith, L

Permalink

Publication Type:: Conference Proceeding
Citation:: Currents in Pharmacy Teaching and Learning, 2017, 9 (6), pp. 989 - 995
Issue Date:: 2017-11-01

Open Access

Copyright Clearance Process

Recently Added
In Progress
Open Access

This item is open access.

Adobe PDF

Download Accepted Manuscript versionAdobe PDF (160.82 kB)

View on publisher's site

View statistics

Full metadata record

Field	Value	Language
dc.contributor.author	Lucas, C https://orcid.org/0000-0002-7857-2338	en_US
dc.contributor.author	Bosnic-Anticevich, S	en_US
dc.contributor.author	Schneider, CR	en_US
dc.contributor.author	Bartimote-Aufflick, K	en_US
dc.contributor.author	McEntee, M	en_US
dc.contributor.author	Smith, L	en_US
dc.date.available	2017-07-28	en_US
dc.date.issued	2017-11-01	en_US
dc.identifier.citation	Currents in Pharmacy Teaching and Learning, 2017, 9 (6), pp. 989 - 995	en_US
dc.identifier.issn	1877-1297	en_US
dc.identifier.uri	http://hdl.handle.net/10453/49756
dc.description.abstract	© 2017 Elsevier Inc. Introduction Many health education programs aspire to foster reflective practices to enhance the reflective thinking of students during their study years and throughout their professional careers. Given the increasing attention paid to incorporating reflective practice activities into health education and evidence attributed to rater variability, it is important for educators to carefully consider the assessment criteria and reliability of assessment tools. A reflective rubric was developed to assess pharmacy students’ levels of reflection from a reflective writing task. This study investigates the inter-rater reliability (IRR) of a rubric as a tool to assess the levels of reflective thinking apparent in students’ written statements. Methods The research involved four raters from different disciplines utilizing a rubric to assess a random sample of the same forty-three reflective statements from a cohort of two hundred and sixty-four students. The IRR was measured using the intra-class correlation coefficient (ICC), using a two-way random effects model (ANOVA) with absolute agreement, to determine reliability of the assessment tool among the raters. Results Results showed measures between the raters for (i) overall reflective statement scores, and (ii) average scores for stages of reflection with an “almost perfect” agreement, ICC = 0.81 (95% CI 0.61–0.90), (F(42, 126) = 7.83, p < 0.01); and ICC = 0.89 (95% CI 0.83–0.93), (F(42, 840) = 12.49, p < 0.01) respectively. Discussion and conclusions The proposed rubric utilized by four raters showed high agreement with each other's scores, and is a tool for academic assessment of pharmacy students’ reflective thinking processes.	en_US
dc.relation.ispartof	Currents in Pharmacy Teaching and Learning	en_US
dc.relation.isbasedon	10.1016/j.cptl.2017.07.025	en_US
dc.subject.mesh	Humans	en_US
dc.subject.mesh	Observer Variation	en_US
dc.subject.mesh	Analysis of Variance	en_US
dc.subject.mesh	Thinking	en_US
dc.subject.mesh	Education, Pharmacy	en_US
dc.subject.mesh	Educational Measurement	en_US
dc.subject.mesh	Students, Pharmacy	en_US
dc.title	Inter-rater reliability of a reflective rubric to assess pharmacy students’ reflective thinking	en_US
dc.type	Conference Proceeding
utslib.citation.volume	6	en_US
utslib.citation.volume	9	en_US
utslib.for	1115 Pharmacology and Pharmaceutical Sciences	en_US
utslib.for	1302 Curriculum and Pedagogy	en_US
pubs.embargo.period	Not known	en_US
pubs.organisational-group	/University of Technology Sydney
pubs.organisational-group	/University of Technology Sydney/Graduate School of Health
utslib.copyright.status	open_access
pubs.issue	6	en_US
pubs.publication-status	Published	en_US
pubs.volume	9	en_US

Abstract:

© 2017 Elsevier Inc. Introduction Many health education programs aspire to foster reflective practices to enhance the reflective thinking of students during their study years and throughout their professional careers. Given the increasing attention paid to incorporating reflective practice activities into health education and evidence attributed to rater variability, it is important for educators to carefully consider the assessment criteria and reliability of assessment tools. A reflective rubric was developed to assess pharmacy students’ levels of reflection from a reflective writing task. This study investigates the inter-rater reliability (IRR) of a rubric as a tool to assess the levels of reflective thinking apparent in students’ written statements. Methods The research involved four raters from different disciplines utilizing a rubric to assess a random sample of the same forty-three reflective statements from a cohort of two hundred and sixty-four students. The IRR was measured using the intra-class correlation coefficient (ICC), using a two-way random effects model (ANOVA) with absolute agreement, to determine reliability of the assessment tool among the raters. Results Results showed measures between the raters for (i) overall reflective statement scores, and (ii) average scores for stages of reflection with an “almost perfect” agreement, ICC = 0.81 (95% CI 0.61–0.90), (F(42, 126) = 7.83, p < 0.01); and ICC = 0.89 (95% CI 0.83–0.93), (F(42, 840) = 12.49, p < 0.01) respectively. Discussion and conclusions The proposed rubric utilized by four raters showed high agreement with each other's scores, and is a tool for academic assessment of pharmacy students’ reflective thinking processes.

Please use this identifier to cite or link to this item:

http://hdl.handle.net/10453/49756