Prompt Potential: A Pilot Assessment of Using Generative Artificial Intelligence (ChatGPT-4) as a Tutor for Engineering and Maths

Nikolic, S; Heath, A; Vu, BA; Daniel, S; Alimardani, A; Sandison, C; Lu, X; Stappenbelt, B; Hastie, D

Prompt Potential: A Pilot Assessment of Using Generative Artificial Intelligence (ChatGPT-4) as a Tutor for Engineering and Maths

Nikolic, S Heath, A Vu, BA Daniel, S

Alimardani, A Sandison, C Lu, X Stappenbelt, B Hastie, D

Permalink

Publication Type:: Conference Proceeding
Citation:: SEFI 2024 - 52nd Annual Conference of the European Society for Engineering, Proceedings: Educating Responsible Engineers, 2024, pp. 769-781
Issue Date:: 2024-01-01

Open Access

Copyright Clearance Process

Recently Added
In Progress
Open Access

This item is open access.

Adobe PDF

Download Submitted versionAdobe PDF (464.36 kB)

View on publisher's site

View statistics

Full metadata record

Field	Value	Language
dc.contributor.author	Nikolic, S
dc.contributor.author	Heath, A
dc.contributor.author	Vu, BA
dc.contributor.author	Daniel, S https://orcid.org/0000-0002-7528-9713
dc.contributor.author	Alimardani, A
dc.contributor.author	Sandison, C
dc.contributor.author	Lu, X
dc.contributor.author	Stappenbelt, B
dc.contributor.author	Hastie, D
dc.date.accessioned	2025-03-11T00:58:32Z
dc.date.available	2025-03-11T00:58:32Z
dc.date.issued	2024-01-01
dc.identifier.citation	SEFI 2024 - 52nd Annual Conference of the European Society for Engineering, Proceedings: Educating Responsible Engineers, 2024, pp. 769-781
dc.identifier.uri	http://hdl.handle.net/10453/185648
dc.description.abstract	The meteoric rise of GenAI has caused many educators to be consternated by its potential to undermine assessment. However, there is a more optimistic view to instead focus on the pedagogical affordances that GenAI can bring, for example, in tailoring personalised learning experiences for students. In this pilot study, we investigate ChatGPT-4's potential to act as a one-on-one tutor for engineering and mathematical concepts. We use three research-informed prompt strategies and simulate interactions with high-, mid-, and low-performing students. We find that the learning experience is best tailored to high-performing students. However, to gain comfort in using it, the experience must be error-free. We discovered performance varied by topic, but there indeed are topics that ChatGPT-4 can engage with error-free or with a slight chance of errors.
dc.language	en
dc.relation.ispartof	SEFI 2024 - 52nd Annual Conference of the European Society for Engineering, Proceedings: Educating Responsible Engineers
dc.relation.isbasedon	10.5281/zenodo.14254786
dc.rights	info:eu-repo/semantics/openAccess
dc.title	Prompt Potential: A Pilot Assessment of Using Generative Artificial Intelligence (ChatGPT-4) as a Tutor for Engineering and Maths
dc.type	Conference Proceeding
pubs.organisational-group	University of Technology Sydney
pubs.organisational-group	University of Technology Sydney/Faculty of Engineering and Information Technology
pubs.organisational-group	University of Technology Sydney/Faculty of Engineering and Information Technology/School of Professional Practice and Leadership
utslib.copyright.status	open_access	*
dc.rights.license	This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License (CC BY-NC 4.0). To view a copy of this license, visit https://creativecommons.org/licenses/by-nc/4.0/
dc.date.updated	2025-03-11T00:58:31Z
pubs.publication-status	Published

Abstract:

The meteoric rise of GenAI has caused many educators to be consternated by its potential to undermine assessment. However, there is a more optimistic view to instead focus on the pedagogical affordances that GenAI can bring, for example, in tailoring personalised learning experiences for students. In this pilot study, we investigate ChatGPT-4's potential to act as a one-on-one tutor for engineering and mathematical concepts. We use three research-informed prompt strategies and simulate interactions with high-, mid-, and low-performing students. We find that the learning experience is best tailored to high-performing students. However, to gain comfort in using it, the experience must be error-free. We discovered performance varied by topic, but there indeed are topics that ChatGPT-4 can engage with error-free or with a slight chance of errors.

Please use this identifier to cite or link to this item:

http://hdl.handle.net/10453/185648