Challenges in Assessing Learning Outcomes of Undergraduates in Web Development Due to AI Assisted Solutions

Abstract

Increasing fascination with artificial intelligence (AI), Large Language Models (LLMs) are a powerful type of Artificial Intelligence (AI) that simulates how humans organize language and are able to interpret, predict, and generate text. Most of the students have been adopted and practicing for taking the quick responses generated by the AI content generators. It has become challenging to determine whether the students have achieved the expected level of learning outcomes and then evaluate the success of education through the evaluation of student’s assessments. The study analyzed the knowledge, cognitive abilities, and creative skills of 450 first-year undergraduates in a web development module at private university computing faculty. Students were divided into two categories ass experiment group and control group. A limited time period was given to complete creating a webpage as a in class assessment. The primary languages taught in the module were HTML, CSS, JavaScript, and PHP for web development. The undergraduate’s knowledge, cognitive abilities and creative skills were assessed based on the score gained for ten selected attributes code quality and readability, problem solving skills, logical thinking and debugging skills time management, understanding of programming concepts, complexity management, novel solutions, user friendly interfaces, completeness well designed and functional and use of development tool. Automation testing and manual testing were employed. Code quality and the readability tested by using SonarQube integrated Jenkin. The outcome of the research identified that experiment group, which utilized ChatGPT scores higher compared to control group students. The analysis of the experimental group's scores revealed insights into meeting learning outcomes and student proficiency levels. The study found it challenging to accurately assess full achievement of learning outcomes when assessments include AI-generated content.