Supporting Self-Regulated Learning with Student-Facing Learning Analytics: User-centric Design Guidelines
Author(s)
Primary Supervisor
Wang, Kewen
Other Supervisors
Torrisi, Rosaria G
Binnewies, Sebastian
Year published
2021-01-07
Metadata
Show full item recordAbstract
Universities are investing heavily in online learning in a bid to remain competitive in a globalized world and in harsh economic times. The need to enhance and strengthen online learning is even greater given the current Covid-19 pandemic that makes it difficult to conduct face-to-face learning sessions due to the need for social distancing. Normally, retention and success rates in online courses are much lower as compared to traditional face-to-face courses – a major concern for universities. This issue is attributed to the lack of adequate self-regulated learning (SRL) skills; a situation, which is particularly problematic ...
View more >Universities are investing heavily in online learning in a bid to remain competitive in a globalized world and in harsh economic times. The need to enhance and strengthen online learning is even greater given the current Covid-19 pandemic that makes it difficult to conduct face-to-face learning sessions due to the need for social distancing. Normally, retention and success rates in online courses are much lower as compared to traditional face-to-face courses – a major concern for universities. This issue is attributed to the lack of adequate self-regulated learning (SRL) skills; a situation, which is particularly problematic in online learning where students have greater levels of autonomy and flexibility. SRL skills enable students to actively and independently control their own learning processes and contribute to academic success. The proliferation of online learning in education institutions brings to greater focus on the importance of supporting students’ SRL skills. It is known that SRL skills can be fostered in students and one possible way to achieve this is to embed tools that support the development of SRL in day-to-day online learning tools. Student-facing learning analytics (SFLA) are one possible avenue for supporting SRL in online learning environments. This is attributed to the fact that they present new opportunities for collecting and analysing students’ learning data and reporting it back directly to students. They make use of visual tools such as charts, graphs, and network diagrams to present feedback to students. This feedback can enable learners to gain insight into their learning process and reflect on their learning thereby supporting students’ SRL activities. However, the potential of SFLA to support students’ SRL skills is failing to be realized. This is largely attributed to the current design methods that are flawed and techno-centric, focusing on availability of data with little attention to learning science theory and student needs as confirmed by the exploratory study. As interest in SFLA to foster SRL grows and higher education institutions continue to implement SFLA on a widespread scale, there is an urgent need for design guidelines that are studentcentred and learning science theory-driven. For an emerging field, the need for developing a body of knowledge to address the design, development, and implementation issues in LA systems cannot be underestimated. The work presented in this thesis is a response to this need. Therefore, the central research question addressed in this study is: How can student-facing learning analytics be designed to best support SRL skills among students? This question was broken down into the following specific questions: i. What are the students’ self-regulated learning support needs based on the self-regulated learning theory? ii. What are the students’ perceptions of student-facing learning analytics? iii. What student-facing learning analytics features are most appropriate to support students’ self-regulated learning? To answer these research questions, Zimmerman’s cyclic model of selfregulation was adopted as the theoretical basis and a user-centred design approach was taken. A mixed-methods approach was used to investigate how the design of SFLA for supporting SRL may be improved. The study focused on understanding student's SRL support needs and how they should be addressed while being grounded in learning science theory; establishing students SFLA preferences and concerns; generating both general and specific design guidelines; and proposing an overall framework will support the design of SFLA for supporting SRL that will enhance learning experiences, learning practices and improve the learning process. To achieve the study aim, an exploratory study was first conducted with learning analytics experts to ascertain the relevancy and urgency of the research problem. From the insights gained, a conceptual framework for the optimal design of SFLA for supporting SRL was proposed. The three research questions stated above were formulated based on this conceptual framework. The study was conducted in three phases with each phase addressing one research question as follows: In the first phase, RQN 1 was answered to establish student's SRL support needs. This involved conducting a survey with online students undertaking business courses at an Australian public university to examine student's SRL differences and SRL support needs. Cluster analysis using K-means revealed four SRL profiles (nonself- regulators, basic self-regulators, proficient self-regulators, and expert selfregulators) based on Zimmerman’s SRL framework. Each profile exhibited different characteristics hence differing SRL support needs. The results confirmed that students have low SRL skills as the non-self-regulators constituted the largest profile with 121 students (40%) while the expert self-regulators were the smallest with 20 students (7%) of the study respondents. In the second phase, RQN 2 was answered by examining students’ perceptions of SFLA using a survey with undergraduate university students. The results revealed SFLA features and data considered most important from the student perspective. Notably, students considered data related to their emotional aspects as extremely important, even though current LA applications have given less attention to the emotional aspects of the learning process. Student concerns towards SFLA were also established and these included the loss of autonomy, privacy and security, teacher role, accuracy, and timing of the feedback, and depression, anxiety, and stress. Hence, learning analytics designers, researchers, and educators should address these concerns during the design and implementation process of SFLA for supporting SRL. In phase three, RQN 3 was addressed through an experiment that was conducted with undergraduate students to determine the most appropriate SFLA features to support SRL for students in each of the identified SRL profiles. The findings revealed both positive and negative relationships between students SFLA preferences and SRL profiles. Some students SFLA preferences conflicted with the kind of SRL support they needed. Based on these results, profile-specific SFLA features and generic SFLA features were generated and summarised in the form of design cards. Cumulatively, the investigations yielded student-centred and theory-based guidelines to inform the design of SFLA that will likely support students’ SRL skills. Specifically, the study yielded the following contributions: A conceptual framework for the optimal design of SFLA for supporting SRL; Self-regulated learning profiles, and their SRL support needs. The kinds of user data and SFLA features that students consider important in SFLA; The specific design guidelines with design cards for each of the identified SRL profile; the overall research-based framework for designing SFLA for supporting SRL.
View less >
View more >Universities are investing heavily in online learning in a bid to remain competitive in a globalized world and in harsh economic times. The need to enhance and strengthen online learning is even greater given the current Covid-19 pandemic that makes it difficult to conduct face-to-face learning sessions due to the need for social distancing. Normally, retention and success rates in online courses are much lower as compared to traditional face-to-face courses – a major concern for universities. This issue is attributed to the lack of adequate self-regulated learning (SRL) skills; a situation, which is particularly problematic in online learning where students have greater levels of autonomy and flexibility. SRL skills enable students to actively and independently control their own learning processes and contribute to academic success. The proliferation of online learning in education institutions brings to greater focus on the importance of supporting students’ SRL skills. It is known that SRL skills can be fostered in students and one possible way to achieve this is to embed tools that support the development of SRL in day-to-day online learning tools. Student-facing learning analytics (SFLA) are one possible avenue for supporting SRL in online learning environments. This is attributed to the fact that they present new opportunities for collecting and analysing students’ learning data and reporting it back directly to students. They make use of visual tools such as charts, graphs, and network diagrams to present feedback to students. This feedback can enable learners to gain insight into their learning process and reflect on their learning thereby supporting students’ SRL activities. However, the potential of SFLA to support students’ SRL skills is failing to be realized. This is largely attributed to the current design methods that are flawed and techno-centric, focusing on availability of data with little attention to learning science theory and student needs as confirmed by the exploratory study. As interest in SFLA to foster SRL grows and higher education institutions continue to implement SFLA on a widespread scale, there is an urgent need for design guidelines that are studentcentred and learning science theory-driven. For an emerging field, the need for developing a body of knowledge to address the design, development, and implementation issues in LA systems cannot be underestimated. The work presented in this thesis is a response to this need. Therefore, the central research question addressed in this study is: How can student-facing learning analytics be designed to best support SRL skills among students? This question was broken down into the following specific questions: i. What are the students’ self-regulated learning support needs based on the self-regulated learning theory? ii. What are the students’ perceptions of student-facing learning analytics? iii. What student-facing learning analytics features are most appropriate to support students’ self-regulated learning? To answer these research questions, Zimmerman’s cyclic model of selfregulation was adopted as the theoretical basis and a user-centred design approach was taken. A mixed-methods approach was used to investigate how the design of SFLA for supporting SRL may be improved. The study focused on understanding student's SRL support needs and how they should be addressed while being grounded in learning science theory; establishing students SFLA preferences and concerns; generating both general and specific design guidelines; and proposing an overall framework will support the design of SFLA for supporting SRL that will enhance learning experiences, learning practices and improve the learning process. To achieve the study aim, an exploratory study was first conducted with learning analytics experts to ascertain the relevancy and urgency of the research problem. From the insights gained, a conceptual framework for the optimal design of SFLA for supporting SRL was proposed. The three research questions stated above were formulated based on this conceptual framework. The study was conducted in three phases with each phase addressing one research question as follows: In the first phase, RQN 1 was answered to establish student's SRL support needs. This involved conducting a survey with online students undertaking business courses at an Australian public university to examine student's SRL differences and SRL support needs. Cluster analysis using K-means revealed four SRL profiles (nonself- regulators, basic self-regulators, proficient self-regulators, and expert selfregulators) based on Zimmerman’s SRL framework. Each profile exhibited different characteristics hence differing SRL support needs. The results confirmed that students have low SRL skills as the non-self-regulators constituted the largest profile with 121 students (40%) while the expert self-regulators were the smallest with 20 students (7%) of the study respondents. In the second phase, RQN 2 was answered by examining students’ perceptions of SFLA using a survey with undergraduate university students. The results revealed SFLA features and data considered most important from the student perspective. Notably, students considered data related to their emotional aspects as extremely important, even though current LA applications have given less attention to the emotional aspects of the learning process. Student concerns towards SFLA were also established and these included the loss of autonomy, privacy and security, teacher role, accuracy, and timing of the feedback, and depression, anxiety, and stress. Hence, learning analytics designers, researchers, and educators should address these concerns during the design and implementation process of SFLA for supporting SRL. In phase three, RQN 3 was addressed through an experiment that was conducted with undergraduate students to determine the most appropriate SFLA features to support SRL for students in each of the identified SRL profiles. The findings revealed both positive and negative relationships between students SFLA preferences and SRL profiles. Some students SFLA preferences conflicted with the kind of SRL support they needed. Based on these results, profile-specific SFLA features and generic SFLA features were generated and summarised in the form of design cards. Cumulatively, the investigations yielded student-centred and theory-based guidelines to inform the design of SFLA that will likely support students’ SRL skills. Specifically, the study yielded the following contributions: A conceptual framework for the optimal design of SFLA for supporting SRL; Self-regulated learning profiles, and their SRL support needs. The kinds of user data and SFLA features that students consider important in SFLA; The specific design guidelines with design cards for each of the identified SRL profile; the overall research-based framework for designing SFLA for supporting SRL.
View less >
Thesis Type
Thesis (PhD Doctorate)
Degree Program
Doctor of Philosophy (PhD)
School
School of Info & Comm Tech
Copyright Statement
The author owns the copyright in this thesis, unless stated otherwise.
Subject
Learning analytics
student-facing learning analytics
self-regulated learning
learning science theory