The design and learning analytics of minimized collaborative scientific problem-solving activities considering cultural differences

Summary of work

Researchers and educators consider CPS as one of the core competencies of the 21^st century. However, students often fail to solve a problem as they do not coordinate with peers to reflect upon their CPS activities. To help teachers develop collaborative activities to foster CPS ability, my work in the host institute focused on two main parts: the development of collaborative science learning activities and the analysis of active learning practice, specifically the collaborative learning practice, in the new ALCs. Regarding the collaborative science learning activity, my work has implemented twenty CPS simulations for CPS learning activities, both in Chinese and English, and has established a collaboration network between multiple countries including the European Union, Spain, Thailand and Singapore. Regarding the analysis of active learning practices, the work in the host institution and my prior works were integrated to form a Pedagogy, Space, and Technology (PST) model of space and technology design in supporting collaborative learning on campus which was featured by the IEEE Technical Committee on Learning Technology as one of the keynote speeches of The 18th IEEE International Conference on Advanced Learning Technologies in Bombay.

Background

It has been stressed by social constructivists that in-depth learning occurs when a group of students engage in building a shared understanding of a problem and solve the problem collaboratively. Collaborative problem solving (CPS) activities involve the “process whereby two or more agents attempt to solve a problem by sharing the understanding and effort required to come to a solution and pooling their knowledge, skills and efforts to reach that solution” (OECD, 2013). Researchers and educators consider CPS as one of the core competencies of the 21^st century (Griffin, McGaw, & Care, 2012). Such emphasis on the CPS abilities is mainly rooted in social constructivism which asserts that in-depth learning occurs when students engage in building a shared understanding of a problem through social interactions (Alvarez, Salavati, Nussbaum, & Milrad, 2013; Jermann & Dillenbourg, 2008; Pear & Crone-Tood, 2002). However, CPS is a complicated collaborative learning process as it involves both social and cognitive processes, meaning that students communicate, negotiate and exchange perspectives to understand a problem and regulate actions to solve the problem (Hesse, Care, Buder, Sassenberg, & Griffin, 2015). It is thus necessary to understand how teachers may design activities and implement the activity in classrooms to assist students in developing CPS abilities.

The notion of learning analytics is receiving the attention of researchers with the aim of understanding students’ inability to learn, since the accumulation of students’ learning records are becoming significant (Powell & MacNeill, 2012). The literature of CPS has applied diverse learning analytics techniques to analyze student learning records, and has identified several critical challenges when students are engaged in CPS activities (e.g., Lin, Hou, Wu, & Chang, 2014; Tan, Caleon, Jonathan, & Koh, 2014). The body of the literature shows that in some student groups the discussion often does not lead to an effective solution to solve a problem (Lin et al., 2014). In other student groups, students often fail to solve a problem as they do not coordinate with peers to reflect upon their CPS activities (Tan et al., 2014).

To address the aforementioned issues, this project aims to design and implement learning materials to help science teachers develop students’ CPS abilities and explore how students from engage in the course. Firstly, this project will develop minimized CPS activities that could be applied by science teachers to help students learn CPS abilities in science courses. The CPS activities will be minimized so that science teachers can flexibly implement them in 20 minutes in science classes. Secondly, this study will apply learning analytics techniques to analyze student learning records so as to understand the impact of the minimized CPS activity.

Objective and work summary

The learning of the CPS involves different learning settings including both physical classrooms and the cyber learning spaces. After arriving at the host institution (Teaching and Learning Transformation Center, University of Maryland at College Park), the host professor, colleagues in the host institution, and I discussed possible collaboration directions and set up the goal of the Fulbright project based on the research interests and the constraints of the host institution. During the discussion, I understood that the university had just completed the construction of a new building that is specifically dedicated to teaching and learning. Several Active Learning Classrooms (ALCs) were set up to implement active learning practice for Science, Technology, Engineering and Mathematics (STEM) classes. The work in my Fulbright project after the discussion thus includes (1) developing learning materials for the cyber learning spaces and (2) understanding how the physical classrooms in the US support the collaborative learning teaching practice. My work in the host institutes focused on two main parts: the development of collaborative science learning activities and the analysis of active learning practice, specifically the collaborative learning practice, in the new ALCs. In this vein, the first part of my work developed collaborative learning activities supporting multiple languages that can be used by teachers from different countries including the USA and Taiwan. The second part may achieve a better understanding of the design and evaluation of the physical classroom that support the teaching of the CPS skills.            

Work report

A series of works has been completed in the two parts to achieve the goal of this project. These works are detailed below: 

Part 1: The development of the CPS activities

To facilitate the CPS process, this project has implemented twenty CPS simulations for CPS learning activities. These simulations were developed in collaboration with multiple team members in my lab in Taiwan and the student volunteers in the US. The twenty simulations were also translated from Chinese into English and were published on our website(http://cosci.tw/). Three featured simulations include:

1. Buoyancy-Submarine: This simulation allows the student to control the depth of a submarine’s submersion in water. The submarine has a mass of 30,000 grams and a volume of 52,500 cm^3 and is floating upon the surface of the ocean at the start of the simulation (density of seawater: 1.03 g/cm^3). The student can add or remove water from the submarine’s ballast tank to control the depth of the submarine’s submersion and observe changes in the buoyancy of the submarine. The student and partner are required to control the submarine to keep it balanced under the water. 

2. How Much Does The Elephant Weigh?: This simulation simulates changes to the load line of a boat when it carries different goods. The weight of the barrels is given (100kg or 50kg), but the weight of the elephant and the little boy (Cao Chong) are unknown. The student pair is required to use the barrels to measure the weight of Cao Chong and the elephant to find out their weight.

3. Get wetter if you run?: This simulation simulates two giants running in the rain at different speeds and calculates how many raindrops they will catch. In the simulation we assume the body volume of the giant as rectangular blocks. The effective volumes of both giants are the same and do not change while they are running. Raindrops fall vertically (no wind) onto the giant at a constant velocity (terminal speed). The density of the rain shower is uniform in space and time. The running speed of giant A is twice as fast as that of giant B. The students are required to explore whether the giants will get wetter if they run faster. 

The above simulations are collaborative simulations that extend the capability of individual-based simulations to allow multiple students to participate in CPS activities in a shared simulation session. Such simulations together with a text chatroom can support both social and cognitive processes, helping students to communicate and understand the meaning of problem-solving activities in a shared simulation session. 

With the aid of these simulations, students work with peers to solve an everyday-life science problem to enhance their understanding of the science concepts and to develop CPS abilities. Each CPS activity will be minimized so that science teachers can flexibly implement the activity in science classes. Students could use mobile devices to participate in the CPS activity. The use of ready-to-hand devices increases the feasibility of implementing the CPS activity in classrooms.

The works of collaborative simulations have established an international collaboration network among Spain, Singapore, Thailand, and Taiwan with the aim of developing sharable science simulations that can be used to support science learning in different countries. During the Fulbright visiting period, we were invited by the EU Next-Lab project (https://www.golabz.eu/) which endeavors to collect the existing science simulations to maximize the effect of these simulations on science learning. The invitation is to enrich the variability and richness of the EU project. To broaden the impact of the simulations we created, student volunteers in the US were invited to help with the translation of these simulations. Consequently, the results of the works can reach more students and teachers to develop the CPS skills in the science curriculum. Such a result also demonstrates the impact of the Fulbright project in creating collaboration networks in international society.

Part 2: Evaluation of the ALCs in the US supporting CPS teaching

The second work part during the Fulbright visiting period is to evaluate how new ALCs support/limit CPS teaching. University of Maryland at College Page has just completed a construction project to build the Edward St. John (ESJ) Learning & Teaching Center specifically for learning and teaching. The new building was completed in 2017 and set up new ALCs. The study to evaluate these classrooms was conducted in collaboration with the faculty and students of the Teaching and Learning Transformation Center including Dr. Benjamin Bederson (CO-PI), Dr. Alice Donlan (PI) and Virginia Byrne (Project member). The goal of this part was to explore the ways students and instructors are using the new ALCs, and their perceptions of active learning within these ALCs. Special focus was particularly on how active teaching/learning practices including CPS and interactive teaching were implemented/demonstrated during class meetings. 

Three new types of ALCs were set up in the new building: 6-round, tiered-collaborative and media share classrooms. Tiered-collaborative ALCs are designed for large class teaching, however, with specialized seating arrangements allowing students to collaborate. The tables are not of a uniform shape but are of different types that make the classroom look more creative. Two rows of tables are allocated in the same tier. Students in two consequent rows can form a group to participate in collaborative learning activities. Furthermore, many small whiteboards are provided along the walls of the classroom. Students can use these whiteboards during group discussion.

Differing from traditional classrooms, the 6-Round ALCs do not provide front-facing seats, but round tables for small groups of students to facilitate group discussions. Teachers in such ALCs are not to lecture. Instead, they need to facilitate the whole-class or group discussion. Several 6-round ALCs were set up in the ESJ as the classroom type is the main classroom in the building. 

The media share ALCs specifically support group work by setting up several work stations. Each work station is equipped with shared displays that are connected to each student’s laptop computers. Therefore, students in a group can work closely with each other in a collaborative learning activity. The classroom is also equipped with a large video display that allows teachers to display the image from each of the workstations. 

The part investigated both students’ and teachers’ perceptions and learning activities in the classrooms. My study particularly focused on how teachers’ beliefs about learning may impact their teaching practices in the classroom. For the teachers using the new ALCs, we administered pre- and post-semester surveys, and arranged classroom observations to understand teachers’ teaching activities. Instructors who taught their courses in an ALC starting in the Fall of 2017 were invited to participate in the study by an email from the Teaching and Learning Transformation Center. Participating instructors accessed the online pre- and post-test surveys via Qualtrics. The survey took approximately 15 minutes to complete.

The survey contains five components to describe teachers’ expectations/experience of using the ALCs to implement collaborative learning as detailed below:

1. Teachers’ beliefs about learning: this dimension involves the beliefs held by teachers about teaching/learning and their preferred ways of teaching and learning. It consists of two parts: Transmissive view and constructivist view. Regarding the transmissive view, teachers believe that learning is a knowledge acquisition process in which students receive the information transmitted by teachers. On the contrary, teachers who hold the constructivist view believe that knowledge is constructed through participation in knowledge construction activities such as reflective discussion, discovery, and collaboration.  
2. Teaching practice: the teaching practices that teachers think they will apply or have applied when they teach in the ALCs. Example practices include group work, extensive lecturing, and class discussions.
3. Assessment tools: the assessment tools that teachers think they will apply or have applied when they teach in the ALCs. Example tools include multiple-choice questions, essay questions, and group projects.
4. The facilities/technologies: the facilities/technologies that teachers think they will apply or have applied when they teach in the ALCs. Example facilities/technologies include multiple screens, document cameras, and whiteboards.
5. Teachers’ emotion: the emotions (enthusiasm, enjoyment, happiness, tenseness, etc.) teachers feel before or after teaching in the ALCS. 

Sixty-one teachers in the University of Maryland who taught in the new ALC responded to the survey. The analysis of teachers’ experience of using new ALCs in teaching with the collaborative approach may shed light on how to design new spaces for cultivating CPS skills. It should be noted that not all the teachers’ responses were included in the analysis because of missing data for some questions. Our analysis of the teachers’ responses found five main observations regarding the use of the new ALCs in supporting the collaborative constructivist approach to learning and teachers’ beliefs.   

#Observation 1 

Our results reflect that teachers who taught in the new ALCs showed higher levels of constructivist views on teaching and learning than they did on transmissive views. Tables 1 and 2 display the teachers’ responses to sample questions on the constructivist and transmissive views. 

#Observation 2:

The data also show that teachers’ teaching practice in the new ALCs is related to their beliefs about teaching and learning. As shown in Table 3, teachers’ constructivist views were positively correlated with their frequencies of the application of group work and writing assignment practice, but negatively correlated with the application of reading primary literature.

#Observation 3:

It was also found that teachers’ beliefs about teaching and learning are also related to their application of the facilities and technologies in the new ALCs. Teachers’ constructivist views were positively correlated with their usage of the facilities (student white boards and tables/chairs) to facilitate the discussion process and enable students to express themselves through the whiteboards, as shown in Table 4. 

#Observation 4:

We also found a close relationship between teachers’ beliefs and their assessment tools. The analysis (Table 5) indicated that teachers’ constructivist views were positively correlated with the application of alternative assessment tools including writing assignments (e.g., reflective writing, journals, essays, reports), group projects, and peer evaluations.

#Observation 5

The data also indicated that the teachers’ beliefs about teaching and learning also largely influenced their emotion when teaching in the new ALCs. Table 6 indicates that teachers’ transmissive views were negatively correlated with their positive feeling in using the ALCs. In other words, when teachers hold a traditional view on teaching and learning, they will not consider the use of the ALCs for teaching as enjoyable or feel enthusiastic about using them. 

The above analysis reflects that teachers’ beliefs about teaching and learning play an important role in their teaching in the ALCs. The results are consistent with the finding of a previous study which suggested that the innovative learning space can make significant differences only when the instructors plan their lessons accordingly (Gurzynski-Weiss, Avizia, & Solon, 2015). As there are more and more ALCs being set up in universities in both Taiwan and the US, there is a need to provide teachers with support in using these new ALCs to achieve higher level educational goals such as CPS abilities. Professional development is necessary to help teachers understand different perspectives on learning/teaching and active learning practices. 

The above analysis is based on teachers’ subjective response to the pre- and post-survey. To gain a broader understanding of how teachers used the new ALC to implement collaborative and constructive approaches in class, we also conducted classroom observation in two class meetings led by two different teachers. The study follows the evidence-based research methodology. Therefore, we used the class observation tool, GORP, developed at UC Davis to conduct the observation. The tool allows observers to record what activities students and teachers demonstrate during a class. The Pedagogy-Space-Technology framework was applied to guide the observations. In other words, we listed what teaching and learning activities teachers and students can do and what facilities/technologies they used to perform these activities in the class (Figure 7). When detecting a teacher using a facility to perform a teaching practice, the observer will click the corresponding icons to log the event. 

The observation result in one of the class observations is displayed in Figure 8. It was found that in this class, the most frequent learning activity demonstrated is small group learning and the most frequent teaching activity demonstrated is 1-on-1 discussion with students. The most frequently used facility is the whiteboard next to the round tables.

To obtain comprehensive information from the analysis, we applied Lag Sequential Analysis to the time sequence we obtained from the class observation. Such an analysis can obtain critical instructional paths during a class meeting that help us understand the overall teaching practice in a classroom. The analysis (as shown in Figure 9) found that the teaching in the classroom often started from posing a question by the teacher and then the teacher interacted with the students on a 1-on-1 basis. It is also observed that after writing on the whiteboard, the teacher often moved towards a student to interact with that student. The observations reflect that the teacher of this case study could effectively use the ALCs to implement a collaborative and constructive learning activity. 

The above results reflect some methodological considerations when studying ALCs. First, the classroom observation when assisted by computer tools can serve as a scientific basis to understand the teaching and learning practice in a new space design. Furthermore, the analytic techniques such as LSA on the observation data are helpful for depicting the overall teaching practice in the learning space. Last but not least, the study should consider pedagogy, space and technology as a guiding protocol to understand how a space supports a certain teaching practice. 

Impact of the Fulbright project

The CPS abilities have been deemed as one of the core competencies of the 21^st century. Therefore, the pedagogical design that can develop students’ CPS abilities in schools is becoming increasingly important. The results of this project contributed to the design of applications and learning spaces that make the CPS learning activities available to students on the campus and all around the world. In particular, this project has established a collaboration network between multiple countries including the European Union, Spain, Thailand and Singapore. With the help of the student volunteers in the US, the computer simulations we developed were translated into English and thus could be uploaded to the EU Next-Lab project platform (https://www.golabz.eu/) and can be accessed by students all over the world. 

The 2nd part of the work contributed to a better understanding of how ALCs may be studied to reveal the impact of the ALCs on teaching and learning. The study in the host institution and my prior works were integrated to form a Pedagogy, Space, and Technology (PST) model of space and technology design in supporting collaborative learning on campus. Due to the current emphasis on ALCs, I was featured by the IEEE Technical Committee on Learning Technology as one of the keynote speakers of The 18th IEEE International Conference on Advanced Learning Technologies in Bombay this July. 

Managing editor: Chiung-Yao “Carolyn” Ho 何瓊瑤

Reference

Alvarez, C., Salavati, S., Nussbaum, M., & Milrad, M. (2013). Collboard: Fostering new media literacies in the classroom through collaborative problem solving supported by digital pens and interactive whiteboards. Computers & Education, 63, 368-379.

Griffin, P., McGaw, B., & Care, E. (Eds.). (2012). Assessment and teaching of 21st century skills: Methods and Approach. New York, NY: Springer.

Hesse, F., Care, E., Buder, J., Sassenberg, K., & Griffin, P. (2015). A framework for teachable collaborative problem solving skills. In P. Griffin, B. McGaw, & E. Care, (Eds.), Assessment and teaching of 21st century skills: Methods and Approach (pp. 37-56). New York, NY: Springer.

Jermann, P. & Dillenbourg, P. (2008). Group mirrors to support interaction regulation in collaborative problem solving. Computers & Education, 51(1), 279-296.

Lin, P. C., Hou, H. T., Wu, S. Y., & Chang, K. E. (2014). Exploring college students’ cognitive processing patterns during a collaborative problem-solving teaching activity integrating Facebook discussion and simulation tools. The Internet and Higher Education, 22, 51-56.

OECD(Organisation for Economic Co-operation and Development). (2013). PISA 2015: Draft collaborative problem solving framework. https://www.oecd.org/pisa/pisaproducts/Draft%20PISA%202015%20Collaborative%20Problem%20Solving%20Framework%20.pdf. Accessed 7 April 2015

Pear, J. J., & Crone-Tood, D. E. (2002). A social constructivist approach to computer-mediated instruction. Computers & Education, 38(1-3), 221–231.

Powell, S. & MacNeill, S. (2012). Institutional Readiness for Analytics A Briefing Paper. CETIS Analytics Series. JISC CETIS, 2012. http://publications.cetis.ac.uk/wp-content/uploads/2012/12/Institutional-Readiness-for-Analytics-Vol1-No8.pdf.

Tan, J. P. L., Caleon, I. S., Jonathan, C. R., & Koh, E. (2014). A Dialogic Framework for Assessing Collective Creativity in Computer-Supported Collaborative Problem-Solving Tasks. Research & Practice in Technology Enhanced Learning, 9(3), 411-437.