 |
| (Both images dated 1632). |
(10 minute read)
Almost 400 years ago, Rembrandt painted the ‘Anatomy Lesson of Dr. Nicolaes Tulp’ showing a medical lesson in progress as Dr. Tulp dissects a cadaver. To overcome many challenges (including dealing with cutting up dead bodies), the modern medical world is using VR, AR and 360 video to support learning.
I wonder which format Rembrandt would have chosen?
On 21st November I attended the Media and Education special interest group ‘Seminar on interactive video, virtual reality and 360 degree video in education’ at PLNT in Leiden. As a member of the Media and Education special interest group, I'd been asked to prepare a short overview on empirical evidence-based research regarding what we know about how this type of video supports learning. Is there a relationship with the degree of immersion and the involvement of the learner? And under what circumstances does it support the desired learning results.
AR, VR and 360 Video
In Augmented Reality, devices such as the HoloLens, a pair of transparent screen glasses project a 3d virtual image over reality and the user can interact with the virtual 3d object in a real physical space. Click points can be added at various locations and objects can be manipulated and rotated.
In Virtual Reality, the degree of immersion can increase resulting in what Mel Slater calls the three illusions of VR
- Place illusion; feeling of being in a virtual place, even though you know you are not there.
- Plausibility illusion; the degree of how real you take the events to be.
- Body ownership illusion; believing the body you see in the VR world is actually your body.
Within education, AR, VR and 360 video is being used in many ways. To give access to students to learning experiences that would normally be inaccessible, too expensive, too dangerous, impractical or impossible. Disciplines in which this is being used and researched include Medicine, Psychology and Education. 360 video enables you to step into a Ubiquitous 360 Learning Environment (ULE). With 360 video, the camera can be fixed or moving and can take still images or a video with sound. The video can become interactive with different scenarios being selected by the user.
While the user is interacting with the video, a huge amount of data can be collected. A common technique is creating heat maps of viewer eye movements. YouTube automatically generates a heat map in its 360 videos for and video with more than 1000 views.
< !--[if !supportLineBreakNewLine]-->
< !--[endif]-->
 |
| Listening to a healthy lung (Augmented Reality) viewed through HoloLens and a plastic coffee cup! |
I have clocked limited time in the VR world and have probably been immersed for less than 60 minutes total in my entire life. So I am a novice rooted in the 3d world interacting with screens via smartphones, tablets and computers. I’ve had occasional encounters with VR glasses at trade show or watched a 3d travel video at a fair.
As a teacher I greatly value eye contact and attention from my students when in class, and place specific emphasis on students being physically and mentally present during the learning process (e.g. distracting lap top and smartphone away during instruction and discussion).
As a teacher I greatly value eye contact and attention from my students when in class, and place specific emphasis on students being physically and mentally present during the learning process (e.g. distracting lap top and smartphone away during instruction and discussion).
So what would my pedagogical approach be when my entire class is wearing VR glasses and there is no eye contact? Actively sending students into the VR world of Pandora as Avitars also requires waking them from their reverie, and what happens during this transition from virtual back to the real world? Could the teacher enter the VR world as a pedagogical presence in order to give instructions and guide learners and hold their attention? Would students be more focused on the digital avatar teacher than the real teacher in the class?
I was curious whether these questions would be addressed by the four presentations during the seminar.
< !--[if !supportLineBreakNewLine]-->
< !--[endif]-->
I was curious whether these questions would be addressed by the four presentations during the seminar.
< !--[if !supportLineBreakNewLine]-->
< !--[endif]-->
Nikki Verseput of Radboud Universiteit uses 360 video on the forensic pedagogy course.
Using a 360 camera, live footage was shot in a law court in which a judge was discussing a real case in court regarding juveniles (present, but anonymised in the video). Wearing VR glasses, students enter the virtual world (25 pairs of glasses, students worked in pairs to take it in turns). In the real world it is not possible for 50 students to be so closely involved, but this format gives students a realistic image of professional practice. Due to the extreme privacy and issues of sensitivity regarding the video content, it could not be shared publicly. There was a steep learning curve for students to use the headset, but in class they felt deeply immersed as they were dropped into the real world environment.
Using a 360 camera, live footage was shot in a law court in which a judge was discussing a real case in court regarding juveniles (present, but anonymised in the video). Wearing VR glasses, students enter the virtual world (25 pairs of glasses, students worked in pairs to take it in turns). In the real world it is not possible for 50 students to be so closely involved, but this format gives students a realistic image of professional practice. Due to the extreme privacy and issues of sensitivity regarding the video content, it could not be shared publicly. There was a steep learning curve for students to use the headset, but in class they felt deeply immersed as they were dropped into the real world environment.
In the first viewing, the judge from the court room sat in the back of the classroom watching the students experience the court, and was available to answer questions in person, to the delight and surprise of the students who were highly motivated by the whole experience. The next step is planned for all students to view one environment synchronously with the teacher playing a didactic role during the viewing to step in and discuss issues.
For this course, the focus was not on learning standard knowledge, but more about letting students have the experience in a real world setting, viewing at their own pace, and choosing where to look. Evaluation data was collected from students based on the ARCS model of motivational learning (Keller, 1987) and confirmed a high level of motivation due to the intense immersive experience. Students found it fun and engaging, clearly explained, related to the course content and valuable. Students said the VR added to clarification in course content, increased interest in the course, better understanding of material and good preparation for work.
Joris van Hamersveld of Hogeschool Windesheim showed us the VR world developed to help students practice presentation skills. Joris had taught presentation skills in various contexts and always found it difficult to provide objective data to support the assessment process. When do you give what feedback and how (without interrupting the presentation)?
 |
| Annemiek practices presenting to a virtual audience, but we're real! |
The company BrainStud developed the system SpeechSupport. Over three years, an audience of twenty 3d computer graphic individuals has been created. Each unique individual has 20 minutes of animation expressing different levels of interest and involvement in the presentation (from full eye-contact, to looking at smartphone or putting the feet on the table). The audience sit (randomly allocated each session to different places) in a theatre or university lecture hall setting.
The student learns their presentation in advance and then enters the practice arena by putting on the VR headset. Holding two motion sensors in the hands, the system tracks hand and body movement, where the presenter is looking (gaze heat-map) and tone of voice (based on volume and tone variation). During the practice presentation, the speaker tries to engage with the 20 members of the virtual audience, speaking clearly and loudly enough with variety of intonation, moving enough and creating eye contact. A second participant (fellow student), manipulates the interest level of the audience (making them more bored, decreasing eye contact, audience start checking their phones, or full eye contact). Disruptions can be added (noise outside, phone ringing) and the student practising has to try to stay on track with their presentation. All this data is recorded and available for analysis and feedback. Although students said they found this setting unrealistic, they were still able to gain valuable practice time.
After the session I got to experience it myself and it was very interesting. I was in the classroom for three minutes. Wearing the VR headset, with headphones and two hand held devices, I found myself standing on a university lecture hall stage with 20 students spread out across the room. Although I had no speech to practice, I found myself trying to engage the audience and to keep them interested. As my partner in the real world manipulated the interest level of the audience, they all began to lose interest and I found myself struggling to get them back on board.
 |
| A heat map of a speakers voice. Vertical axis is volume, horizontal is tone. |
This was a very interesting format and one that has taken a huge amount of work to programme and which I hope can be made available to a wider audience. My 11 year-old daughter who gets very nervous presenting, is preparing her book presentation for her class on Friday, and she was particularly interested in the idea of being able to practice her presentation in a safe virtual world!
 |
| Practicing a bad news conversation. |
Jantine te Molder and Sjors Groeneveld of Hogeschool Saxion presented their Serious Talks programme which they’ve developed. How should someone handle a ‘bad news’ conversation with a patient? A student puts on the VR headset and finds themselves sitting in a room facing a table and chair. The door opens and a patient comes in and sits down. The student chooses on the screen from three options of what to say. Based on which option is clicked, the scenario follows different routes. This required a complex ‘Christmas Tree’ of options, and 40 different scenes to be filmed, each one in which the actress had to react accordingly to the specific information. The app was developed as an intermediate step between theory and role play. When students started using the app, rather than following the correct process to handle the ‘bad news’ conversation, they deliberately chose the worst answers to see the actress become upset and angry.
 |
How good are you at telling someone bad news? Uh-oh... I just said the wrong thing. |
After the session I had the chance to try out the scenarios myself and immediately followed the students’ examples, picking the answers that were clearly wrong. Quite soon I had a very angry, red-faced and upset patient standing up and leaning forward screaming at me. It was certainly unnerving but I could experience the result of my inappropriate choices without actually upsetting anyone. A very good learning tool and a concept that can be scaled up in different contexts. You can download the app (for Android) here:
Roxy van de Langkruis from Hogeschool van Amsterdam uses 360 in the forensic research course.
Imagine you are a team of detectives in training. At the crime scene, the door is closed, but inside you know there is a dead body. What do you need to look at, and take into consideration? What clues can you find and what is your search strategy? A crime scene was created and photographed with a 360 camera from five different positions. Outside the room (door closed and open), inside the room in three positions. In addition, each of the five photo positions was photographed at two heights (standing, and squatting). Clues and objects were positioned in the scene including a knife outside the closed door if you zoom in carefully, and the body inside which was hard to miss!
Imagine you are a team of detectives in training. At the crime scene, the door is closed, but inside you know there is a dead body. What do you need to look at, and take into consideration? What clues can you find and what is your search strategy? A crime scene was created and photographed with a 360 camera from five different positions. Outside the room (door closed and open), inside the room in three positions. In addition, each of the five photo positions was photographed at two heights (standing, and squatting). Clues and objects were positioned in the scene including a knife outside the closed door if you zoom in carefully, and the body inside which was hard to miss!
 |
| Suicide or murder? And did you just step in an important clue..? |
This format was the only version demonstrated that did not need a VR headset to view and did not have a moving image. It was watched and navigated through a computer screen. The student can move around the space, look in all directions, zoom in, squat down to check under tables or chairs, add comments and questions as they go along and take their time to understand the scene. Of particular interest is the log book that is generated for each student. This details all actions and movements. Some students spent a total of two minutes, others twenty. The log data was shared back to each project group and could be analysed and reflected upon as a group, resulting in one reflective report. The assessor could then comment on the search strategy and the critical reflective abilities of the students in assessing their own work. Student Merel de Geest who had participated in the first prototype had also analysed much of the data and was enthusiastic in how this format had helped her learning. 34 usable log books have been analysed and the data is currently being processed for a report. Since the crime scene is frozen in time it is not necessary in this format to use video and a five different 360 photo points provide more than enough data.
< !--[endif]-->
Enriched learning experiences
These four presentations showed very interesting, creative and well worked out scenarios in which AR, VR and 360 video was being used to support education. The technology and programming needs to work, and the complex scenarios need to be clearly aligned to learning goals and assessment. Much of the discussion revolved around simply letting students ‘experience’ the environment, giving them increased time on task, and that this has positive impact on engagement. But what happens in the future when all classes are super realist immersive 4d environments? Will this learning format loose its innovative edge?
At the end of the session, as a member of the Media and Learning special interest group, I was given the floor to summarise the presentations in the context of evidence-based research. Since we had run over by 45 minutes (!) I only had a few minutes to wrap up. The presentations connected with themes seen in the research (including increased learning experience and motivation linked to the immersive nature of the learning environment) and managing issues such as getting technology to work and dealing with CyberSickness.
Outside the seminar there was a VR event ‘Think outside the books!’ arranged by the University of Leiden where I entered a gold mine in Ghana and almost listened to an unhealthy 3d lung.
Rembrandt's choice?
So which format would Rembrandt have chosen? I think Rembrandt would have opted for the 360 fixed image photo scene, with some incredible contrasts of light and dark, and some amusing practical jokes added in the dark corners...
What do you think?
Selected sources (focused mainly on 360 video)
Selected sources (focused mainly on 360 video)
Aitamurto, T. (2018). Normative paradoxes in 360° journalism: contested accuracy and objectivity. New Media and Society. http://doi.org/10.1177/1461444818785153
Narciso, D., Bessa, M., Melo, M., Coelho, A., & Raposo, J. V. (2017). Immersive 360◦ video user experience: impact of different variables in the sense of presence and cybersickness. Universal Access in the Information Society. http://doi.org/10.1007/s10209-017-0581-5
Rupp, M. A., Odette, K. L., Kozachuk, J., Michaelis, J. R., Smither, J. A., & Mcconnell, D. S. (2019). Investigating learning outcomes and subjective experiences in 360- degree videos. Computers & Education, 128(August 2018), 256–268. http://doi.org/10.1016/j.compedu.2018.09.015
Sundar, S. S., Kang, J.,& Oprean, D. (2017). Being there in the midst of the story: how immersive journalism affects our perceptions and cognitions. CyberPsychology & Behavior, 20(11), 672–683. http://doi.org/10.1089/cyber.2017.0271
Virtanen, M. A., & Kääriäinen, M. (2017). Use of ubiquitous 360° learning environment enhances students’ knowledge in clinical histotechnology: a quasi-experimental study. Medical Science Education, 27, 589–596. http://doi.org/10.1007/s40670-017-0429-x