Designer and Researcher for Carnegie Mellon Articulab

An experience that allows virtual agents to tangibly interact with children

See our project in this video

Background

The SCIPR project - housed by Carnegie Mellon's Articulab - is working to bridge the gap between virtual agents, education, and the physical world. By investigating how virtual agents can extend beyond their digital confines to affect their environment in tangible ways, SCIPR seeks to understand how collaboration and curiosity in learning can be improved. In terms of technology the SCIPR project aims to fulfill a research gap for virtual agents to interact without VR or AR means. 

As a team, we were tasked with exploring and designing how to extend virtual agents’ influence to the physical world. We sought to provide valuable research and insight on three specific criteria: collaboration between virtual agents and children, curiosity and exploration, and player engagement.

Process

Research

We performed a literature review, conducted expert interviews, and analysed videos of children in play provided by the Articulab to inform our research phase.

Major Findings

  1. A robot with non-humanoid form can still elicit natural human interactions with a child.

  2. Virtual agents can interact effectively in some cases through the use of a proxy.

  3. Digital visualization and sensing can be as important to a successful experience as responsive actuation.

  4. Project, robotics, and digital visualization have been used in various scenarios to create collaborative and engaging environments in play

Lo-fi Prototyping

Our lo-fi prototypes were used to test different activities and modes of interaction. We wanted to see how these different prototypes would affect collaboration, engagement, and curiosity. We simulated the possibility of the virtual agent interacting through a smart surface and through the means of a proxy (we named it Beep Boop)

Mid-fi Prototyping

For our mid-fidelity testing, we focused on testing different ways of embodying the ECA in the physical space. To create a shared experience, we created a 2.5D game "Marble Run"- in this game children would work with the virtual agent to create a physical course for a marble to reach a goal. We prototyped and tested three conditions that varied in forms of ECA embodiment:

Hi-fi Prototyping

The goal of this next phase of testing was to improve upon the actuation techniques utilized in our mid-fidelity, as well as create a more polished and replicable testing environment. We hoped to specifically test a virtual agent’s reception as a player rather than a teacher, collaboration in gaming, and communication between children and virtual agents.

We created a table to create different Wizard of Oz techniques which would the virtual agent to draw on a table surface and manipulate physical blocks without breaking a "digital" wall. The conditions include:

Results

Final Takeaways

  • The virtual agent needs to feel like another human peer, not a teacher
  • Creating a personality through physical interaction allows users to connect with the virtual agent 
  • Building rapport between the virtual agent and the user allows for better collaboration
  • Children users always assume the virtual agent is right
  • Participants saw Alex as a helpful and supportive player

Recommendation for Client

At the project's end we handed over all research and prototypes to the Articulab.

We recommended they create a hi-fidelity apparatus with mixed technologies of both physical actuation and visualization technologies.