The avatar, or robot, will have to move to match the way the patient moves, and this convergence will encourage the patient to close the ‘social’ gap with the artificial agent required for the exchange. To this end, a capturing and modeling module is being developed, in order to map the relevant and salient features of the patient’s behaviour onto the animated artificial agent.
More technically, this entails three innovations that are needed to achieve the ‘perception’ module of the avatar:
i) new 3D body scanner technologies, including shape, appearance (colour, texture) and animation,
ii) new combination of new filtering and fusion algorithms for body modeling from coarse depth data
iii) new combination of motion tracking systems.
Complementing this perception module, the artificial agent also includes a decision component, which enables the device to establish, from the patient’s behaviour, a pro-social vs. avoidance/deficient profile. This enables the agent to mimic this profile, which facilitates the social interaction. A robot and avatar controller is being created to transform the perception of human agent movements into the artificial agent’s social reaction. Finally this initial stage produces reliable mathematical models that allow real-time adaptation of the robot’s/avatar’s behaviour in order to maintain an efficient interaction.
In addition to the design and implementation of similarity and its variation in time, AlterEgo now compares the impact of virtual reality (avatar) versus real presence (robot) of the artificial therapeutic agent. The humanoid robot iCub is being used. The patient moves from interacting with the iCub’s avatar or human’s avatar (doctor or confederate) to interacting with the iCub robot or with a human (doctor or confederate). After having created the conditions for an efficiently coordinated interaction, mainly based on the robot’s and avatar’s effort to converge toward the patient, the second challenge of the project is to set the stage for a rehabilitation scheme. This requires smoothly changing the avatar’s behavior to increase the difference with the patient, which forces the patient to share more and more of the workload to sustain a cooperative performance. In doing so, AlterEgo scientists assume that robust change will be promoted in the patient’s social skills repertoire. In this project modeling, engineering and experiments go hand-in- hand in an iterative manner, where modeling and engineering assist the experimental design, and the experimental results will be used to validate and modify and/or refine the models and the artificial agent devices accordingly. Consequently, all five partners are necessarily fully engaged in the project.
Read further HERE.