Objectives: The project goal is to design, implement and test a robotic system (arm with a gripper, and two cameras) that would, in an interaction with a human, learn to execute commands, respond to questions, comment its own actions as well as those of a human. To make the tasks manageable, the interaction will focus on simple sensory-motor actions, such as object manipulation on a table. Project aims to yield novel insights into embodied cognitive processing and especially its developmental aspects, since most of the mechanisms will not be pre-programmed, but learned, using mostly artificial neural networks. In the sensory-motor domain, learning will be achieved by reinforcement or by human instruction, in the language domain by the human naming his/her own as well as robot’s actions. At first, different subtasks will be solved separately using the robotic simulator and then will be integrated in a physical robotic system with the same architecture.
Objectives: Technological advancements based on mixed reality (MR) offer various challenges for research and medical treatment. The project focuses on two objectives related to healthy subjects and hemiparetic patients after stroke. First, we will test the hypothesis whether cognitive training using appropriately designed MR environment will enhance perceptual and cognitive performance in healthy subjects. This will be tested by computerized psychological experiments as well as by measuring event-related potentials or ERPs. Second, we will test the hypothesis whether experience with training in MR (in combination with motor-imagery based brain-computer interface developed by us) will enhance oscillatory sensory-motor rhythms. This will be tested by measuring subject’s EEG activity before and after each training session, clinical testing, as well as by the questionnaires aiming to learn about human factors including mental fatigue, motivation, irritation or sleepiness due to training. In both objectives, we will design and implement a set of testing procedures, carry out a battery of dedicated experiments, and critically evaluate the results with the goal to validate MR designs.