10:00—10:20  

Welcome

10:20—10:40  

Masaaki Mochimaru, AIST

10:40—11:10  

Alois Knoll, TUM

11:10—11:40  

Rüdiger Dillmann, KIT / FZI Karlsruhe

Translation of Neuromorphic Principles Towards Closed Loop SNN-based Sensomotoric Robot Controls

The long term goal of this research is to model, translate and understand biomorphic neural principles focusing on brain related senso-motoric control tasks and to ground it with the help of real robots. This involves interdisciplinary and collaborating research as it is done within the HBP. Spiking neural networks (SNN) have the potential on replicating real neurons reflecting parts of their biological characteristics. SNNs are capable to perform synaptic spike-based communication, they allow modeling of local brain functionalities including learning and plasticity mechanisms. We assume, that the brain is forming sensor-motor primitives as building blocks for prediction, generation and execution of motions. The combination of neural simple motion primitives represent muscle synergies and yield towards more complex and advanced motions. The authors present results of their research on bio-inspired SNN control architectures capable to perform tasks like neural object recognition, object tracking, target reaching and grasping as well as collision- and obstacle avoidance. The experimental evaluation with a robot arm in closed loop control mode is performed without any planning algorithms nor calculation of kinematic transformations. Spiking neural network are used to represent motions in a hierarchy of motor primitives. Biologically plausible reward-learning rules based on synaptic sampling show that the SNNs are capable of learning performing policies. Neuroscientists have established links between reward-modulated synaptic plasticity and online reinforcement learning.

The hyper-parameters of this neuromodulation and their impact on performance were studied on closed-loop sensorimotor experiments. The potential of deep reinforcement learning for target reaching affects object interaction, manipulation and grasping tasks and can contribute to adapt its execution to different situations. An active binocular DVS system is used in stereo mode is driven by micro saccades.

The SNN feedback information from the DVS vision system and from proprioception are coupled for executing reaching movements and in general for motion generation.

Future work is related to an effective use of neuromorphic vision with emphasis to eye movement, micro saccades, visual affordance learning and visual prediction. Biomorphic navigation and mapping (BSLAM) requires episodic neural memories supported by multi-scale learning capabilities.

11:40—12:10

Shingo Shimoda, RIKEN

Title: Bottom-up stimulation of dormant muscles for post-stroke patient rehabilitation

One of the most important features of biological control system is its bi-directional control method. Biological control system with the feature can be modeled as the hierarchical system where the top-down signal to create the prime movements of the voluntary motions and the bottom-up control loop to help the prime movements adapting to the environment play the major roles of behavior control. In some cases of motion paralysis by stroke, the bottom-up control loops are survived though the patients cannot control their limbs. We developed the system to stimulate the local control loop for the grasping motions using the grasping support robot. We tested the robot with the post-stroke patients and observed the muscle activities on the paralyzed arm. We are planning to apply the bottom-up stimulation for the efficient rehabilitation.

12:10—13:40

Lunch Break

13:40—14:10  

Keynote speech: Yasuo Kuniyoshi, U of T/RIKEN

VR-based Neuro-Rehabilitation for Alleviating Phantom-Limb Pain

14:10—14:40  

Hidekazu KANEKO and Ko AYUSAWA, AIST

Motion analysis of a mechanical intervention intended to facilitate learning in rats

Robotic rehabilitation has the potential to facilitate the plasticity of injured neural networks by inducing motor sensations. In previous studies, we demonstrated that the application of external force near the response time of a choice reaction-time task facilitates learning in rats. The rats were trained to respond to a tactile stimulus (air puff) by releasing a lever pressed by the stimulated or non-stimulated forepaw. When an error trial was followed by a lever activation trial in which a lever on the incorrect-response side was automatically elevated at a preset time close to the rats’ responses, the performance improvement rate was increased by the lever activation procedure. In this study, we tried to analyze rats’ motions after lever activations. Using motion capture with infrared-red reflective markers, we noticed that the movements of the skin were not necessarily the same as those of the joints. However, these discrepancies in movements between the skin and joints were small for markers placed on the dorsal side along the trunk of a rat. Therefore, we analyzed the movements of the trunk. We found that the trunk moved to the dorsal side upon incorrect-side lever activation, i.e., the rat swung back due to lever activation. This movement may have induced a motor sensation similar to that accompanied by the correct voluntary response. We hypothesized that learning can be facilitated by inducing a motor sensation similar to that resulting from the correct voluntary response, but not by inducing motion.

14:40—15:10

Alexander Woodward, RIKEN

Computational techniques for 3D brain atlas construction of the Common Marmoset

15:10—15:30  

Coffee Break

15:30—16:00

Oleg Gusev, RIKEN

Transcriptional heterogeneity in primate skeletal  muscle in normal and pathological conditions

16:00—16:30

Takayuki Michikawa and Atsushi Miyawaki, RIKEN

Large-scale optical measurements of neural activity evoked by sensory stimuli in the mouse brain

Sensory information is considered to be essential not only for controlling ongoing movements but also for updating internal models of the motor apparatus and the external world using sensory prediction error-based learning. Here we show systems-level neuronal dynamics evoked by sensory stimulations in both the cerebrum and the cerebellum using a newly developed brain-wide imaging technique and engineered calcium reporter mouse lines. In the cerebral cortex, different sensory stimulations, such as vibrissae, auditory and visual stimuli, triggered responses in corresponding primary sensory areas with somatotopic, tonotopic and retinotopic maps, respectively, according to the localization theory which hold that specific functions are controlled by specific parts of the brain. In contrast, in the cerebellum electrical stimulation of four limb muscles individually evoked similar global responses over the cortex. Bayesian inference showed that the ensemble pattern of cerebellar cortical activities represents the timing and location of a stimulus. Our findings suggest that sensory signals are processed in discrete modules in the cerebrum while in spatially distributed, overlapped populations of neurons in the cerebellum, and that the difference in sensory processing may reflects the functional difference between the cerebrum and the cerebellum for motor control and sensory perception.

16:30—17:00  

Satoshi Oota, RIKEN

Development of computational animal models and prospects of the RIKEN-AIST joint project: Toward the ‘biological human augmentation (BHA)’

We are plunging into the ‘super-aging’ society (especially in Japan), which overcasts our future. To handle this issue with the robotics technology, we have been working for the RIKEN-AIST challenge project for the last two years. Our fundamental approach is not gadget-oriented functional enhancements or replacements, but the biological remodeling of mind and body. We call it the ‘biological human argumentation (BHA).’ The project spans from basic sciences to engineerings: e.g., gene expression analysis, electrophysiology, connectome analysis, biomechanics, virtual/augmented reality, evolution, and soft robotics. All activities are for the BHA through cognitive and physical interventions provided by an endoskeletal robot suit: StillSuit. To achieve our goal, it is evident that we need a deep understanding of the human as a biological system in terms of cognitive science, biomechanics, and genetics. We believe that our project has a high affinity to the neurorobotics framework, by which we can bridge multimodal data from the above different domains. We overview recent progress of the project, especially regarding the development of computational animal models, which are one of the essential hubs that integrate deep (invasive) data and big (non-invasive) data in cross-reference ways.

17:00—18:00  

Lab tour

18:00—20:00

Reception

10:00—10:30  

Keynote speech: Andreas Rowald

Targeted neurotechnology restores locomotion in humans with spinal cord injury

Epidural Electrical Stimulation (EES) applied over the lumbosacral spinal cord restored volitional control of locomotion in animal models and individuals with spinal cord injury (SCI). Here, I will describe the computational and scientific framework that supported the development of this therapy. We first used in vivo imaging and transgenic animal models to deconstruct the sensorimotor circuits engaged by EES. We then combined computational simulations and behavioural experiments to determine the mechanisms underlying the restoration of locomotion after paralysis when engaging these sensorimotor circuits. This knowledge translated into targeted neurotechnologies that modulate spatially-defined circuits with a temporal structure that reproduces the natural recruitment of these circuits during locomotion. This biomimetic neuromodulation strategies, termed spatiotemporal neuromodulation, is markedly more efficacious than continuous stimulation to facilitate locomotion. To mitigate the impact of inter-subject variability and the burden of time-consuming optimization procedures, we established a computational platform that supports the semi-automated creation of hybrid computational models from high-resolution MRI datasets of each human participant. These personalized computational models enable determining the optimal site of electrode array implantation, predicting the specificity of the stimulation, and identifying optimal electrode configurations. Computer simulations also guided the design of new spinal electrode arrays optimized for motor-related applications. These first-in-human data are paving the way towards a commonly available neurotechnology to improve the recovery of locomotion after spinal cord injury.

10:30—11:00

Ko Ayusawa, AIST

Human motion retargeting for simulating and analyzing human body and movement

When analyzing human movements, there is a difficulty in measuring and quantifying the forces generated inside human body. The motion analysis with a human anatomical model and a motion capture system enables the quantitative estimation. However, it often suffers from unmeasurable information or unknown model parameters. This talk will present an approach for human motion analysis by a humanoid robot through the technology called motion retargeting. The motion retargeting makes the robot to imitate human movements as closely as possible and minimizes the gap in the experiments between the human and the robot. The robot sensory information can provide the additional quantitative information in the human motion analysis. A more advanced challenge of retargeting between laboratory animals and humans is also introduced.

11:00—11:30  

Nobuo KUNORI, AIST

Modulation of sensory signal transmission by electrical DC stimulation

Transcranial direct current (DC) stimulation, a non-invasive brain stimulation technique, has been used in the clinical studies to improve cognitive and motor functions. However, underlying mechanisms of this treatment are not well understood. To investigate the effects of the DC stimulation to the cortex, we measured sensory responses in the primary somatosensory cortex (S1) and secondary motor cortex (M2) in the anesthetized rats. Forelimb-evoked sensory responses were recorded by VSD imaging and LFP recording with applying the DC stimulation to the S1 (0-350 µA, total stimulation time; 20 minutes). In the S1, the amplitude of the evoked-LFPs were increased during the DC stimulation to the S1 by two- to three-fold depend on the stimulation intensity. However, when we compared the response amplitude between the first- and last-five minutes of the stimulation period, the response amplitude was gradually decreased if the current density of the stimulation exceeding approximately 20 A/m2. VSD imaging performed in the last five minute of the DC stimulation period showed similar results. In the M2, the sensory responses after the forelimb stimulation was slightly increased by the DC stimulation to the S1, but no significance was observed between with and without stimulation conditions. These results suggest that the cortical DC stimulation increases cortical excitability locally at the stimulation site though the effectiveness can be changed along with time for the stimulation.

11:30—12:00  

Shusuke KANAZAWA, AIST

Flexible sensors based on printing technology toward the monitoring of human action and environment

Sensors made on thin- flexible substrate have potential to be novel interfaces to acquire unknown information which exists in our life. They have particularly advantage for the monitoring of human action due to their fitness to human body like clothes and bandages. In this session, we introduce some development on flexible sensors. In addition, printing technologies which we have developed to achieve the efficient manufacturing of flexible sensors is also presented.

12:00—13:30

Lunch Break

13:30—14:00  

Motion Analysis for Anybody Anywhere Anytime

Yoshihiko Nakamura, Yosuke Ikegami, Ayaka Yamada

University of Tokyo

14:00—14:30  

Bernd Froehlich, BAUW

The Virtual Reality Neurorobotics Lab

www.uni-weimar.de/vr

We are developing an innovative virtual reality environment for the collaborative and immersive interaction with real-time neurorobotics simulations. To this end, we coupled the Neurorobotics Platform (NRP) developed by the European Human Brain Project (www.humanbrainproject.eu) with an innovative soft- and hardware infrastructure for collaborative and distributed virtual reality. In our Virtual Reality Neurorobotics labs, users are captured by multiple RGBD-cameras in real-time and represented in the virtual neurorobotics simulation by realistic and life-size 3D avatars. Hence, our system does not only allow users to observe and provide input to the simulated robots in real time but also enables robots to detect the avatars through their virtual sensors and react accordingly. Thus, simulated robots can be trained and evaluated with complex inputs coming from real human full-body postures, gestures and movements. Furthermore, researchers from different locations can meet in and interact with ongoing experiments and discuss them as if they were standing next to each other. Besides fostering such virtual face-to-face collaboration, we think that our approach has the potential to raise the realism of real-time human interaction with robot simulations to an entirely new level.

14:30—15:00

Akihiko Murai, AIST

Designing motivation for sports: Environment modulation for tacit skill training

15:00—15:20

Coffee Break

15:20—15:50

Shintaro Oyama, Nagoya University Hospital Medical IT Center/RIKEN

Functional Reconstruction and Prosthesis

Numerous limb loss and deficiency occur every day worldwide. Adapting Prosthesis is one of major reconstruction options. Lower limb prosthesis has good success rate in younger patients but still low adaptation rate in elder patients. Furthermore, upper limb prosthesis, is still long way to go. We human use tacit and explicit mode to learn skills. The former takes place unconsciously, the latter consciously. Human movement derives from a complex process based upon tacit ability primarily acquired through experience. We think high success rate of gait reconstruction is because that gait movements mostly consists of gross motor movements and upper limb movements is complexed combination of fine and gross motor movements. Reconstruction of hierarchical control and optimal mapping of output redundant control signal to actuators is mandatory in reconstruction of upper limb.

15:50—16:20

Axel von Arnim, Fortiss

The Neurorobotics Platform: simulating and benchmarking brain-driven robots

The Neurorobotics Platform developed within the framework of the Human Brain Project is a suite of software tools that connect robotic simulation with neuroscience. Controlling robots with neural networks (spiking or classical) is a trend with considerable staying power, which no current robotic simulator enables natively. The strength of the Neurorobotics Platform lies in the seamless integration of physics simulation with neural simulation. Both simulators are embedded and linked in one comprehensive graphical tool that enables user to create complex closed-loop simulations where embodiment is key. On the one hand, neuroscientists and AI developers can test their neural models in a virtual world, with sensor input and command output. On the other hand, roboticist can try out neural controllers for their robots, benchmark them, test neuromorphic hardware, compare solutions and take informed decisions. The complexity of today’s robotic engineering makes neurorobotics simulation a must-have tool for developing robots with better cognitive abilities.

Furthermore, the Neurorobotics Platform provides muscle simulation, skeletal models and the tools required to simulate realistic musculoskeletal systems or biologically inspired compliant robots. In silico testing of spinal cord models or controllers for walking and, soon, swimming on biologically realistic virtual models is thus made possible.

With constant improvements and new capabilities added through a regular release schedule, the Neurorobotics Platform is the link between two worlds that are ever getting closer to each other."

16:20—16:50

Egidio Falotico, SSSA

AI-based approaches for soft robot control

16:50—17:20

Kunihiro OGATA, AIST

Rehabilitation Robot to Assist Motor Learning Based on Voluntary Movement

Hemiplegia patients have complete paralysis of half their body, and encounter many challenges in living an independent life. Rehabilitation of the lower body is more important than that of the upper body for independent living; thus, recovering upper body functions of their paralyzed side is not enough. Rehabilitation robots may be used to assist training without therapists. In this study, a small portable rehabilitation robot was developed for use at home, and a new training method was proposed. This robot consists on an omni wheel mechanism and a force sensor, and is capable of deciding the motion based on the force value. Voluntary movement of a hemiplegia patient is recovered by the rehabilitation robot and proposed training method. Thus, verification experiments were performed using participants with hemiplegia. The CCI (Co-Contraction Index) from after training were smaller than ones of before training, thus the movement skills of the participants improved with respect to controlling force direction and magnitude. Moreover, manual function test (MFT) scores increased as reflected by improvements in the motor function of the upper limb using the proposed training method.

17:20—17:30

Closing