計測自動制御学会 フェロー

平松 知樹, 今井 将人, 山田 弘幸, 河村 晃宏, 倉爪 亮, 小型センサ端末によるパーソナルモビリティ・ビークルの誘導制御システムの開発, 日本機械学会ロボティクスメカトロニクス講演会2018, pp.1A1-L10, 2018.6.2-5

ロボットミドルウェアROS の普及への貢献

IEEE Senior Member

日本機械学会 フェロー

計測自動制御学会 システムインテグレーション部門 学術業績賞

日本ロボット学会 フェロー

新崎 誠, 岩下 友美, 小川原 光一, 倉爪 亮, 統計的モデルを用いた見えの変化 に頑強な歩容による個人識別, 第20回ロボティクスシンポジア, pp.342-347, 2015.3.15

小山 翔平, 岩下 友美, 倉爪 亮, "クワッドコプタ搭載LRFと単眼カメラによるBaySACを用いた地表面計測", 第15回計測自動制御学会システムインテグレーション部門講演会 SI2014, 1M2-2, 2014.12.15

新崎 誠, 岩下 友美, 小川原 光一, 倉爪 亮, "統計的モデルを用いた見えの変化 に頑強な歩容認証", 第4回バイオメトリクスと認識・認証シンポジウム, O-1-1, 2014.11.26

Hojung Jung, Oscar Martinez Mozos, Yumi Iwashita, Ryo Kurazume, Indoor Place Categorization using Co-Occurrences of LBPs in Gray and Depth Images from RGB-D Sensors, Fifth International Conference on Emerging Security Technologies (EST-2014), pp.40-45, Alcala, Spain, Sep. 10-12, 2014

鄭 龍振, 倉爪 亮, 岩下 友美, 長谷川 勉, "大規模な3次元環境地図とRGB-Dカメラを用いた移動ロボットの広域位置同定", 日本ロボット学会誌, Vol.31, No.10, pp.896-906, 2013

園木秀治, 諸岡健一, 倉爪亮, 橋爪誠, 長谷川勉, オーダーメイド医療のための軟性肝臓モデルの物性パラメータ推定, 第31回計測自動制御学会九州支部学術講演会, 2012.12.9

受賞理由：複数移動ロボットの協調制御やロボットビジョン，医用画像処理，環境情報構造化など多くの研究分野で研究開発を行い，学術・技術両面で高く評価されている．また，研究成果の産業応用やロボット教育にも積極的に取り組み，ロボティクス・メカトロニクスの分野の発展に寄与した．

大石 修士, 倉爪 亮, 岩下 友美, 長谷川 勉, Trilareral filterとBelief Propagationを用いた距離画像の平滑化及び欠損修復, 第16回ロボティクスシンポジア講演予稿集, pp.1-7, 2011.3.14

Yukihiro Tobata, Ryo Kurazume, Yumi Iwashita and Tsutomu Hasegawa, Automatic laser-based geometrical modeling using multiple mobile robots, 2010 IEEE International Conference on Robotics and Biomimetics (ROBIO 2010), pp.363-369, Tianjin, Dec. 14-18, 2010 (T.J.Tarn Best Paper in Robotics)

Yumi Iwashita, Ryosuke Baba, Koichi Ogawara, Ryo Kurazume, "Person identification from spatio-temporal 3D gait", Int. Conf. on Emerging Security Technologies, pp.30-35 (2010.9)

山田 弘幸, 倉爪 亮, 村上 剛司, 長谷川 勉, SIR/MCMCパーティクルフィルタを用いたカメラとレーザレンジファインダによる複数移動体の同時追跡, 第13回ロボティクスシンポジア講演会予稿集, pp.252-258, (2008.3).

学会活動への貢献

岩下 友美, 倉爪 亮, 原 健二, 長谷川 勉, 局所境界の移動方向予測値に基づく実物体の高速な3次元形状復元, 第7回計測自動制御学会システムインテグレーション部門講演会講演予稿集, pp.914-915, 2006.12

長谷川 勉, 浦本 征吾, 向井 大樹, 倉爪 亮, 村上 剛司, 直接操縦から自律制御へのオンライン移行型遠隔作業ロボットシステム 対象物体のポインティング指示と動作プリミティブによる自動作業, 第6回計測自動制御学会システムインテグレーション部門講演会, 1A3-2, 2005 12

吉田 和哉，倉爪 亮，梅谷 陽二， 宇宙ロボットの多腕協調制御(スタビライジングアームの利用による 制御トルクの最適化)， 日本ロボット学会誌, Vol.9,No.6, pp.718-726, (1991).

Publishing type：Research paper (international conference proceedings)   Publisher：Proceedings - IEEE International Conference on Robotics and Automation  

Generative modeling of 3D LiDAR data is an emerging task with promising applications for autonomous mobile robots, such as scalable simulation, scene manipulation, and sparse-to-dense completion of LiDAR point clouds. While existing approaches have demonstrated the feasibility of image-based LiDAR data generation using deep generative models, they still struggle with fidelity and training stability. In this work, we present R2DM, a novel generative model for LiDAR data that can generate diverse and high-fidelity 3D scene point clouds based on the image representation of range and reflectance intensity. Our method is built upon denoising diffusion probabilistic models (DDPMs), which have shown impressive results among generative model frameworks in recent years. To effectively train DDPMs in the LiDAR domain, we first conduct an in-depth analysis of data representation, loss functions, and spatial inductive biases. Leveraging our R2DM model, we also introduce a flexible LiDAR completion pipeline based on the powerful capabilities of DDPMs. We demonstrate that our method surpasses existing methods in generating tasks on the KITTI-360 and KITTI-Raw datasets, as well as in the completion task on the KITTI-360 dataset. Our project page can be found at https://kazuto1011.github.io/r2dm.

DOI： 10.1109/ICRA57147.2024.10611480

Scopus

researchmap

Other Link： https://dblp.uni-trier.de/db/conf/icra/icra2024.html#NakashimaK24

Publishing type：Research paper (scientific journal)  

DOI： 10.48550/arXiv.2405.04889

researchmap

Language：Japanese   Publisher：Kyushu Association for Bridge and Structural Engineering  

Collaborative AI robots for adaptation of diverse environments and innovation of infrastructure construction

DOI： 10.60345/kabse.39.0_11

CiNii Research

researchmap

Language：Others   Publishing type：Research paper (scientific journal)   Publisher：IEEE Access  

Gait recognition is a biometric identification method based on individual walking patterns. This modality is applied in a wide range of applications, such as criminal investigations and identification systems, since it can be performed at a long distance and requires no cooperation of interests. In general, cameras are used for gait recognition systems, and previous studies have utilized depth information captured by RGB-D cameras, such as Microsoft Kinect. In recent years, multi-layer LiDAR sensors, which can obtain range images of a target at a range of over 100 m in real time, have attracted significant attention in the field of autonomous mobile robots and self-driving vehicles. Compared with general cameras, LiDAR sensors have rarely been used for biometrics due to the low point cloud densities captured at long distances. In this study, we focus on improving the robustness of gait recognition using LiDAR sensors under confounding conditions, specifically addressing the challenges posed by viewing angles and measurement distances. First, our recognition model employs a two-scale spatial resolution to enhance immunity to varying point cloud densities. In addition, this method learns the gait features from two invariant viewpoints (i.e., left-side and back views) generated by estimating the walking direction. Furthermore, we propose a novel attention block that adaptively recalibrates channel-wise weights to fuse the features from the aforementioned resolutions and viewpoints. Comprehensive experiments conducted on our dataset demonstrate that our model outperforms existing methods, particularly in cross-view, cross-distance challenges, and practical scenarios.

DOI： 10.1109/ACCESS.2023.3333037

Web of Science

Scopus

researchmap

Language：Others   Publishing type：Research paper (other academic)   Publisher：IECON Proceedings (Industrial Electronics Conference)  

It is difficult to automate the grasping of delicate fruits such as peaches, which require sensitive work. Envelope grasping is an effective approach for grasping such objects due to its large contact area with the object. A method to realize envelope grasping is the fingered gripper, which can perform grasping that follows the shape of the object through compliant interaction. However, this method has the problem that it is structurally difficult to fill the space between fingers. Another method of achieving enveloping grasp is to envelop the object by membrane. This method can touch the object without gap between fingers, but it is difficult to grasp unexpectedly large objects because it cannot deform as large as a fingered gripper. In this study, we propose a soft gripper that realizes envelope grasping while largely deforming several fingers. The proposed method opens and closes using the turning over of the hemispherical shell due to the 3D snap through buckling mechanism. By employing this mechanism, allowing all objects touching the edge of the hemispherical shell to be moved simultaneously and significantly, fingers arranged without gaps can be deformed largely. Another advantage is that it requires no energy to maintain the grasping state due to its bistable, snap through buckling characteristics. In this paper, we present a design methodology based on specific examples and an evaluation of performance.

DOI： 10.1109/IECON51785.2023.10312018

Scopus

researchmap

Other Link： https://dblp.uni-trier.de/db/conf/iecon/iecon2023.html#HanamoriKK23

Language：Others   Publishing type：Research paper (other academic)   Publisher：Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)  

Cardiac magnetic resonance (CMR) imaging is widely used to visualise cardiac motion and diagnose heart disease. However, standard CMR imaging requires patients to lie still in a confined space inside a loud machine for 40–60 min, which increases patient discomfort. In addition, shorter scan times decrease either or both the temporal and spatial resolutions of cardiac motion, and thus, the diagnostic accuracy of the procedure. Of these, we focus on reduced temporal resolution and propose a neural network called FusionNet to obtain four-dimensional (4D) cardiac motion with high temporal resolution from CMR images captured in a short period of time. The model estimates intermediate 3D heart shapes based on adjacent shapes. The results of an experimental evaluation of the proposed FusionNet model showed that it achieved a performance of over 0.897 in terms of the Dice coefficient, confirming that it can recover shapes more precisely than existing methods. This code is available at: https://github.com/smiyauchi199/FusionNet.git.

DOI： 10.1007/978-3-031-47425-5_4

Web of Science

Scopus

researchmap

Other Link： https://dblp.uni-trier.de/db/conf/miccai/misail2023.html#ChangMMKM23

Language：Others   Publishing type：Research paper (scientific journal)  

DOI： 10.48550/arXiv.2309.09256

researchmap

Language：Others   Publishing type：Research paper (other academic)   Publisher：Proceedings of the International Symposium on Automation and Robotics in Construction  

Soil compaction is one of the most important basic elements in construction work because it directly affects the quality of structures. Compaction work using vibratory rollers is generally applied to strengthen ground stiffness, and the method that focuses on the number of compaction cycles is widely used to manage the ground stiffness by vibratory rollers. In contrast to this method, the continuous compaction control (CCC) using accelerometers installed on the vibratory rollers has been proposed as a quantitative evaluation method more suited to actual ground conditions. This method quantifies the distortion rate of the acceleration waveform of the vibratory roller. However, this method based on acceleration response has problems in measurement discrimination accuracy and sensor durability because the accelerometer is installed on the vibration roller, which is the source of vibration. In this paper, we propose a new ground stiffness evaluation method using multiple accelerometers installed on the ground surface. The proposed method measures the acceleration response during compaction work by vibratory rollers using multiple accelerometers on the ground surface. Experiments show the proposed method has a higher discrimination than the conventional methods.

DOI： 10.22260/ISARC2023/0037

Scopus

researchmap

Language：English   Publishing type：Research paper (scientific journal)   Publisher：PLoS ONE  

It is important for caregivers of people with dementia (PwD) to have good patient communication skills as it has been known to reduce the behavioral and psychological symptoms of dementia (BPSD) of PwD as well as caregiver burnout. However, acquiring such skills often requires one-on-one affective training, which can be costly. In this study, we propose affective training using augmented reality (AR) for supporting the acquisition of such skills. The system uses see-through AR glasses and a nursing training doll to train the user in both practical nursing skills and affective skills such as eye contact and patient communication. The experiment was conducted with 38 nursing students. The participants were assigned to either the Doll group, which only used a doll for training, or the AR group, which used both a doll and the AR system. The results showed that eye contact significantly increased and the face-to-face distance and angle decreased in the AR group, while the Doll group had no significant difference. In addition, the empathy score of the AR group significantly increased after the training. Upon analyzing the correlation between personality and changes of physical skills, we found a significant positive correlation between the improvement rate of eye contact and extraversion in the AR group. These results demonstrated that affective training using AR is effective for improving caregivers' physical skills and their empathy for their patients. We believe that this system will be beneficial not only for dementia caregivers but for anyone looking to improve their general communication skills.

DOI： 10.1371/journal.pone.0288175

Web of Science

Scopus

PubMed

researchmap

Language：Others   Publishing type：Research paper (other academic)   Publisher：Proceedings of the Annual International Conference of the IEEE Engineering in Medicine and Biology Society, EMBS  

Federated learning (FL) has attracted attention as a technology that allows multiple medical institutions to collaborate on AI without disclosing each other's patient data. However, FL has the challenge of being unable to robustly learn when the data of participating clients is non-independently and non-identically distributed (Non-IID). Personalized Federated Learning (PFL), which constructs a personalized model for each client, has been proposed as a solution to this problem. However, conventional PFL methods do not ensure the interpretability of personalization, specifically, the identification of which data samples are contributed to each personalized learning, which is important for AI in medical applications. In this study, we propose a novel PFL framework, Federated Adjustment of Covariate (FedCov), which acquires a propensity score model representing the covariate shift among clients through prior FL, then learns a final model by weighting the contribution of each training sample to PFL based on the estimated propensity score. This approach enables both the learning of personalized models through covariate adjustment and the visualization of the contribution of each client to PFL. FedCov was evaluated in the prediction of in-hospital mortality across 50 hospitals in the eICU Collaborative Research Database, achieving an ROC-AUC of 0.750. This result outperformed the AUCs in the 0.720-0.735 range achieved by conventional FL methods and was closest to the AUC of 0.754 achieved by centralized learning.Clinical Relevance - This study demonstrates the feasibility of providing sophisticated and personalized AI-driven clinical decision support to any medical institution through personalized federated learning.

DOI： 10.1109/EMBC40787.2023.10339940

Web of Science

Scopus

PubMed

researchmap

Other Link： https://dblp.uni-trier.de/db/conf/embc/embc2023.html#TarumiSYMK23

Language：Others   Publishing type：Research paper (scientific journal)   Publisher：Robotics and Autonomous Systems  

This study investigates operability and acceptability issues in the teleoperation of robots. Prior studies have proposed efficient approaches to increase human perceptual ability and robot autonomy but with reduced operability and acceptance. We propose a novel teleoperation method that overcomes the weaknesses of existing approaches while inheriting their strengths. The key feature of our method is switching the teleoperated robot world from real to virtual. The user study results showed that the proposed method offered an improved user experience compared to the conventional methods, while task efficiency was equivalent in all methods. The contributions of this paper include the proposal of the teleoperation method by seamless switching between real and virtual space, the proposal of an image transformation method and visual effect to achieve seamless switching, and verification of the practicality of the proposed system through experiments on actual mobile robots.

DOI： 10.1016/j.robot.2023.104405

Web of Science

Scopus

researchmap

researchmap

Language：Others   Publishing type：Research paper (other academic)   Publisher：Proceedings - 2023 IEEE Winter Conference on Applications of Computer Vision, WACV 2023  

3D LiDAR sensors are indispensable for the robust vision of autonomous mobile robots. However, deploying LiDAR-based perception algorithms often fails due to a domain gap from the training environment, such as inconsistent angular resolution and missing properties. Existing studies have tackled the issue by learning inter-domain mapping, while the transferability is constrained by the training configuration and the training is susceptible to peculiar lossy noises called ray-drop. To address the issue, this paper proposes a generative model of LiDAR range images applicable to the data-level domain transfer. Motivated by the fact that LiDAR measurement is based on point-by-point range imaging, we train an implicit image representation-based generative adversarial networks along with a differentiable ray-drop effect. We demonstrate the fidelity and diversity of our model in comparison with the point-based and image-based state-of-the-art generative models. We also showcase upsampling and restoration applications. Furthermore, we introduce a Sim2Real application for LiDAR semantic segmentation. We demonstrate that our method is effective as a realistic ray-drop simulator and outperforms state-of-the-art methods.

DOI： 10.1109/WACV56688.2023.00131

Web of Science

Scopus

researchmap

Other Link： https://dblp.uni-trier.de/db/conf/wacv/wacv2023.html#NakashimaIK23

Language：Others   Publishing type：Research paper (other academic)   Publisher：IEEE  

This paper proposes a novel bolt type force sensor that measures forces applied to an object fixed by the sensor. This sensor consists of a iron bolt and four strain gauges and can be made inexpensively compared to a 6-axis force sensor. Due to the feature of the bolt type sensor, it can measure the force applied to the object without dedicated holds, extra jigs, gaps between holds and walls. Therefore, the sensor can be applied to existing climbing holds without changes of difficulty of climbing problems. In this paper, the structure of the sensor is introduced and its calibration method is discussed. Finally, a force measurement experiments are conducted, and it was confirmed that the sensor is able to achieve highly accurate force estimation in practical situations.

DOI： 10.1109/SII55687.2023.10039421

Web of Science

Scopus

researchmap

Other Link： https://dblp.uni-trier.de/db/conf/sii/sii2023.html#KawamuraNDK23

Language：Others   Publishing type：Research paper (other academic)   Publisher：IEEE  

This paper proposes a method for non-line-of-sight (NLoS) position estimation utilizing wireless distance sensors. Recently, the accuracy of wireless distance sensors that use ultra-wideband (UWB) or ultrasonic technologies to measure the distance between two sensor devices has increased significantly. By placing these sensors in the environment, it is possible to precisely determine the position of mobile robots in indoor environments. Owing to reflections in the environment, these sensors have a large measurement error in NLoS conditions, limiting their applicability to environments that satisfy the line-of-sight (LoS) condition. This study aims to develop a stable method for estimating the position of mobile robots in indoor environments, including NLoS conditions, using wireless distance sensors. Experiments were conducted in two real environments: one with obstacles in front of the beacon and one with dynamic obstacles. In both cases, the combining 2D-LiDAR and wireless distance sensors using the proposed method considering NLOS was more accurate than the method considering LoS only.

DOI： 10.1109/SII55687.2023.10039195

Web of Science

Scopus

researchmap

Other Link： https://dblp.uni-trier.de/db/conf/sii/sii2023.html#ItsukaK23

Language：Others   Publishing type：Research paper (other academic)   Publisher：2023 IEEE/SICE International Symposium on System Integration, SII 2023  

People depend on medical equipment to support their movements when their motor function declines. Our previous study developed a method to estimate motor function from the force applied to a vertical handrail while standing. However, the effect of the handrail direction on movement remains unclear. Additionally, the force applied to the handrail and floor reaction forces on the buttocks and feet may also change with a decline in motor function. Here, this study constructed a system with force plates and handles in both the horizontal and vertical directions to measure the forces applied to the handrails, buttocks, and feet. Furthermore, the change in accuracy of the estimation of motor function, depending on the direction of the handrails and input information, was investigated. In the experiment, healthy participants stood up using a handrail with unrestricted movement and while wearing elderly experience kits that artificially impaired their motor function. The results showed that people exert more downward force on horizontal handrails than on vertical handrails. However, people rely on the vertical handrail for a longer period of time to stabilize anterior-posterior movement. These results indicate that different directions of handrails cause different strategies of the standing-up motion. Additionally, the accuracy of the estimation of motor function improved when the horizontal handrail was used rather than the vertical handrail. This suggests that the classification accuracy could be improved by using different handrail directions, depending on the subject's condition and standing-up motion.

DOI： 10.1109/SII55687.2023.10039452

Web of Science

Scopus

researchmap

Other Link： https://dblp.uni-trier.de/db/conf/sii/sii2023.html#KiharaATINK23

Language：Japanese   Publisher：The Japan Society of Mechanical Engineers  

<p>We have been working on the realization of an autonomous mowing robot to automate weeding work, which is an essential part of agriculture. The robot developed so far can do mowing work in open environments by localization using high-precision GNSS and performing exhaustive sweeping of the work area. This paper reports on the third robot, which is newly equipped with a Visual SLAM camera, and a path following algorithm is implemented to assume use in environments that are difficult to benefit from GNSS.</p>

DOI： 10.1299/jsmermd.2023.2p1-b03

CiNii Research

Language：Japanese   Publisher：The Japan Society of Mechanical Engineers  

<p>In facility construction of a building, illuminance measurement is required to verify that the illuminance condition satisfies JIS regulations before its completion. In order to perform illuminance measurement accurately, measurements are conducted at night, causing long working hours at night. In this paper, we propose an autonomous illuminance measurement robot system consisting of multiple robots. The experiments show that the measurement errors of illuminance compared to the human measurements is 1.36% and the measurement time is reduced about 12%.</p>

DOI： 10.1299/jsmermd.2023.1a1-b03

CiNii Research

Language：Japanese   Publisher：The Japan Society of Mechanical Engineers  

<p>The civil engineering and construction industry has seen an increase in occupational accidents in recent years. Although several systems to improve the efficiency of earthwork operations have been developed, they have not directly involved in improving safety. The objective of this study is to explore the use of a cyber-physical system (CPS) to improve both the efficiency and safety of earthwork operations. The CPS platform should have a function to store and manage environmental information in a database. The ROS2-TMS, which is the CPS platform for service robots, satisfies this requirement. However, the ROS2-TMS cannot handle environmental information specific to earthwork sites, such as terrain and ground data. Therefore, in this research, we developed an improved version of ROS2-TMS named ROS2-TMS for Construction that can store and manage site-specific environmental information in a database. Additionally, we developed a VR interface to visualize the stored environmental information.</p>

DOI： 10.1299/jsmermd.2023.1p1-b03

CiNii Research

Language：Japanese   Publisher：The Japan Society of Mechanical Engineers  

<p>In this study, we developed an outdoor surveillance robot system using public 5G. The robot patrolling outdoors is connected to a remote observer via 5G and ROS2, and the observer control the robot while monitoring camera images remotely. We tested three network configurations to realize remote communication. Experiments for remote autonomous surveillance were conducted.</p>

DOI： 10.1299/jsmermd.2023.1a1-i02

CiNii Research

Language：Japanese   Publisher：The Japan Society of Mechanical Engineers  

<p>Illusory control (IC) that is teleoperation method we proposed was based on the premise that the virtual environment is prepared in advance. Therefore, the mobile robot using IC can function only in known environment. This paper presents ”Instant illusory control” that no need to prepare the virtual environment in advance. The proposed robot system can instantly create a virtual environment from real images of the robot in motion using Instant NGP as one of methods in Neural Radiance Fields. Additionally, using 360-degree images, the environment around the robot can be comprehensively virtualized without demanding on the camera poses. Furthermore, using the depth estimation result of the images in advance for the optimization of the density of neural radiance fields, the reconstruction accuracy at unknown poses is ensured as well. Using this virtual environment instantly created, teleoperation by transition between real and virtual environments is expected to function in unknown environment. The experimental result shows that the proposed system can achieve a consistent back and forth between virtual and real spaces in mobile robot teleoperation.</p>

DOI： 10.1299/jsmermd.2023.2a2-i02

CiNii Research

Language：Japanese   Publisher：The Japan Society of Mechanical Engineers  

<p>We propose a novel navigation method applying an offline reinforcement learning method based on Implicit Policy Constraint to mobile robot navigation in environments with pedestrians. The proposed method utilizes a flow-based generative model for the behavior policy, and the latent policy is trained using a method based on Advantage-Weighted Regression. The proposed method is evaluated in a simulation environment.</p>

DOI： 10.1299/jsmermd.2023.2a2-g01

CiNii Research

Language：Japanese   Publisher：The Japan Society of Mechanical Engineers  

<p>This paper presents a method to improve the accuracy of human pose estimation using RGBD images. 3D pose estimation based on mapping depth images to RGB images has a problem. It is the error in the depth direction caused by the low accuracy of the depth image and the fact that the estimated points are on the surface of the body. This paper proposes a method to improve the accuracy by using a correction algorithm based on ICP. The method consists of a measurement system that measures RGBD images and estimates human posture, and a correction algorithm using ICP.</p>

DOI： 10.1299/jsmermd.2023.2a2-h01

CiNii Research

Publishing type：Research paper (international conference proceedings)   Publisher：Conference Proceedings - IEEE International Conference on Systems, Man and Cybernetics  

We proposed a teleoperation method, illusory control (IC), that provides a comfortable operation experience using a seamless transition between real and pre-prepared virtual environments. Therefore, the mobile robot with IC could function solely in familiar environments. To make IC applicable in unfamiliar environments, this study proposes a novel method, instant IC, that eliminates the requirement for a pre-prepared virtual environment. The proposed robot system can instantly generate a virtual environment using actual 360° images of the robot in motion, utilizing instant neural graphics primitives and neural radiance fields. The 360° images allow the entire surrounding environment to be virtualized without requiring specific camera orientations. In addition, by optimizing the density of neural radiance fields using depth estimation results beforehand, the reconstruction accuracy at unknown poses can be guaranteed. Furthermore, we propose a depth scaling method based on the actual measurements obtained by LiDAR to increase the consistency of virtual and real environments. With this instant virtual environment, the proposed system enables teleoperation in unknown environments via the seamless transition between real and virtual environments. The experimental results exhibit consistent and smooth back-and-forth transitions between virtual and real space in mobile robot teleoperation.

DOI： 10.1109/SMC53992.2023.10394235

Scopus

researchmap

Language：English   Publishing type：Research paper (scientific journal)  

DOI： 10.1002/alz.062055

Language：Others   Publishing type：Research paper (scientific journal)  

Abstract

Background

While communication with dementia patients is challenging, known educational methods to improve communication skill for medical professionals are lacking. Our study aimed to assess the efficacy of simulated communication skills training for nursing students using augmented reality (AR) with real‐time feedback.

Methods

This is a randomized control study. Twenty‐five nursing students enrolled and learned standardized multimodal comprehensive care communication skills through a self‐learning material, which includes pathophysiology and clinical manifestations of dementia. Subsequently, participants were randomly assigned to one of the two learning systems—AR training or conventional nursing mannequin training. Each group had the training intervention to change the clothes of mannequins for one hour. The mannequin of the AR group was a superimposed computer graphic of an elderly woman’s face which reacted to participants' communication. Further, the communication skills of gaze and voice were evaluated by artificial intelligence (AI) and feedbacked to the participants’ head‐mounted display in real‐time. The conventional training group had the self‐training with nursing mannequins. All participants performed basic nursing care, including changing clothes and bed bath, to simulated patients before and after the training intervention, which were video‐recorded by an eye‐tracking camera and a fixed camera, then the videos were analyzed the communication skills by AI. Additionally, participants' empathy to patients was evaluated by the Jefferson Scale of Empathy‐Health Professions Students Version (JSPE‐HSP). The primary outcome was the proportion of time spent in eye contact during the care to simulated patients. The secondary outcome was the empathy score of JSPE‐HSP.

Results

After the training intervention, the proportion of time spent in eye contact in the AR training group significantly increased than the conventional training group (eye contact 13.6% versus 4.4%, P<0.05). Moreover, the JSPE‐HSP score increased from pre‐training to post‐training in the AR training group, whereas it decreased in the conventional training group [Mean (SD): 9.1 (6.6) versus ‐1.3 (3.8), P<0.01].

Conclusions

The simulated communication skills training for nursing students using augmented reality with real‐time feedback was associated with increased interactive communication skills to simulated patients and the empathy to patients.

DOI： 10.1002/alz.062055

Language：English   Publishing type：Research paper (other academic)  

Web of Science

Language：Others   Publishing type：Research paper (other academic)   Publisher：Conference Proceedings - IEEE International Conference on Systems, Man and Cybernetics  

Assisting patients with dementia is a significant social issue. Currently, to assist patients with dementia, a multimodal care technique called Humanitude is gaining popularity. In Humanitude, the patients are assisted through various techniques to stand up independently by utilizing their motor functions as much as possible. Humanitude care techniques encourage caregivers to increase the area of contact with patients during the sit-to-stand motion. However, Humanitude care techniques are not accurately performed by novice caregivers. Therefore, in this study, a smock-type wearable sensor was developed to measure the proximity between caregivers and care recipients during sit-to-stand motion assistance. A measurement experiment was conducted to evaluate the proximity differences between Humanitude care and simulated novice care. In addition, the effects of different care techniques on the center of mass (CoM) trajectory and muscle activity of the care recipients were investigated. The results showed that the caregivers tend to bring their top and middle trunk closer in Humanitude care compared with novice simulated care. Furthermore, it was observed that the CoM trajectory and muscle activity under Humanitude care were similar to those observed when the care recipient stands up independently. These results validate the effectiveness of Humanitude care and provide useful information for teaching techniques in Humanitude.

DOI： 10.1109/SMC53654.2022.9945156

Scopus

researchmap

Other Link： https://dblp.uni-trier.de/db/conf/smc/smc2022.html#AnTNSSK22

Language：Others   Publishing type：Research paper (scientific journal)   Publisher：Informa UK Limited  

This paper presents a comparative study of two AR training systems for Humanitude dementia care. Humanitude is a multimodal comprehensive care methodology for patients with dementia and has attracted considerable attention as a gentle and effective care technique. To provide an effective training system for Humanitude, we developed HEARTS 1, which realizes simultaneous sensing and interaction by combining a real training entity and augmented reality technology. However, training experiments using HEARTS 1 indicated that HEARTS 1 had certain weaknesses that should be addressed, namely, the usability of the AR display or the impression of the CG model. This paper presents a new prototype called HEARTS 2 consisting of Microsoft HoloLens 2 as well as realistic and animated CG models of older women. Psychological experiments were conducted to identify the difference in the usability of the AR displays, such as the weight and the field of view, and the impression of the CG models. We also conducted questionnaire surveys for beginners and medical professionals and preliminary training experiments to evaluate the usefulness of HEARTS 2. Based on the psychological experiments and questionnaire surveys, we concluded that HEARTS 2 was more comfortable and suitable for a training system for dementia care.

DOI： 10.1080/01691864.2021.2017342

Web of Science

Scopus

researchmap

Language：Others   Publishing type：Research paper (scientific journal)   Publisher：Springer Science and Business Media {LLC}  

This study proposes a new software platform, called ROS2-TMS, for an informationally structured environment. An informationally structured environment is vital for developing intelligent service robots by embedding various sensors in the environment to enhance the sensing capability and intelligence of robots. Thus far, we have been developing a software platform, named ROS-TMS, for an informationally structured environment, which connects various sensors and robots using ROS architecture. In recent years, ROS2, a next-generation version of ROS, has been released. ROS2 has many advantages, such as enhanced security, QoS control, and support for various platforms. ROS2-TMS, a new version of ROS-TMS, is developed not only by porting existing modules in ROS-TMS, such as the control system for a communication robot, but also by adding useful functions utilizing new features in ROS2. For instance, we added a voice user interface to control robots and various devices in the environment, such as lights or a bed. In addition, we implemented a new task scheduler that provides a cancelation function to stop running tasks and improve the security of the platform.

DOI： 10.1186/s40648-021-00216-2

Web of Science

Scopus

J-GLOBAL

researchmap

Language：Others   Publishing type：Research paper (other academic)  

DOI： 10.1109/SII52469.2022.9708775

Language：Others   Publishing type：Research paper (other academic)  

DOI： 10.1109/SII52469.2022.9708776

Language：Others   Publishing type：Research paper (other academic)  

DOI： 10.1109/SII52469.2022.9708891

Language：Others   Publishing type：Research paper (other academic)  

DOI： 10.1109/SII52469.2022.9708899

Language：Others   Publishing type：Research paper (scientific journal)  

DOI： 10.48550/arXiv.2210.11750

researchmap

Language：Japanese   Publisher：The Society of Instrument and Control Engineers  

Preface to Special Issue on the 21st SICE System Integration Division Annual Conference

DOI： 10.9746/sicetr.58.1

CiNii Research

Language：English   Publishing type：Research paper (international conference proceedings)  

Language：English   Publishing type：Research paper (international conference proceedings)  

DOI： 10.1109/SII52469.2022.9708775

Language：English   Publishing type：Research paper (international conference proceedings)  

DOI： 10.1109/SII52469.2022.9708776

Language：English   Publishing type：Research paper (international conference proceedings)  

DOI： 10.1109/SII52469.2022.9708899

Language：English   Publishing type：Research paper (international conference proceedings)  

DOI： 10.1109/SII52469.2022.9708891

Language：English   Publishing type：Research paper (scientific journal)   Publisher：Journal of Endourology  

BACKGROUND: Early intravesical recurrence after transurethral resection of bladder tumors (TURBT) is often caused by overlooking of tumors during TURBT. Although narrow-band imaging and photodynamic diagnosis were developed to detect more tumors than conventional white-light imaging, the accuracy of these systems has been subjective, along with poor reproducibility due to their dependence on the physician's experience and skills. To create an objective and reproducible diagnosing system, we aimed to assess the utility of artificial intelligence (AI) with Dilated U-Net to reduce the risk of overlooked bladder tumors when compared with the conventional AI system, termed U-Net. MATERIAL AND METHODS: We retrospectively obtained cystoscopic images by converting videos obtained from 120 patients who underwent TURBT into 1,790 cystoscopic images. The Dilated U-Net, which is an extension of the conventional U-Net, analyzed these image datasets. The diagnostic accuracy of the Dilated U-Net and conventional U-Net were compared using the following four measurements: pixel-wise sensitivity (PWSe); pixel-wise specificity (PWSp); pixel-wise positive predictive value (PWPPV), representing the AI diagnostic accuracy per pixel; and dice similarity coefficient (DSC), representing the overlap area between the bladder tumors in the ground truth images and segmentation maps. RESULTS: The cystoscopic images were divided as follows, according to the pathological T-stage: 944, Ta; 412, T1; 329, T2; and 116, carcinoma in-situ. The PWSe, PWSp, PWPPV, and DSC of the Dilated U-Net were 84.9%, 88.5%, 86.7%, and 83.0%, respectively, which had improved when compared to that with the conventional U-Net by 1.7%, 1.3%, 2.1%, and 2.3%, respectively. The DSC values were high for elevated lesions and low for flat lesions for both Dilated and conventional U-Net. CONCLUSIONS: Dilated U-Net, with higher DSC values than conventional U-Net, might reduce the risk of overlooking bladder tumors during cystoscopy and TURBT.

DOI： 10.1089/end.2021.0483

Web of Science

Scopus

PubMed

researchmap

Language：English  

researchmap

Language：Japanese   Publisher：The Japan Society of Mechanical Engineers  

<p>In this study, we develop a sensor terminal with multiple and various sensors named sensor pod, which collects various environmental information at a construction site. The sensor pod is equipped with a 3D-LiDAR and a vibration sensor, which can be used to predict the surrounding hazards and evaluate the ground stiffness. In this paper, we introduce a method of evaluating ground stiffness using the waveform distortion of multi-point synchronized vibration data obtained by the distributed sensor pods.</p>

DOI： 10.1299/jsmermd.2022.1a1-e04

CiNii Research

Language：Japanese   Publisher：The Japan Society of Mechanical Engineers  

<p>This paper presents a novel teleoperation method called “Illusory control” to address the problem of reduced acceptance due to the disagreement between human and robot intentions in shared control. We have reported the simulation results of the teleoperation behavior using Illusory control. This paper describes the improvements of the proposed system and shows the results of experiments using an actual robot and realistic virtual images by CycleGAN.</p>

DOI： 10.1299/jsmermd.2022.2p2-g06

CiNii Research

Language：Japanese   Publisher：The Japan Society of Mechanical Engineers  

<p>This paper presents a retrofit backhoe remote control system that is inexpensive, compact, and easy to install. The system consists of a remote control system using a teleoperation system embedded by a construction machinery manufacturer and a small robot arm, and a remote sensing system using a multi-core microcomputer.</p>

DOI： 10.1299/jsmermd.2022.1p1-c07

CiNii Research

Language：Others   Publishing type：Research paper (scientific journal)  

DOI： 10.1109/TNSRE.2021.3127504

Language：English   Publishing type：Research paper (scientific journal)  

Many patients suffer from declined motor abilities after a brain injury. To provide appropriate rehabilitation programs and encourage motor-impaired patients to participate further in rehabilitation, sufficient and easy evaluation methodologies are necessary. This study is focused on the sit-to-stand motion of post-stroke patients because it is an important daily activity. Our previous study utilized muscle synergies (synchronized muscle activation) to classify the degree of motor impairment in patients and proposed appropriate rehabilitation methodologies. However, in our previous study, the patient was required to attach electromyography sensors to his/her body; thus, it was difficult to evaluate motor ability in daily circumstances. Here, we developed a handrail-type sensor that can measure the force applied to it. Using temporal features of the force data, the relationship between the degree of motor impairment and temporal features was clarified, and a classification model was developed using a random forest model to determine the degree of motor impairment in hemiplegic patients. The results show that hemiplegic patients with severe motor impairments tend to apply greater force to the handrail and use the handrail for a longer period. It was also determined that patients with severe motor impairments did not move forward while standing up, but relied more on the handrail to pull their upper body upward as compared to patients with moderate impairments. Furthermore, based on the developed classification model, patients were successfully classified as having severe or moderate impairments. The developed classification model can also detect long-term patient recovery. The handrail-type sensor does not require additional sensors on the patient's body and provides an easy evaluation methodology.

DOI： 10.1109/TNSRE.2021.3127504

Language：Others   Publishing type：Research paper (scientific journal)  

Language：Others   Publishing type：Research paper (other academic)  

DOI： 10.1109/IROS51168.2021.9636747

Language：English   Publishing type：Research paper (scientific journal)  

This paper introduces an enhanced MSM (Mutual Subspace Method) methodology for gait recognition, to provide robustness to variations in walking speed. The enhanced MSM (eMSM) methodology expands and adapts the MSM, commonly used for face recognition, which is a static/physiological biometric, to gait recognition, which is a dynamic/behavioral biometrics. To address the loss of accuracy during calculation of the covariance matrix in the PCA step of MSM, we use a 2D PCA-based mutual subspace. Furhtermore, to enhance the discrimination capability, we rotate images over a number of angles, which enables us to extract richer gait features to then be fused by a boosting method. The eMSM methodology is evaluated on existing data sets which provide variable walking speed, i.e. CASIA-C and OU-ISIR gait databases, and it is shown to outperform state-of-the art methods. While the enhancement to MSM discussed in this paper uses combinations of 2D-PCA, rotation, boosting, other combinations of operations may also be advantageous.

DOI： 10.1371/journal.pone.0255927

Language：English   Publishing type：Research paper (international conference proceedings)  

DOI： 10.1109/RO-MAN50785.2021.9515457

Language：Others   Publishing type：Research paper (scientific journal)  

Hyper-adaptability is an ability of humans and animals to adapt to large-scale changes in the nervous system or the musculoskeletal system, such as strokes and spinal cord injuries. Although this adaptation may involve similar neural processes with normal adaptation to usual environmental and body changes in daily lives, it can be fundamentally different because it requires ‘construction’ of the neural structure itself and ‘reconstitution’ of sensorimotor control rules to compensate for the changes in the nervous system. In this survey paper, we aimed to provide an overview on how the brain structure changes after brain injury and recovers through rehabilitation. Next, we demonstrated the recent approaches used to apply computational and neural network modeling to recapitulate motor control and motor learning processes. Finally, we discussed future directions to bridge the gap between conventional physiological and modeling approaches to understand the neural and computational mechanisms of hyper-adaptability and its applications to clinical rehabilitation.

DOI： 10.1080/01691864.2021.1943710

Language：Others   Publishing type：Research paper (scientific journal)  

Autonomous robots are expected to replace dangerous, dirty, and demanding (3D) jobs. At a theme park, surveillance, cleaning, and guiding tasks can be regarded as 3D jobs. The present paper attempts to develop an autonomous tour guide robot system and co-experience system at a large theme park. For realizing such autonomous service robots used in our daily environment, localization is one of the most important and fundamental functions. A number of localization techniques, including simultaneous localization and mapping, have been proposed. Although a global navigation satellite system (GNSS) is most commonly used in outdoor environments, its accuracy is approximately 10 m, which is inadequate for navigation of an autonomous service robot. Therefore, a GNSS is usually used together with other localization techniques, such as map matching or camera-based localization. In the present study, we adopt the quasi-zenith satellite system (QZSS), which became available in and around Japan in November 2018, for the localization of an autonomous service robot. The QZSS provides high-accuracy position information using quasi-zenith satellites (QZSs) and has a localization error of less than 10 centimeters. In the present paper, we compare the positioning performance of the QZSS and the real-time kinematic GPS and verify the stability and the accuracy of the QZSS in an outdoor environment. In addition, we introduce a tour guide robot system using the QZSS and present the results of a guided tour experiment in a theme park. On the other hand, based on the tour guide system, we also develop a co-experience robot system in a theme park, which realizes the sharing of experiences using an immersive VR display and the 5th-generation mobile communication system (5G). The robot is equipped with a 360-degree video camera and real-time 4K video is transmitted to a remote operator using the large communication capacity of the 5G network. The experimental results at a theme park showed that the guided tour experiment was successful and that the co-experience system allowed sharing of the experience with high immersion by a remote operator.

DOI： 10.1186/s40648-021-00192-7

Language：Others   Publishing type：Research paper (other academic)  

This paper presents a new system for predicting the spatial change of Nasopharyngeal carcinoma(NPC) and organ-at-risks (OARs) volumes over the course of the radiation therapy (RT) treatment for facilitating the workflow of adaptive radiotherapy. The proposed system, called” Tumor Evolution Prediction (TEP-Net)”, predicts the spatial distributions of NPC and 5 OARs, separately, in response to RT in the coming week, week n. Here, TEP-Net has (n-1)-inputs that are week 1 to week n-1 of CT axial, coronal or sagittal images acquired once the patient complete the planned RT treatment of the corresponding week. As a result, three predicted results of each target region are obtained from the three-view CT images. To determine the final prediction of NPC and 5 OARs, two integration methods, weighted fully connected layers and weighted voting methods, are introduced. From the experiments using weekly CT images of 140 NPC patients, our proposed system achieves the best performance for predicting NPC and OARs compared with conventional methods.

DOI： 10.1109/ICPR48806.2021.9412924

Language：Others   Publishing type：Research paper (scientific journal)  

Automatic analysis of our daily lives and activities through a first-person lifelog camera provides us with opportunities to improve our life rhythms or to support our limited visual memories. Notably, to express the visual experiences, the task of generating captions from first-person lifelog images has been actively studied in recent years. First-person images involve scenes approximating what users actually see; therein, the visual cues are not enough to express the user’s context since the images are limited by his/her intention. Our challenge is to generate lifelog captions using a meta-perspective called “fourth-person vision”. The “fourth-person vision” is a novel concept which complementary exploits the visual information from the first-, second-, and third-person perspectives. First, we assume human–robot symbiotic scenarios that provide a second-person perspective from the camera mounted on the robot and a third-person perspective from the camera fixed in the symbiotic room. To validate our approach in this scenario, we collect perspective-aware lifelog videos and corresponding caption annotations. Subsequently, we propose a multi-perspective image captioning model composed of an image-wise salient region encoder, an attention module that adaptively fuses the salient regions, and a caption decoder that generates scene descriptions. We demonstrate that our proposed model based on the fourth-person concept can greatly improve the captioning performance against single- and double-perspective models.

DOI： 10.1186/s40648-020-00181-2

Language：Others   Publishing type：Research paper (scientific journal)  

DOI： 10.1016/s0167-8140(21)01686-8

Language：English   Publishing type：Research paper (international conference proceedings)  

Language：English   Publishing type：Research paper (scientific journal)  

The present paper introduces a near-future perception system called Previewed Reality. In a co-existence environment of a human and a robot, unexpected collisions between the human and the robot must be avoided to the extent possible. In many cases, the robot is controlled carefully so as not to collide with a human. However, it is almost impossible to perfectly predict human behavior in advance. On the other hand, if a user can determine the motion of a robot in advance, he/she can avoid a hazardous situation and exist safely with the robot. In order to ensure that a user perceives future events naturally, we developed a near-future perception system named Previewed Reality. Previewed Reality consists of an informationally structured environment, a VR display or an AR display, and a dynamics simulator. A number of sensors are embedded in an informationally structured environment, and information such as the position of furniture, objects, humans, and robots, is sensed and stored structurally in a database. Therefore, we can forecast possible subsequent events using a robot motion planner and a dynamics simulator and can synthesize virtual images from the viewpoint of the user, which will actually occur in the near future. The viewpoint of the user, which is the position and orientation of a VR display or an AR display, is also tracked by an optical tracking system in the informationally structured environment, or the SLAM technique on an AR display. The synthesized images are presented to the user by overlaying these images on a real scene using the VR display or the AR display. This system provides human-friendly communication between a human and a robotic system, and a human and a robot can coexist safely by intuitively showing the human possible hazardous situations in advance.

DOI： 10.1080/01691864.2020.1829041

Language：English   Publishing type：Research paper (scientific journal)  

Terrain classification is critically important for Mars rovers, which rely on it for planning and autonomous navigation. On-board terrain classification using visual information has limitations, and is sensitive to illumination conditions. Classification can be improved if one fuses visual imagery with additional infrared (IR) imagery of the scene, yet unfortunately there are no IR image sensors on the current Mars rovers. A virtual IR sensor, estimating IR from RGB imagery using deep learning, was proposed in the context of a MU-Net architecture. However, virtual IR estimation was limited by the fact that slope angle variations induce temperature differences within the same terrain. This paper removes this limitation, giving good IR estimates and as a consequence improving terrain classification by including the additional angle from the surface normal to the Sun and the measurement of solar radiation. The estimates are also useful when estimating thermal inertia, which can enhance slip prediction and small rock density estimation. Our approach is demonstrated in two applications. We collected a new data set to verify the effectiveness of the proposed approach and show its benefit by applying to the two applications.

DOI： 10.1109/LRA.2020.3013909

Language：Others   Publishing type：Research paper (other academic)  

This study develops a training system for a multimodal comprehensive care methodology for dementia patients called Humanitude. Humanitude has attracted much attention as a gentle and effective care technique. It consists of four main techniques, namely, eye contact, verbal communication, touch, and standing up, and more than 150 care elements. Learning Humanitude thus requires much time. To provide an effective training system for Humanitude, we develop a training system that realizes sensing and interaction simultaneously by combining a real entity and augmented reality technology. To imitate the interaction between a patient and a caregiver, we superimpose a three-dimensional CG model of a patient's face onto the head of a soft doll using augmented reality technology. Touch information such as position and force is sensed using the whole body wearable tactile sensor developed to quantify touch skills. This training system enables the evaluation of eye contact and touch skills simultaneously. We build a prototype of the proposed training system and evaluate the usefulness of the system in public lectures.

DOI： 10.1109/IROS45743.2020.9341039

Language：Others   Publishing type：Research paper (other academic)  

In this paper, we propose a method for predicting dose distribution images of patients with Nasopharyngeal carcinoma (NPC) from contoured computer tomography (CT) images. The proposed system is based on our previous method [1]. The first phase is to obtain the feature maps of 2D dose images of each beam from contoured CT images of a patient by convolutional deep neural network model. In the second phase, dose distribution images are predicted from the obtained feature maps by the integration network. Our modified system predicted dose distribution images accurately. From the experimental results using 80 NPC patients' images, the average number of pixels that satisfy the dose constraints of tumors and OARs regions is 81.9 &#37; and 86.1 &#37;, respectively. The proposed system had a global 3D gamma passing rates varying from 82.1 &#37; to 97.2 &#37; for all regions and an overall mean absolute errors (MAEs) was 1.0 ±1.2. From the obtained results, our modified system is superior to the results obtained in our previous system results and conventional methods. Contribution-The use of the predicted 7-beam weights, as input, into our CNN network leads to improve the predicted dose distribution. Contribution-The use of the predicted 7-beam weights, as input, into our CNN network leads to improve the predicted dose distribution.

DOI： 10.1109/ICIEVicIVPR48672.2020.9306610

Language：English   Publishing type：Research paper (scientific journal)  

Gait recognition is one measure of biometrics, which also includes facial, fingerprint, and retina recognition. Although most biometric methods require direct contact between a device and a subject, gait recognition has unique characteristics whereby interaction with the subjects is not required and can be performed from a distance. Cameras are commonly used for gait recognition, and a number of researchers have used depth information obtained using an RGB-D camera, such as the Microsoft Kinect. Although depth-based gait recognition has advantages, such as robustness against light conditions or appearance variations, there are also limitations. For instance, the RGB-D camera cannot be used outdoors and the measurement distance is limited to approximately 10 meters. The present paper describes a long short-term memory-based method for gait recognition using a real-time multi-line LiDAR. Very few studies have dealt with LiDAR-based gait recognition, and the present study is the first attempt that combines LiDAR data and long short-term memory for gait recognition and focuses on dealing with different appearances. We collect the first gait recognition dataset that consists of time-series range data for 30 people with clothing variations and show the effectiveness of the proposed approach.

DOI： 10.1080/01691864.2020.1793812

Language：English   Publishing type：Research paper (scientific journal)  

In this paper, we describe a wearable first-person video (FPV) analysis system for evaluating the skill levels of caregivers. This is a part of our project that aims to quantize and analyze the tender-care technique known as Humanitude by using wearable sensing and AI technology devices. Using our system, caregivers can evaluate and elevate their care levels by themselves. From the FPVs of care sessions taken by wearable cameras worn by caregivers, we obtained the 3D facial distance, pose and eye-contact states between caregivers and receivers by using facial landmark detection and deep neural network (DNN)-based eye contact detection. We applied statistical analysis to these features and developed algorithms that provide scores for tender-care skill. In experiments, we first evaluated the performance of our DNN-based eye contact detection by using eye contact datasets prepared from YouTube videos and FPVs that assume conversational scenes. We then performed skill evaluations by using Humanitude training scenes involving three novice caregivers, two Humanitude experts and seven middle-level students. The results showed that our eye contact detection outperformed existing methods and that our skill evaluations can estimate the care skill levels.

DOI： 10.1007/s10846-019-01052-8

Language：English   Publishing type：Research paper (scientific journal)  

DOI： 10.1007/s10846-020-01148-6

Language：English   Publishing type：Research paper (other academic)  

Sport climbing is a form of rock climbing and has currently increased in popularity. This kind of climbing is played all over the world and becomes an Olympic sport. Motion measurement and analysis system have been required to improve the performance of climbers and instruct them properly. Several systems using motion capture have been proposed to analyze the movement of climbers so far. However, the conventional systems are not sufficient to describe climbing motions since the forces on holds have not been taken into account. This paper proposes a new 3D motion and force measurement system for sport climbing that can obtain the information on human motion and forces on holds simultaneously. The motion and force data are synchronized. Then, we can measure and analyze climbing motions geometrically and dynamically by using the proposed system. The performance of the proposed system is qualitatively verified by experiments.

DOI： 10.1109/SII46433.2020.9026213

Language：English   Publishing type：Research paper (other academic)  

For autonomous service robots used in our daily environment, such as a personal mobility vehicles or delivery robots, localization is one of the most important and fundamental functions. A number of localization techniques, including simultaneous localization and mapping, have been proposed. Although a Global Navigation Satellite System (GNSS) is most commonly used in outdoor environments, its accuracy is around 10 meters and so is inadequate for navigation of an autonomous service robot. Therefore, a GNSS is usually used together with other localization techniques, such as map matching or camera-based localization. In the present study, we adopt the Quasi-Zenith Satellite System (QZSS), which became available in and around Japan on November 2018, for the localization of an autonomous service robot. The QZSS provides high-accuracy position information using electronic reference points and four quasi-zenith satellites, and has a localization error of less than 10 centimeters. In the present paper, we compare the positioning performance of the QZSS and real-time kinematic GPS, and verify the stability and the accuracy of the QZSS in an outdoor environment. In addition, we introduce a tour guide robot system using the QZSS and present the results of a guided tour experiment in a theme park.

DOI： 10.1109/SII46433.2020.9025964

Language：Others   Publishing type：Research paper (scientific journal)  

DOI： 10.1186/s40648-019-0134-1

Language：English   Publishing type：Research paper (scientific journal)  

Spatial change detection is a fundamental technique for finding the differences between two or more pieces of geometrical information. This technique is critical in some robotic applications, such as search and rescue, security, and surveillance. In these applications, it is desirable to find the differences quickly and robustly. The present paper proposes a fast and robust spatial change detection technique for a mobile robot using an on-board range sensors and a highly precise 3D map created by a 3D laser scanner. This technique first converts point clouds in a map and measured data to grid data (ND voxels) using normal distributions transform. The voxels in the map and the measured data are then compared according to the features of the ND voxels. Three techniques are introduced to make the proposed system robust for noise, that is, classification of point distribution, overlapping of voxels, and voting using consecutive sensing. The present paper shows the results of indoor and outdoor experiments using an RGB-D camera and an omni-directional laser scanner mounted on a mobile robot to confirm the performance of the proposed technique.

DOI： 10.1186/s40648-019-0148-8

Language：Others   Publishing type：Research paper (other academic)  

This paper presents a method for synthesizing multi-focus cell images by using generative adversarial networks (GANs). The proposed method, called multi-focus image GAN (MI-GAN), consists of two generators. A base image generator synthesizes a 2D base cell image from random noise. Using the generated base image, a multi-focus cell image generator produces 11 realistic multi-focus images of the cell while considering the relationships between the images acquired at successive focus points. From experimental results, MI-GAN achieves the good performance to generate realistic multi-focus cell images.

DOI： 10.1007/978-3-030-39770-8_8

Language：Others   Publishing type：Research paper (other academic)  

This paper presents a method for reconstructing 3D image from multi-focus microscopic images captured with different focuses. We model the multi-focus imaging by a microscopy and produce the 3D image of a target object based on the model. The 3D image reconstruction is done by minimizing the difference between the observed images and the simulated images generated by the imaging model. Simulation and experimental result shows that the proposed method can generate the 3D image of a transparent object efficiently and reliably.

DOI： 10.1007/978-3-030-39770-8_6

Language：Others   Publishing type：Research paper (scientific journal)  

© Springer Nature Switzerland AG 2019. In Intensity-modulated radiation therapy, the planning of the optimal dose distribution for a patient is a complex and time-consuming process. This paper proposes a new automatic method for predicting of dose distribution of Nasopharyngeal carcinoma (NPC) from contoured computer tomography (CT) images. The proposed method consists of two phases: (1) predicting the 2D optimal dose images of each beam from contoured CT images of a patient by convolutional deep neural network model, called OTNet, and (2) integrating the optimal dose images of all the beams to predict the dose distribution for the patient. From the experiments using CT images of 80 NPC patients, our proposed method achieves a good performance for predicting dose distribution compared with conventional predicted dose distribution methods.

DOI： 10.1007/978-3-030-32486-5_16

Language：English   Publishing type：Research paper (scientific journal)  

In the paper, we propose a new deep learning-based method for segmenting nasopharyngeal carcinoma (NPC) in the nasopharynx from three orthogonal CT images. The proposed method introduces a cascade strategy composed of two-phase manners. In CT images, there are organs, called non-target organs, which NPC never invades. Therefore, the first phase is to detect and eliminate non-target organ regions from the CT images. In the second phase, NPC is extracted from the remained regions in the CT images. Convolutional neural networks (CNNs) are applied to detect non-target organs and NPCs. The proposed system determines the final NPC segmentation by integrating three results obtained from coronal, axial and sagittal images. Moreover, we construct two CNN-based NPC detection systems using one kind of overlapping patches with a fixed size and various overlapping patches with different sizes. From the experiments using CT images of 70 NPC patients, our proposed systems, especially the system using various patches, achieves the best performance for detecting NPC compared with conventional NPC detection methods.

DOI： 10.1016/j.compmedimag.2019.101644

Language：English   Publishing type：Research paper (other academic)  

Terrain imagery collected by satellite remote sensing or by rover on-board sensors is the primary source for terrain classification used in determining terrain traversibility and mission plans for planetary rovers. Mapping models between RGB and IR for terrain classes are learned from real RGB and IR data examples in the same or similar terrain. This paper adds a new class of deep learning architectures called MU-Net (Multiple U-Net) and shows its efficiency in deriving better RGB-to-IR mapping models, improving over past work the estimation of thermal IR images from incoming RGB images and learned RGB-IR mappings.

DOI： 10.1109/CVPRW.2019.00134

Language：Others   Publishing type：Research paper (other academic)  

© 2019 IEEE. Detection of spatial change around a robot is indispensable in several robotic applications, such as search and rescue, security, and surveillance. The present paper proposes a fast spatial change detection technique for a mobile robot using an on-board RGB-D/stereo camera and a highly precise 3D map created by a 3D laser scanner. This technique first converts point clouds in a map and measured data to grid data (ND voxels) using normal distributions transform and classifies the ND voxels into three categories. The voxels in the map and the measured data are then compared according to the category and features of the ND voxels. Overlapping and voting techniques are also introduced in order to detect the spatial changes more robustly. We conducted experiments using a mobile robot equipped with real-time range sensors to confirm the performance of the proposed real-time localization and spatial change detection techniques in indoor and outdoor environments.

DOI： 10.1109/ICRA.2019.8794173

Language：English   Publishing type：Research paper (other academic)  

Detection of spatial change around a robot is indispensable in several robotic applications, such as search and rescue, security, and surveillance. The present paper proposes a fast spatial change detection technique for a mobile robot using an on-board RGB-D/stereo camera and a highly precise 3D map created by a 3D laser scanner. This technique first converts point clouds in a map and measured data to grid data (ND voxels) using normal distributions transform and classifies the ND voxels into three categories. The voxels in the map and the measured data are then compared according to the category and features of the ND voxels. Overlapping and voting techniques are also introduced in order to detect the spatial changes more robustly. We conducted experiments using a mobile robot equipped with real-time range sensors to confirm the performance of the proposed real-time localization and spatial change detection techniques in indoor and outdoor environments.

Language：English   Publishing type：Research paper (other academic)  

© 2019 IEEE. This paper proposes a novel inflatable robotic arm on a mobile platform. To date, inflatable structures have been utilized for robotic arms to provide the workforce in the welfare and nursing care fields. In these fields, inflatable robots, which is lightweight, soft and compactable, is suitable to perform daily tasks near humans and things because of safe contact. In our research group, an inflatable robotic arm consisting of inflatable links, inflatable actuators and a gripper with pleated sheet structures have been developed.In this paper, the robotic arm is utilized in combination with a mobile platform in order to realize practical daily tasks. As an example of the task, a fetch-and-give task is demonstrated. The posture of the robotic arm and the position of the mobile platform is measured by a motion capture system. The whole system is controlled based on the visual information. Finally, it is confirmed that the inflatable robot has sufficient performance for a daily task in living space.

DOI： 10.1109/SII.2019.8700431

Language：English   Publishing type：Research paper (other academic)  

© 2019 IEEE. This study proposes two kinds of systems for building intelligent service robots, namely a sensor terminal named 'Portable' for making various personal mobility robots intelligent and a distributed sensor system for constructing an informationally structured environment consisting of laser range finders and active beacons. The sensor terminal Portable is equipped with a laser range finder and a gyro. Two types of personal mobility robot, namely standing type and wheelchair type, are made intelligent by installing Portable to provide navigation functions such as localization, obstacle detection, and path planning. The sensor system is mainly used to acquire position information about the personal mobility robots, obstacles, and moving objects (e.g., people); this information is used by Portable for navigation. The obtained information is transmitted to the robot, allowing it to operate in a complicated environment. The ability of the proposed system to navigate personal mobility robots is verified in two real environments.

DOI： 10.1109/SII.2019.8700446

Language：English   Publishing type：Research paper (other academic)  

© 2019 IEEE. An inflatable structure has been focused by several researchers in the field of soft robotics, owing to its features: flexible, lightweight and compactable. In particular, inflatable robotic arms have been expected to perform daily tasks near humans due to safe contact. To date, several inflatable robotic arms have been proposed, however, the number of robotic arms composed of only plastic materials is quite limited. In addition, the control performance of the robots has not been discussed sufficiently.This paper proposes a novel inflatable robotic arm composed of poly-laminated sheets and low-density polyethylene (LDPE) sheets. The combination of the two materials provides high-pressure resistance. Several advantages are also given: high torque, high control performance, and high load capacity. Moreover, both pressure and visual feedback controls are utilized to realize precise motion control. The end-effector of the robot is composed of pleated structures and grasps flexibly according to object shapes. The performance of the robot is finally described by some experiments.

DOI： 10.1109/SII.2019.8700329

Language：English   Publishing type：Research paper (scientific journal)  

© The Author(s) 2019. This paper presents several multi-modal 3D datasets for the problem of categorization of places. In this problem. a robotic agent should decide on the type of place/environment where it is located (residential area, forest, etc.) using information gathered by its sensors. In addition to the 3D depth information, the datasets include additional modalities such as RGB or reflectance images. The observations were taken in different indoor and outdoor environments in Fukuoka city, Japan. Outdoor place categories include forests, urban areas, indoor parking, outdoor parking, coastal areas, and residential areas. Indoor place categories include corridors, offices, study rooms, kitchens, laboratories, and toilets. The datasets are available to download at http://robotics.ait.kyushu-u.ac.jp/kyushu_datasets.

DOI： 10.1177/0278364919835603

Language：English   Publishing type：Research paper (other academic)  

© 2019 SPIE. We propose a new method for mapping onto a brain volume model including inner organs with complicated shapes such as lateral ventricles. The proposed method is based on a volumetric Self-organizing Deformable Model (vSDM) which allows to control the mapping positions of inner organs while preserving geometrical features before and after the mapping. The control sometimes causes the self-intersection of the volume model. The solution for the self-intersection in vSDM is to move vertices of the volume model. However, when the inner organ has complicated shape, the vertex movement cannot always correct the self-intersection. To solve this problem, we extend vSDM by introducing a new process of editing the mesh structure of the volume model. Moreover, by applying the proposed method to six brain volume models, a volumetric Statistical Shape Model (SSM) is constructed which represents the shape variations of not only brain surface but also brain inner organs. From experimental results, we confirmed the volumetric SSM has an acceptable performance compared with general surface SSMs generated by organ surface models.

DOI： 10.1117/12.2519819

Language：English   Publishing type：Research paper (other academic)  

© 2019 IEEE. In this paper we propose two novel deep learning-based terrain classification methods robust to illumination changes. The use of cameras is challenged by a variety of factors, of most importance being the changes in illumination. On the other hand, since the temperature of various types of terrains depends on the thermal characteristics of the terrain, the terrain classification can be aided by utilizing the thermal information in addition to visible information. Thus we propose 'TU-Net (Two U-Net)' based on the U-Net and 'TDeepLab (Two DeepLab)' based on DeepLab, which combine visible and thermal images and train the network robust to illumination changes implicitly. To improve the network's learning capability, we expand the proposed methods to the Siamese-based method, which explicitly trains the network to be robust to illumination changes. We also investigate multiple options to fuse the visible and thermal images at at the bottom layer, middle layer, or the top layer of the network. We evaluate the proposed methods with a challenging new dataset consisting of visible and thermal images, which were collected from 10 am till 5 pm (after sunset), and we show the effectiveness of the proposed methods.

DOI： 10.1109/MIPR.2019.00057

Language：English   Publishing type：Research paper (scientific journal)  

© 2018, Springer-Verlag GmbH Germany, part of Springer Nature. We propose a new method for recovering the pelvis of an ancient skeleton from its three component bones with collapsed surfaces. The proposed method uses four types of statistical shape models (SSMs) for the bones. The SSM for each bone describes the mean shape and shape variations of a class of bones. The SSMs for the three component bones are employed to restore the shapes of the component bones. The SSM for the whole pelvis provides the natural anatomical shape of the pelvis and the spatial relationship among the sacrum and the hip bones. Therefore, the three component bones are aligned by using the SSM for the pelvis. The experimental results show that our method achieves reliable reconstruction of the ancient pelvis shape despite having collapsed surfaces in its component bones.

DOI： 10.1007/s00138-018-0972-5

Language：English  

Language：English   Publishing type：Research paper (other academic)  

© 2018 IEEE. We aim to develop a supporting system which enhances the ability of human's short-term visual memory in an intelligent space where the human and a service robot coexist. Particularly, this paper focuses on how we can interpret and record diverse and complex life events on behalf of humans, from a multi-perspective viewpoint. We propose a novel method named 'fourth-person captioning', which generates natural language descriptions by summarizing visual contexts complementarily from three types of cameras corresponding the first-, second-, and third-person viewpoint. We first extend the latest image captioning technique and design a new model to generate a sequence of words given the multiple images. Then we provide an effective training strategy that needs only annotations supervising images from a single viewpoint in a general caption dataset and unsupervised triplet instances in the intelligent space. As the three types of cameras, we select a wearable camera on the human, a robot-mounted camera, and an embedded camera, which can be defined as the first-, second-, and third-person viewpoint, respectively. We hope our work will accelerate a cross-modal interaction bridging the human's egocentric cognition and multi-perspective intelligence.

DOI： 10.1109/SMC.2018.00365

Language：Others   Publishing type：Research paper (scientific journal)  

© 2018, The Author(s). An informationally structured environment (ISE) is a key technology for realizing service robots in daily life in the near future. In ISE, the information of the service robot and its surroundings is structured and provided to the robot on time and on demand. We started the development of a management system for ISE named TMS (Town Management System) in Robot Town Project in 2005. Since then we are continuing our efforts for the improvement of the performance and the enhancement of the functions of TMS. In this paper, we propose the latest system of TMS named ROS-TMS 5.0, which adopts ROS to utilize high scalability and rich resources of ROS. Next, we introduce the hardware platform of ISE called Big Sensor Box which incorporates various sensors under ROS-TMS management and operates service robots based on structured information. Robot service experiments including watching service of a care receiver, voice control of a communication robot and a robotic bed, and ing information by voice are also conducted in Big Sensor Box.

DOI： 10.1186/s40648-018-0123-9

Language：Others  

Real-time distant 3D tracking by 3D laser scanner (Developments of new scan function for RobotEye RE05 and algorithms for real-time 3D tracking)

Language：Japanese  

Computational Tender-Care Science: Computational and Cognitive Neuroscientific Approaches for Understanding the Tender Care

Language：English   Publishing type：Research paper (other academic)  

© 2018 TSI Press. We propose a method that increases the capability of a conventional sensor, transforming it into an enhanced virtual sensor. This paper focuses on a virtual thermal Infrared Radiation (IR) sensor based on a conventional visual (RGB) sensor. The estimation of thermal IR images can enhance the ability of terrain classification, which is crucial for autonomous navigation of rovers. The estimate in IR from visual band has inherent limitations, as these are different bands, yet correlations between visual RGB and thermal IR images exist, as different terrains, which visually may appear different, also have different thermal inertia. This paper describes the developed deep learning-based algorithm that estimates thermal IR images from RGB images of terrains, providing the feasibility of the idea with average 1.21 error [degree Celsius].

DOI： 10.23919/WAC.2018.8430480

Language：English   Publishing type：Research paper (other academic)  

© 2018 TSI Press. This paper presents a new method for generating object mesh models composed of hexahedra. The proposed method is based on a self-organizing deformable model (SDM) which is a deformable surface model guided by competitive learning and an energy minimization approach. Extending the SDM, the proposed method generates a hexahedral mesh model of a target object by fitting a cuboid composed of rectangular voxels to the object. Moreover, the shape of each hexahedron in the model is corrected by dividing the hexahedron into sub-hexahedra and moving the nodes of the hexahedron. From our experimental results, the proposed method obtains the hexahedral mesh model which consists of many regular hexahedra while recovering the shape of the object.

DOI： 10.23919/WAC.2018.8430386

Language：English   Publishing type：Research paper (scientific journal)  

© 2018, © 2018 Taylor & Francis and The Robotics Society of Japan. Semantic place categorization, which is one of the essential tasks for autonomous robots and vehicles, allows them to have capabilities of self-decision and navigation in unfamiliar environments. In particular, outdoor places are more difficult targets than indoor ones due to perceptual variations, such as dynamic illuminance over 24 hours and occlusions by cars and pedestrians. This paper presents a novel method of categorizing outdoor places using convolutional neural networks (CNNs), which take omnidirectional depth/reflectance images obtained by 3D LiDARs as the inputs. First, we construct a large-scale outdoor place dataset named Multi-modal Panoramic 3D Outdoor (MPO) comprising two types of point clouds captured by two different LiDARs. They are labeled with six outdoor place categories: coast, forest, indoor/outdoor parking, residential area, and urban area. Second, we provide CNNs for LiDAR-based outdoor place categorization and evaluate our approach with the MPO dataset. Our results on the MPO dataset outperform traditional approaches and show the effectiveness in which we use both depth and reflectance modalities. To analyze our trained deep networks, we visualize the learned features.

DOI： 10.1080/01691864.2018.1501279

Language：English  

Language：English   Publishing type：Research paper (scientific journal)  

© 2018 Elsevier B.V. Background and Objective: This paper proposes a new method for mapping surface models of human organs onto target surfaces with the same genus as the organs. Methods: In the proposed method, called modified Self-organizing Deformable Model (mSDM), the mapping problem is formulated as the minimization of an objective function which is defined as the weighted linear combination of four energy functions: model fitness, foldover-free, landmark mapping accuracy, and geometrical feature preservation. Further, we extend mSDM to speed up its processes, and call it Fast mSDM. Results: From the mapping results of various organ models with different number of holes, it is observed that Fast mSDM can map the organ models onto their target surfaces efficiently and stably without foldovers while preserving geometrical features. Conclusions: Fast mSDM can map the organ model onto the target surface efficiently and stably, and is applicable to medical applications including Statistical Shape Model.

DOI： 10.1016/j.cmpb.2018.01.028

Language：English   Publishing type：Research paper (other academic)  

© 2017 IEEE. This paper proposes a new control method for the robotic arm with rotational counterweights. This robotic arm is actuated by gravitational and inertia torque of the counterweights. To date, the control law which achieves the angle control of the robot has been proposed. However, in the research, only 1 DOF robot is discussed. Moreover, the stability analysis of the dynamics hasn't been shown. The new control method proposed in this paper achieves the angle control of 2 DOFs, and we can prove the dynamic stability. In this paper, we show the convergence of the dynamics considering the method. Also we conduct numerical simulations for position control of 2 DOFs robotic arm. Verifying the angles of the robotic arm converge to its desired angles, the utility of the method is demonstrated.

DOI： 10.1109/SII.2017.8279288

Language：English   Publishing type：Research paper (other academic)  

© 2017 IEEE. We present a novel method for the outdoor scene categorization using 2D convolutional neural networks (CNNs) which take panoramic depth images obtained by a 3D laser scanner as input. We evaluate our approach in two outdoor scene datasets including six categories: coast, forest, indoor parking, outdoor parking, residential area, and urban area. Our results on both datasets (over 94&#37;) outperform previous approaches and show the effectiveness of this approach for outdoor scene categorization using depth images. To analyze our trained networks we visualize the learned features by using two visualization methods.

DOI： 10.1109/SII.2017.8279243

Language：English   Publishing type：Research paper (international conference proceedings)  

Language：English   Publishing type：Research paper (other academic)  

© 2017 IEEE. This paper presents a near-future perception system named 'Previewed Reality'. The system consists of an informationally structured environment (ISE), an immersive VR display, a stereo camera, an optical tracking system, and a dynamic simulator. In an ISE, a number of sensors are embedded, and information such as the position of furniture, objects, humans, and robots, is sensed and stored in a database. The position and orientation of the immersive VR display are also tracked by an optical tracking system. Therefore, we can forecast the next possible events using a dynamic simulator and synthesize virtual images of what users will see in the near future from their own viewpoint. The synthesized images, overlaid on a real scene by using augmented reality technology, are presented to the user. The proposed system can allow a human and a robot to coexist more safely by showing possible hazardous situations to the human intuitively in advance.

DOI： 10.1109/IROS.2017.8202181

Language：English   Publishing type：Research paper (international conference proceedings)  

Language：Others   Publishing type：Research paper (other academic)  

© 2017 IEEE. This paper proposes new software and hardware platforms named ROS-TMS and Big Sensor Box, respectively, for an informationally structured environment. We started the development of a management system for an informationally structured environment named Town Management System (TMS) in the Robot Town Project in 2005. Since then we have been continuing our efforts to improve performance and to enhance TMS functions. Recently, we launched a new version of TMS named ROS-TMS, which resolves some critical problems in TMS by adopting the Robot Operating System (ROS) and utilizing the high scalability and numerous resources of ROS. In this paper, we first discuss the structure of a software platform for the informationally structured environment and describe in detail our latest system, ROS-TMS version 4.0. Next, we introduce a hardware platform for the informationally structured environment named Big Sensor Box, in which a variety of sensors are embedded and service robots are operated according to the structured information under the management of ROS-TMS. Robot service experiments including a fetch-and-give task and autonomous control of a wheelchair robot are also conducted in Big Sensor Box.

DOI： 10.1109/ICRA.2017.7989420

Language：Others   Publishing type：Research paper (other academic)  

© 2017 IEEE. Mutual subspace method (MSM), which is one of image-based approaches, showed strong discrimination capability in gait recognition. In general, 2D image matrices are transformed into 1D image vectors to be used as input into MSM, and then principal component analysis (PCA) is applied to 1D vectors to generate a subspace. However, due to the high dimensionalities of 1D vectors, the evaluation accuracy of the covariance matrix in PCA is not high enough. This results in a decrease in performance, especially in case that speed difference between gallery and probe dataset is big. Thus in this paper we propose a method, which expands the MSM-based method, to recognize people with higher accuracy. The proposed method divides the human body area into multiple areas, followed by adaptive choice of areas that have high discrimination capability. Moreover, the proposed method utilizes the frieze pattern, which is one of gait features, as an additional input into MSM. The use of divided areas and the frieze pattern allows us to evaluate the covariance matrix with higher accuracy. In experiments we applied the proposed method to challenging databases with speed variations, and we show the effectiveness of the proposed method.

DOI： 10.1109/ICRA.2017.7989261

Language：English  

Language：English   Publishing type：Research paper (international conference proceedings)  

Language：English   Publishing type：Research paper (other academic)  

© 2017 SPIE. This paper presents a new object classification technique for 3D point cloud data acquired with a laser scanner. In general, it is not straightforward to distinguish objects that have similar 3D structures but belong to different categories based only on the range data. To tackle this issue, we focus on laser reflectance obtained as a side product of range measurement by a laser scanner. Since laser reflectance contains appearance information, the proposed method classifies objects based on not only geometrical features in range data but also appearance features in reflectance data, both of which are acquired by a single laser scanner. Furthermore, we extend the conventional Histogram of Oriented Gradients (HOG) so that it couples geometrical and appearance information more tightly. Experiments show the proposed technique combining geometrical and appearance information outperforms conventional techniques.

DOI： 10.1117/12.2265178

Language：English   Publishing type：Research paper (international conference proceedings)  

Language：English   Publishing type：Research paper (scientific journal)  

© 2016 The Author(s) 3D modeling of real objects by a 3D laser scanner has become popular in many applications, such as reverse engineering of petrochemical plants, civil engineering and construction, and digital preservation of cultural properties. Despite the development of lightweight and high-speed laser scanners, the complicated measurement procedure and long measurement time are still heavy burdens for widespread use of laser scanning. To solve these problems, a robotic 3D scanning system using multiple robots has been proposed. This system, named CPS-SLAM, consists of a parent robot with a 3D laser scanner and child robots with target markers. A large-scale 3D model is acquired by an on-board 3D laser scanner on the parent robot from several positions determined precisely by a localization technique, named the Cooperative Positioning System (CPS), that uses multiple robots. Therefore, this system can build a 3D model without complicated post-processing procedures such as ICP. In addition, this system is an open-loop SLAM system and a very precise 3D model can be obtained without closed loops. This paper proposes an automatic planning technique for a laser measurement by using CPS-SLAM. Planning a proper scanning strategy depending on a target structure makes it possible to perform laser scanning efficiently and accurately even for a large-scale and complex environment. The proposed technique plans an efficient scanning strategy automatically by taking account of several criteria, such as visibility between robots, error accumulation, and efficient traveling. We conducted computer simulations and outdoor experiments to verify the performance of the proposed technique.

DOI： 10.1016/j.cviu.2016.05.008

Language：Japanese   Publishing type：Research paper (scientific journal)  

Language：English   Publishing type：Research paper (other academic)  

© 2016 IEEE. In this paper, for the purpose of monitoring people moving on the ground from a low-altitude aerial vehicle, we propose a method for stable image stabilization using planar information of the ground. In existing methods the homography-based method has been popularly used for image stabilization. However, in case that captured images consist of non-planar areas, the performance of the homography-based method decreases. The essential-based method works well in this scene, but the accuracy gets worse in case that images mainly include planar areas. Thus in the proposed method, we utilize the Geometric Robust Information Criterion (GRIC) model to choose a method from the homography-based method or the essential-based method. After the selection of the method, we extract a planar area in the scene by fitting a plane using RANSAC, followed by detection of people on the ground with high accuracy. Experimental results confirm that the effectiveness of the proposed method.

DOI： 10.1109/SMC.2016.7844929

Language：English   Publishing type：Research paper (international conference proceedings)  

Language：English   Publishing type：Research paper (other academic)  

© 2017 by SCITEPRESS - Science and Technology Publications, Lda. All rights reserved. This paper proposes prediction methods for people flows and anomalies in people flows on a university campus. The proposed methods are based on deep learning frameworks. By predicting the statistics of people flow conditions on a university campus, it becomes possible to create applications that predict future crowded places and the time when congestion will disappear. Our prediction methods will be useful for developing applications for solving problems in cities.

DOI： 10.5220/0006248806760683

Language：Others   Publishing type：Research paper (other academic)  

Simulation of Deforming Human Tissue by Multiple Deep Neural Networks

Language：Others   Publishing type：Research paper (other academic)  

A Method for Estimating Tissue Deformation by Deep Neural Network

Language：English   Publishing type：Research paper (international conference proceedings)  

Language：Others   Publishing type：Research paper (other academic)  

DOI： 10.1109/SII.2016.7844021

Language：English   Publishing type：Research paper (other academic)  

© 2016 IEEE. We propose an object tracking system which recognizes everyday objects and estimates their positions by using distributed sensors in a room and mobile robots. The placement of objects is frequently changed according to human activities. Although a passive RFID tag is attached to each object for the object's recognition, the placement is often not uniquely determined due to the deficiency of measured data. We have already proposed a method for estimating the placement of objects by using the moving trajectories of objects. This estimation result is expressed as the probability distribution of the object placement. However intersections of trajectories cause the decrease of the estimation accuracy. So we propose a new method based on Bayesian inference to improve the estimation accuracy by using the size and the shape of an object measured by laser range finder. Then a mobile robot settles the placement with small workload by using the mounted sensor. The system successfully recognized and localized 10 objects in the experiment.

DOI： 10.1109/IECON.2016.7793355

Language：English   Publishing type：Research paper (international conference proceedings)  

Language：English   Publishing type：Research paper (international conference proceedings)  

Language：English   Publishing type：Research paper (other academic)  

© 2016 IEEE. We present two multi-modal panoramic 3D outdoor (MPO) datasets for semantic place categorization with six categories: forest, coast, residential area, urban area and indoor/outdoor parking lot. The first dataset consists of 650 static panoramic scans of dense (9,000,000 points) 3D color and reflectance point clouds obtained using a FARO laser scanner with synchronized color images. The second dataset consists of 34,200 real-time panoramic scans of sparse (70,000 points) 3D reflectance point clouds obtained using a Velodyne laser scanner while driving a car. The datasets were obtained in the city of Fukuoka, Japan and are publicly available in [1], [2]. In addition, we compare several approaches for semantic place categorization with best results of 96.42&#37; (dense) and 89.67&#37; (sparse).

DOI： 10.1109/IROS.2016.7759669

Language：Japanese  

Language：English   Publishing type：Research paper (international conference proceedings)  

Language：English   Publishing type：Research paper (international conference proceedings)  

Language：English   Publishing type：Research paper (international conference proceedings)  

Language：English   Publishing type：Research paper (international conference proceedings)  

Language：English   Publishing type：Research paper (scientific journal)  

DOI： 10.1016/j.cviu.2016.05.008

Language：English   Publishing type：Research paper (scientific journal)  

Language：English   Publishing type：Research paper (other academic)  

© 2016 SPIE. This paper proposes a new method for mapping volume models of human tissues onto a target volume with simple shapes. The proposed method is based on our modified self-organizing deformable model (mSDM)1, 2 which finds the one-to-one mapping with no foldovers between arbitrary surface model and a target surface. By extending mSDM to apply to volume models, the proposed method, called volumetric SDM (vSDM), establishes the one-to-one correspondence between the tissue volume model and its target volume. At the same time, vSDM can preserve geometrical properties of the original model before and after mapping. This characteristic of vSDM makes it easy to find the correspondence between tissue models.

DOI： 10.1117/12.2217367

Language：English   Publishing type：Research paper (other academic)  

© 2015 IEEE. We propose a novel method for stabilizing videos taken by a low altitude aerial vehicle, for the purpose of monitoring activities on the ground. A popular choice for aerial video stabilization is the use of homography-based methods, which assume that the scene consists mostly of planar regions. When videos are recorded in low altitude flight, the regions with buildings, trees and hills are non-planar areas (with relatively big changes in height), and making the planar assumption would decrease the accuracy of stabilization with respect to the ground. The idea presented in this paper is to stabilize aerial images, where height changes exist, by explicitly estimating a planar area in the scene. For the estimation of planar region, a relative pose between two images is estimated by taking advantages of the homography-based method and the essential matrix-based method. Positions in the 3D space are reconstructed from feature points on the images, which are used for the pose estimation, and points on the plane are estimated based on RANSAC. The points on the flat area are used for stabilization of images with high accuracy. The experimental results with simulated aerial images illustrate the proposed method can estimate relative pose with higher accuracy compared with previous approaches, even when noise is present. The proposed method is also applied to real aerial images, in people tracking. Experimental results show that tracking using the proposed method has better performance than tracking using the homography-based method.

DOI： 10.1109/EST.2015.14

Language：English   Publishing type：Research paper (scientific journal)  

© 2016 Taylor & Francis and The Robotics Society of Japan. This paper presents an effective integration method of multiple modalities such as depth, color, and reflectance for place categorization. To achieve better performance with integrated multi-modalities, we introduce a novel descriptor, local N-ary patterns (LTP), which can perform robust discrimination of place categorization. In this paper, the LNP descriptor is applied to a combination of two modalities, i.e. depth and reflectance, provided by a laser range finder. However, the LNP descriptor can be easily extended to a larger number of modalities. The proposed LNP describes relationships between the multi-modal values of pixels and their neighboring pixels. Since we consider the multi-modal relationship, our proposed method clearly demonstrates more effective classification results than using individual modalities. We carried out experiments with the Kyushu University Indoor Semantic Place Dataset, which is publicly available. This data-set is composed of five indoor categories: corridors, kitchens, laboratories, study rooms, and offices. We confirmed that our proposed method outperforms previous uni-modal descriptors.

DOI： 10.1080/01691864.2015.1120242

Language：English   Publishing type：Research paper (other academic)  

© 2015 IEEE. In this paper, we present a control architecture that enables a service drone to navigate in an informationally structured environment (ISE) and to accomplish a specific service task autonomously based on the ROS-TMS framework. The ROS-TMS is a ROS-based distributed information management system for ISE. It manages a variety of subsystems ranging from sensing by distributed sensors to motion planning and behavior control for service robots. The proposed architecture is designed as a component of the ROS-TMS, and consists of a navigation system that solves a path planning in ISE and a flight control system by a behavior-based finite state machine. The navigation of the drone is allowed by an optical motion tracking system consisting of distributed infrared cameras managed by the ROS-TMS. A graphical user interface is designed to provide simple manipulation of a service task by a drone. Service experiments have been conducted to validate the performance of the architecture.

DOI： 10.1109/SII.2015.7405049

Language：English   Publishing type：Research paper (other academic)  

© 2015 IEEE. This paper proposes new features extracted from images derived from optical flow, for first-person activity recognition. Features from convolutional neural network (CNN), which is designed for 2D images, attract attention from computer vision researchers due to its powerful discrimination capability, and recently a convolutional neural network for videos, called C3D (Convolutional 3D), was proposed. Generally CNN / C3D features are extracted directly from original images / videos with pre-trained convolutional neural network, since the network was trained with images / videos. In this paper, on the other hand, we propose the use of images derived from optical flow (we call this image as "optical flow image") as input images into the pre-trained neural network, based on the following reasons; (i) optical flow images give dynamic information which is useful for activity recognition, compared with original images, which give only static information, and (ii) the pre-trained network has chance to extract features with reasonable discrimination capability, since the network was trained with huge amount of images from big categories. We carry out experiments with a dataset named "DogCentric Activity Dataset", and we show the effectiveness of the extracted features.

DOI： 10.1109/SII.2015.7405050

Language：English  

Language：English   Publishing type：Research paper (other academic)  

© 2015 IEEE. In recent years, several low-cost 3D laser scanners are being brought to the market and 3D laser scanning is becoming widely used in many fields. For example, 3D modeling of architectural structures or digital preservation of cultural heritages are typical applications for 3D laser scanning. Despite of the development of light-weight and high-speed laser scanners, however, the complicated measurement procedure and long measurement time are still a heavy burden for the widespread use of laser scanning. We have proposed a robotic 3D scanning system using multiple robots named CPS-SLAM, which consists of parent robots with a 3D laser scanner and child robots with target markers. In this system, a large-scale 3D model can be acquired by an on-board 3D laser scanner on a parent robot from several positions determined precisely by the localization technique using multiple robots named Cooperative Positioning System, CPS. Therefore, this system enables to build a 3D model without complicated post-processing procedures such as ICP. In addition, this system is an open-loop SLAM system and a quite precise 3D model can be obtained without closed loops. This paper proposes an automatic planning technique of a laser measurement for CPS-SLAM. By planning a proper scanning strategy depending on a target structure, it is possible to perform laser scanning efficiently and accurately even for a large-scale and complex environment. Proposed technique plans an efficient scanning strategy automatically by taking account of several criteria, such as visibility between robots, error accumulation, and efficient traveling. We conducted computer simulations and outdoor experiments to verify the performance of the proposed technique.

DOI： 10.1109/IROS.2015.7354007

Language：English   Publishing type：Research paper (international conference proceedings)  

Language：English   Publishing type：Research paper (international conference proceedings)  

Language：English   Publishing type：Research paper (other academic)  

© 2015 IEEE. This paper proposes a new concept named "fourth-person sensing" for a service robot. The proposed concept combines wearable sensors (the first-person viewpoint), sensors mounted on robots (the second-person viewpoint), and sensors embedded in the environment (the third-person viewpoint), and the disadvantages in individual sensors are compensated by combining sensory information from the first, second, and third-person viewpoints. The fourth-person sensing is effective to understand a user's intention and a context of the scene, thus it enables to provide a proper service by a service robot. As one of applications of the fourth-person sensing, we develop a HCI system combining the first-person and the third-person sensing, and show the effectiveness of the proposed concept through robot service experiments.

DOI： 10.1109/ICSENS.2015.7370461

Language：English   Publishing type：Research paper (scientific journal)  

© 2015 The Authors Daily life assistance is one of the most important applications for service robots. For comfortable assistance, service robots must recognize the surrounding conditions correctly, including human motion, the position of objects, and obstacles. However, since the everyday environment is complex and unpredictable, it is almost impossible to sense all of the necessary information using only a robot and sensors attached to it. In order to realize a service robot for daily life assistance, we have been developing an informationally structured environment using distributed sensors embedded in the environment. The present paper introduces a service robot system with an informationally structured environment referred to the ROS–TMS. This system enables the integration of various data from distributed sensors, as well as storage of these data in an on-line database and the planning of the service motion of a robot using real-time information about the surroundings. In addition, we discuss experiments such as detection and fetch-and-give tasks using the developed real environment and robot.

DOI： 10.1016/j.robot.2015.07.010

Language：English   Publishing type：Research paper (other academic)  

© 2015 IEEE. We propose an evaluation method of grasp stability which takes into account the elastic deformation of fingertips from the viewpoint of energy. An evaluation value of grasp stability is derived as the minimum energy which causes slippage of a fingertip on its contact surface. To formulate the evaluation value, the elastic potential energy of fingertips and the gravitational potential energy of a grasped object are considered. It is ensured that fingertips do not slip on grasped object surfaces if the external energy applied to the object is less than the evaluation value. Since our evaluation value explicitly considers the deformation values of fingertips, grasp stability is evaluated by taking into consideration the contact forces generated by the deformation. The effectiveness of our method is verified through numerical examples.

DOI： 10.1109/IROS.2015.7353688

Language：English   Publishing type：Research paper (international conference proceedings)  

Language：Japanese   Publishing type：Research paper (scientific journal)  

Language：English   Publishing type：Research paper (scientific journal)  

Language：English   Publishing type：Research paper (international conference proceedings)  

Language：English   Publishing type：Research paper (international conference proceedings)  

Language：English   Publishing type：Research paper (international conference proceedings)  

Language：English   Publishing type：Research paper (international conference proceedings)  

Language：English   Publishing type：Research paper (other academic)  

© 2015 IEEE. The problem of multipath propagation in the tracking low-altitude targets with a radar is addressed. It is well known that multipath fading causes bias errors to the target altitude over the sea. Since the bias error caused by multipath propagation depends on a large number of parameters such as the frequency of the radar waveform, the actual target altitude, and range, it is difficult to estimate the bias errors. In this paper, we propose an altitude estimation method using particle filter and multipath propagation model. The performance of the proposed method is verified through computer simulations.

DOI： 10.1109/AIM.2015.7222532

Language：English   Publishing type：Research paper (other academic)  

© 2015 IEEE. Daily life assistance for elderly individuals in hospitals and care facilities is one of the most urgent and promising applications for service robots. Especially, a fetch-and-give task is a frequent and fundamental task for service robots to assist elderlys daily life. In hospitals and care facilities, this task is often performed with a movable platform such as a wagon or a cart to carry and deliver a large amount of objects at once. Thus the navigation and control of not only a service robot but also a movable platform must be planned safely. In addition, a robot motion planning to hand over an object to a person safely and comfortably according to his/her posture is also an important problem in this task, however this has not been discussed so much. In this work, we present a coordinate motion planning technique for a fetch-and-give task using a wagon and a service robot. Handover motion is also planned by considering the manipulability of both a robot and a person. Experiments of a fetch-and-give task using a service robot are successfully carried out.

DOI： 10.1109/AIM.2015.7222657

Language：English   Publishing type：Research paper (other academic)  

© 2015 IEEE. This paper presents a new immersive VR interface bridging cyber and physical worlds for Cyber Physical System (CPS). Informationaly structured environment (ISE) has been proposed in service robotics so far. In ISE, environmental information such as positions of objects, furniture, humans, and robots is gathered by embedded sensor networks and stored in a database structurally. A service robot is able to utilize these information anytime and anywhere by connecting to the network. In this paper, we introduce the ISE architecture named ROS-TMS and a new cyber-physical interface for ROS-TMS. The proposed system consists of an immersive wearable display, a stereo camera, an optical tracking system, and an environmental simulator, and is able to present forecasted images with high reality based on the structured information in ISE.

DOI： 10.1109/AIM.2015.7222802

Language：Japanese  

Language：English   Publishing type：Research paper (scientific journal)  

© 2015 by the authors. The application of assistive technologies for elderly people is one of the most promising and interesting scenarios for intelligent technologies in the present and near future. Moreover, the improvement of the quality of life for the elderly is one of the first priorities in modern countries and societies. In this work, we present an informationally structured room that is aimed at supporting the daily life activities of elderly people. This room integrates different sensor modalities in a natural and non-invasive way inside the environment. The information gathered by the sensors is processed and sent to a centralized management system, which makes it available to a service robot assisting the people. One important restriction of our intelligent room is reducing as much as possible any interference with daily activities. Finally, this paper presents several experiments and situations using our intelligent environment in cooperation with our service robot.

DOI： 10.3390/s150409438

Language：English   Publishing type：Research paper (other academic)  

© 2015 IEEE. Person recognition from gait images is generally not robust to changes in appearance, such as variations of the walking direction. In general conventional methods have focused on training a model to transform gait features or gait images to those at a different viewpoint, but the performance gets worse in case the model is not trained at a viewpoint of a subject. In this paper we propose a novel gait recognition approach which differs a lot from existing approaches in that the subject's sequential 3D models and his/her motion are directly reconstructed from captured images, and arbitrary viewpoint images are synthesized from the reconstructed 3D models for the purpose of gait recognition robust to changes in the walking direction. Moreover, we propose a gait feature, named Frame Difference Frieze Pattern (FDFP), which is robust to high frequency noise. The efficiency of the proposed method is demonstrated through experiments using a database that includes 41 subjects.

DOI： 10.1109/WACV.2015.95

Language：English   Publishing type：Research paper (international conference proceedings)  

Language：English   Publishing type：Research paper (scientific journal)  

Language：Japanese   Publishing type：Research paper (scientific journal)  

Language：English  

Language：English   Publishing type：Research paper (scientific journal)  

© 2014 Taylor & Francis and The Robotics Society of Japan. While laser scanners can produce a high-precision 3D shape of a real object, appearance information of the object has to be captured by an image sensor, such as a digital camera. This paper proposes a novel and simple technique for colorizing 3D geometric models based on laser reflectivity. Laser scanners capture the range data of a target object from the sensors. Simultaneously, the power of the reflected laser is obtained as a by-product of the range data. The reflectance image, which is a collection of laser reflectance depicted as a grayscale image, contains rich appearance information about the target object. The proposed technique is an alternative to texture mapping, which has been widely used to realize photo-realistic 3D modeling but requires strict alignment between range data and texture images. The proposed technique first colorizes a reflectance image based on the similarity of color and reflectance images. Then the appearance information (color and texture information) is added to a 3D model by transferring the color in the colorized reflectance image to the corresponding range image. Some experiments and comparisons between texture mapping and the proposed technique demonstrate the validity of the proposed technique.

DOI： 10.1080/01691864.2014.968616

Language：English   Publishing type：Research paper (other academic)  

© 2014 IEEE. Indoor place categorization is an important capability for service robots working and interacting in human environments. This paper presents a new place categorization method which uses information about the spatial correlation between the different image modalities provided by RGB-D sensors. Our approach applies co-occurrence histograms of local binary patterns (LBPs) from gray and depth images that correspond to the same indoor scene. The resulting histograms are used as feature vectors in a supervised classifier. Our experimental results show the effectiveness of our method to categorize indoor places using RGB-D cameras.

DOI： 10.1109/EST.2014.23

Language：English   Publishing type：Research paper (other academic)  

© 2014 IEEE. This paper presents a person identification technique that uses information from person's shadow, and is robust to appearance changes caused by variations of clothes and carried objects. The technique uses invisible lights and resulting shadows and has advantages from undetected sensing. The shadows on the ground obtained through illumination by multiple lights can be considered as silhouettes captured by multiple virtual cameras placed at light positions. Thus, a single camera, e.g. in the ceiling, is able to obtain multiple silhouettes, equivalent to a multi-camera system. If the person's appearance changes compared to the training cases in the database, e.g. by wearing different clothes or carrying a/another bag, then the identification performance gets worse. To deal with this problem, we introduce a new shadow-based identification technique robust to appearance changes. Firstly, we divide each shadow area into several parts, and estimate the discrimination capability for each part based on gait features between gallery datasets and probe dataset. Next, according to the estimated capability, we adaptively control the priorities of these parts in the person identification method. We constructed a new shadow database with a variety of clothes and bags, and carried out successful experiments to verify the effectiveness of the proposed technique.

DOI： 10.1109/EST.2014.18

Language：English   Publishing type：Research paper (other academic)  

© 2014 IEEE. Service robots, which co-exist with humans to provide various services, obtain information from sensors placed in an environment and/or sensors mounted on robots. In this paper we newly propose the concept of fourth-person sensing which combines wearable cameras (first-person sensing), sensors mounted on robots (second-person sensing), and distributed sensors in the environment (third-person sensing). The proposed concept takes advantages of all three sensing systems, while removing disadvantage of each of them. The first-person sensing can analyze what a person wearing a camera is doing and details around him/her, so the fourth-person sensing has chance to provide pro-active services, which are triggered by predicted human intention and difficult with the second- and third-person systems, by estimating his intention/behavior. We introduce an example scenario using the fourth-person sensing and show the effectiveness of the proposed concept through experiments.

DOI： 10.1109/EST.2014.17

Language：English   Publishing type：Research paper (other academic)  

© 2014 IEEE. This paper introduces the concept of first-person animal activity recognition, the problem of recognizing activities from a view-point of an animal (e.g., a dog). Similar to first-person activity recognition scenarios where humans wear cameras, our approach estimates activities performed by an animal wearing a camera. This enables monitoring and understanding of natural animal behaviors even when there are no people around them. Its applications include automated logging of animal behaviors for medical/biology experiments, monitoring of pets, and investigation of wildlife patterns. In this paper, we construct a new dataset composed of first-person animal videos obtained by mounting a camera on each of the four pet dogs. Our new dataset consists of 10 activities containing a heavy/fair amount of ego-motion. We implemented multiple baseline approaches to recognize activities from such videos while utilizing multiple types of global/local motion features. Animal ego-actions as well as human-animal interactions are recognized with the baseline approaches, and we discuss experimental results.

DOI： 10.1109/ICPR.2014.739

Language：English   Publishing type：Research paper (scientific journal)  

DOI： 10.1080/01691864.2014.968616

Language：English   Publishing type：Research paper (other academic)  

© 2014 IEEE. This paper presents a method for estimating the internal structures of a patient brain by deforming a standard brain atlas. Conventional deformation methods need several landmarks from the brain surface contour to fit the atlas to the patient brain shape. However, since the number and shapes of small sulci on the brain surface are different from each other, the determination of the accurate correspondence between small sulcus is difficult for experienced neurosurgeons. Moreover, the relationship between the surface shape and internal structure of the brain is unclear. Therefore, even if the deformed atlas is fitted to the patient brain shape exactly, the use of the deformed atlas does not always guarantee the reliable estimation of the internal structure of the patient brain. To solve these problems, we propose a new method for estimate the internal structure of a patient brain by the finite element method (FEM). In the deformation, our method select the landmarks from the contours of both the brain surface and the detectable internal structures from MR images.

DOI： 10.1109/EMBC.2014.6944886

Language：English   Publishing type：Research paper (international conference proceedings)  

Language：English   Publishing type：Research paper (other academic)  

© 2014 TSI Press. This paper proposes a new radar tracking filter named Noise-estimate Particle PHD Filter (NP-PHDF). Kalman filter and particle filter are popular filtering techniques for target tracking. However, the tracking performance of the Kalman filter severely depends on the setting of several parameters such as system noise and observation noise. It is an open problem how to choose proper parameters for various scenarios, and they are often regulated in trial-and-error manner. To tackle this problem, Noise-estimate Particle Filter (NPF) has been proposed so far. The NPF estimates proper noise parameters of a Kalman filter on-line based on a scheme of particle filter. In this paper, we extend the NPF so that it enables to track multiple targets simultaneously by combining with Probability Hypothesis Density (PHD) filter, and propose a new Noise-estimate Particle PHD Filter (NP-PHDF). Simulation results show that the proposed filter has higher tracking performance in various scenarios than conventional Kalman filter, particle filter, and PHD filter for multiple-targets tracking.

DOI： 10.1109/WAC.2014.6936154

Language：English   Publishing type：Research paper (scientific journal)  

Conventional methods of gait analysis for person identification use features extracted from a sequence of camera images taken during one or more gait cycles. The walking direction is implicitly assumed not to change. However, with the exception of very particular cases, such as walking on a circle centered on the camera, or along a line passing through the camera, there is always some degree of orientation change, most pronounced when the person is closer to the camera. This change in the angle between the velocity vector and the position vector in respect to the camera causes a decrease in performance for conventional methods. To address this issue we propose in this paper a new method, which provides improved identification in this context of orientation change. The proposed method uses a 4D gait database consisting of multiple 3D shape models of walking people and adaptive virtual image synthesis with high accuracy. Each frame, for the duration of a gait cycle, is used to estimate the walking direction of the subject, and a virtual image corresponding to the estimated direction is synthesized from the 4D gait database. The identification uses affine moment invariants as gait features. The efficiency of the proposed method is demonstrated through experiments using a database that includes 42 subjects. © 2014 Elsevier B.V. All rights reserved.

DOI： 10.1016/j.patrec.2014.04.004

Language：English   Publishing type：Research paper (other academic)  

© 2014 IEEE. This paper presents a grasp planner which allows a robot to grasp the constricted parts of objects in our daily life. Even though constricted parts can be grasped more firmly than convex parts, previous planners have not sufficiently focused on grasping this part. We develop techniques for quadric surface approximation, grasp posture generation, and stability evaluation for grasping constricted parts. By modeling an object into multiple quadric surfaces, the planner generates a grasping posture by selecting one-sheet hyperbolic surfaces or two adjacent ellipsoids as constricted parts. When a grasping posture being generated, the grasp stability is evaluated based on the distribution of the stress applied to an object by the fingers. We perform several simulations and experiments to verify the effectiveness of our proposed method.

DOI： 10.1109/IROS.2014.6942895

Language：English   Publishing type：Research paper (other academic)  

© 2014 TSI Press. This paper proposes a method to identify the spatial distribution patterns of cone cells related with blood vessel in a given retina image. We define three types of the distribution patterns between cones and vessels. Positive correlation distribution (PCD) and negative correlation distribution (NCD) indicate that the cones tend to be close to or far from the vessels. While the cone cells do not have significant correlation with vessels, the cone distribution is regarded as the random distribution (RD). In our method, RD is modeled by many virtual retina images, each of which is generated by the vessels extracted from the original retina image and the virtual cells are selected randomly from the image. Using the virtual images, we estimate the distribution range of RD. When the distribution of the original cells is above the upper limit or below the lower limit of the RD distribution, the cell distribution is NCD or PCD. Otherwise, the cell distribution is regarded as RD.

DOI： 10.1109/WAC.2014.6936144

Language：English   Publishing type：Research paper (scientific journal)  

DOI： 10.1016/j.patrec.2014.04.004

Language：English   Publishing type：Research paper (international conference proceedings)  

Language：English   Publishing type：Research paper (international conference proceedings)  

Language：English   Publishing type：Research paper (international conference proceedings)  

Language：English   Publishing type：Research paper (international conference proceedings)  

Language：English   Publishing type：Research paper (international conference proceedings)  

Language：English   Publishing type：Research paper (international conference proceedings)  

Language：English   Publishing type：Research paper (international conference proceedings)  

Language：English   Publishing type：Research paper (international conference proceedings)  

Language：English   Publishing type：Research paper (international conference proceedings)  

Language：English  

Language：Others  

A Method for Reconstructing 3D Tissue Shapes from Stereo Endoscopic Images Using Wide-Range Edge

Language：English  

3D Model Generation of Brain Block Using Internal Structure Contours for 3D Japanese Brain Atlas Construction (Medical Imaging)
The digital human modeling is one of the challenging topics in the last few decades. We have been studying on the digital brain atlas of Japanese. Our system for generating the brain atlas uses the latest techniques in both human anatomy and computer science. This paper reports the detail of generating the brain model by the 3D contour models of the brain surface and internal neural structures. Especially, we propose a new method for registering two sub-block models of the brain by combining a nonrigid registration and dynamic programming.

Language：English   Publishing type：Research paper (scientific journal)  

This paper describes a new method of measuring the position of everyday objects and a robot on the floor using distance and reflectance acquired by laser range finder (LRF). The information obtained by this method is important for a service robot working in a human daily life environment. Our method uses only one LRF together with a mirror installed on the wall. Moreover, since the area of sensing is limited to a LRF scanning plane parallel to the floor and just a few centimeters above the floor, the scanning covers the whole room with minimal invasion of privacy of a resident, and occlusion problem is mitigated by using mirror. We use the reflection intensity and position information obtained from the target surface. Although it is not possible to identify all objects by additionally using reflection values, it would be easier to identify unknown objects if we can eliminate easily identifiable objects by reflectance. In addition, we propose a method for measuring the robot's pose using the tag which has the encoded reflection pattern optically identified by the LRF. Our experimental results validate the effectiveness of the proposed method. © 2014 by the authors; licensee MDPI, Basel, Switzerland.

DOI： 10.3390/s140407524

Language：English  

Language：English   Publishing type：Research paper (other academic)  

© 2014 IEEE. We propose a grasp stability index which takes into account the elastic deformation of fingertips. In this index, an evaluation value of grasp stability is derived as the minimum energy which causes slippage of a fingertip on a grasped object surface. Elastic potential energy of the fingertips and gravitational potential energy of the object are considered in this index. It is ensured that any fingertip does not slip on a grasped object surface if the external energy added to the object is less than the evaluation value. This index has good features as follows. (i) Contact forces corresponding to the grasping state are taken into account. (ii) It is possible to derive a condition of stable grasp including the kinetic energy of a grasped object. The effectiveness of this index is verified through several numerical examples.

DOI： 10.1109/SII.2014.7028081

Language：English   Publishing type：Research paper (other academic)  

© 2014 IEEE. We present a novel approach for outdoor place categorization using synchronized texture and depth images obtained using a laser scanner. Categorizing outdoor places according to type is useful for autonomous driving or service robots, which work adaptively according to the surrounding conditions. However, place categorization is not straight forward due to the wide variety of environments and sensor performance limitations. In the present paper, we introduce a two-dimensional local ternary pattern (2D-LTP) descriptor using a pair of synchronized texture and depth images. The proposed 2D-LTP describes the local co-occurrence of a synchronized and complementary image pair with ternary patterns. In the present study, we construct histograms of a 2D-LTP as a feature of an outdoor place and apply singular value decomposition (SVD) to deal with the high dimensionality of the place. The novel descriptor, i.e., the 2D-LTP, exhibits a higher categorization performance than conventional image descriptors with outdoor place experiments.

DOI： 10.1109/ICIP.2014.7026158

Language：English   Publishing type：Research paper (international conference proceedings)  

Language：English   Publishing type：Research paper (scientific journal)  

This paper presents an approach to categorize typical places in indoor environments using 3D scans provided by a laser range finder. Examples of such places are offices, laboratories, or kitchens. In our method, we combine the range and reflectance data from the laser scan for the final categorization of places. Range and reflectance images are transformed into histograms of local binary patterns and combined into a single feature vector. This vector is later classified using support vector machines. The results of the presented experiments demonstrate the capability of our technique to categorize indoor places with high accuracy. We also show that the combination of range and reflectance information improves the final categorization results in comparison with a single modality. © 2013 Taylor & Francis and The Robotics Society of Japan.

DOI： 10.1080/01691864.2013.839091

Language：Japanese  

Language：Japanese   Publishing type：Research paper (scientific journal)  

Language：English   Publishing type：Research paper (international conference proceedings)  

Language：English   Publishing type：Research paper (international conference proceedings)  

Language：Japanese  

Size-Adaptive Hepatocellular Carcinoma Detection from 3-D CT Images Based on the Level Set Method
本論文では,CT画像を用いた,様々なサイズの原発性肝がん(以下,腫瘍)に対応可能な腫瘍検出法を報告する.従来,CT画像を用いた腫瘍検出では,1cm以上の腫瘍を自動検出する研究が盛んであり,これらの研究では2種類の異なる画像(動脈相,遅延相)が用いられてきた.このうち,遅延相は正確な腫瘍サイズ計測が可能であるが,コントラストが低いため1cm以下の小結節の検出が難しい.一方,動脈相では小結節検出が可能になるが,雑音と腫瘍の輝度値が類似しているため誤検出が生じやすい,という問題があった.そこで本論文では,様々なサイズの腫瘍が検出可能な動脈相を用いた腫瘍の自動検出法を提案する.提案手法では,複数腫瘍の自動検出を実現するため,腫瘍個数の事前知識が不要なLevel Set Methodを導入する.更に,低コントラストや雑音に対応した,新たな成長速度項を提案する.提案手法を臨床画像24症例に適用した結果,Sensitivity 90&#37;以上,誤検出が1症例当り3.1個となり,従来のLevel Set Methodを用いた検出手法と比較して,検出精度が大幅に向上することを確認した.

Language：English   Publishing type：Research paper (international conference proceedings)  

Language：English   Publishing type：Research paper (international conference proceedings)  

Language：English   Publishing type：Research paper (international conference proceedings)  

Language：English   Publishing type：Research paper (scientific journal)  

Language：English   Publishing type：Research paper (scientific journal)  

In this paper, we propose new denoising techniques for a deteriorated range image taken by a laser scanner. Laser scanner acquires a range value from the scanner to the target by measuring the round-trip time of the emitted laser pulse. At the same time, they can obtain the strength of the reflected light as a side product of the range value. Focusing on the laser intensity, we propose two denoising techniques for a deteriorated range image utilizing the intensity image: smoothing by extended bilateral filter, and completion by belief propagation. The extended bilateral filter makes use of laser intensity in addition to the spatial and range information in order that we can smooth a range image corrupted by noises while the geometric features such as jump and roof edges are preserved. The range image completion technique with belief propagation restores a deteriorated range image using the adjacent range values and the corresponding intensity values simultaneously. We conduct simulations and experiments using synthesized images and actual range images taken by a laser scanner and verify that the proposed techniques suppress noise while preserving jump and roof edges and repair deteriorated range images. © 2013 Taylor & Francis Group, LLC.

DOI： 10.1080/01691864.2013.797141

Language：English   Publishing type：Research paper (international conference proceedings)  

Language：English   Publishing type：Research paper (scientific journal)  

The identification of a person from gait images is generally sensitive to appearance changes, such as variations of clothes and belongings. One possibility to deal with this problem is to collect possible subjects' appearance changes in a database. However, it is almost impossible to predict all appearance changes in advance. In this paper, we propose a novel method, which allows robustly identifying people in spite of changes in appearance, without using a database of predicted appearance changes. In the proposed method, firstly, the human body image is divided into multiple areas, and features for each area are extracted. Next, a matching weight for each area is estimated based on the similarity between the extracted features and those in the database for standard clothes. Finally, the subject is identified by weighted integration of similarities in all areas. Experiments using the gait database CASIA show the best correct classification rate compared with conventional methods experiments. © 2013 by the authors; licensee MDPI, Basel, Switzerland.

DOI： 10.3390/s130607884

Language：English   Publishing type：Research paper (scientific journal)  

Language：English   Publishing type：Research paper (international conference proceedings)  

Language：English   Publishing type：Research paper (international conference proceedings)  

Language：English  

Language：English   Publishing type：Research paper (international conference proceedings)  

Language：English   Publishing type：Research paper (other academic)  

Conventional methods of gait analysis for person identification use features extracted from a sequence of camera images taken during one or more gait cycles. An implicit assumption is made that the walking direction does not change. However, cameras deployed in real-world environments (and often placed at corners) capture images of humans who walk on paths that, for a variety of reasons, such as turning corners or avoiding obstacles, are not straight but curved. This change of the direction of the velocity vector causes a decrease in performance for conventional methods. In this paper we address this aspect, and propose a method that offers improved identification results for people walking on curved trajectories. The large diversity of curved trajectories makes the collection of complete real world data infeasible. The proposed method utilizes a 4D gait database consisting of multiple 3D shape models of walking subjects and adaptive virtual image synthesis. Each frame, for the duration of a gait cycle, is used to estimate a walking direction for the subject, and consequently a virtual image corresponding to this estimated direction is synthesized from the 4D gait database. The identification uses affine moment invariants as gait features. Experiments using the 4D gait database of 21 subjects show that the proposed method has a higher recognition performance than conventional methods. © 2013 IEEE.

DOI： 10.1109/WACV.2013.6475018

Language：Others   Publishing type：Research paper (scientific journal)  

To estimate the internal structures of the patient brain, several methods map a brain atlas to a patient brain shape using some landmarks, which are selected manually from the brain shape. The determination of the correspondence between small sulci of arbitrary two brains is complex. Moreover, the relationship between the surface shape and internal structure of the brain is unclear. Therefore, even if the surface the deformed atlas is fitted to the patient brain shape, the accurate internal structures of the brain can not always be obtained. To solve these problems, this paper proposes a method of selecting landmarks to estimate the internal structures of the patient brain by deforming brain atlas. Firstly, the brain shapes are represented by a simple shape. Secondly, a small number of initial landmarks are selected manually from the contours of the approximated brain shape and the internal structures which can be identified from MR images. Thirdly, some new landmarks are generated automatically on the contours based on the initial landmarks. Finally, the brain atlas is deformed non-rigidly using these landmarks. To evaluate our method, we estimate the patient brain structure using 4 methods. From the experimental results using 10 brain images, our method can estimate the patient brain structure reliably and stably using a small number of landmarks compared with the method using the landmarks on the original brain shape.

Language：English   Publishing type：Research paper (scientific journal)  

Language：English   Publishing type：Research paper (scientific journal)  

Language：English   Publishing type：Research paper (scientific journal)  

Language：English   Publishing type：Research paper (scientific journal)  

Language：English   Publishing type：Research paper (scientific journal)  

Language：English   Publishing type：Research paper (scientific journal)  

We propose a novel biometrics method based on shadows (shadow biometrics, SB) and introduce a SB-based person identification method for a vision-based surveillance system. Conventional biometric identification based on body movements, as is the case in gait recognition, uses cameras that provide a good view of entire human body. Aerial search and surveillance systems only see the human body from top view with a smaller cross-section and with less details than seen in side views, which is further aggravated by the lower resolution associated with this imagery. Shadows, i.e. body projections due to the Sun, or artificial lights at night, can offer body biometrics information that cannot be directly seen in body top view. In this paper we use SB for person identification, automatically extracting shadows in captured video images, and processing them to extract gait features, further analyzed by spherical harmonics. We demonstrate shadow-based person identification in experiments inside a building using artificial light and outside under the Sun. The introduced method using spherical harmonics outperforms methods based on Fourier transform, gait energy image, and active energy image. Furthermore, we show that the combination of body and shadow areas, as seen from an oblique camera on an upper floor of a building, has better performance than using body only or shadow only information. © 2012 Elsevier B.V. All rights reserved.

DOI： 10.1016/j.patrec.2012.07.022

Language：English   Publishing type：Research paper (scientific journal)  

Techniques for creating 3D shape model are expected to be applied to as-built management for structures under construction. However, common techniques currently widely used, still have lacks in usability and accuracy, hence still haven't reached the stage of actual use in the field. Due to the above circumstance, the authors have developed a 3D tunnel shape scanning system that uses a moving platform and auto tracking total station. The system provides improved usability and accuracy. Using the developed system, we have carried out a verification experiment in a mock tunnel, and we confirmed the possibility of real-time evaluation and the system'smeasuring accuracy under actual conditions. We also carried out measurements on three actual tunnel sites. We confirmed that it is fully capable of measurements not only in favorable environments but also in harsh environments, such as in tunnels that are under construction.

DOI： 10.20965/jrm.2012.p0992

Language：English   Publishing type：Research paper (international conference proceedings)  

Language：English  

Language：English   Publishing type：Research paper (scientific journal)  

Language：English   Publishing type：Research paper (scientific journal)  

© 2012 Cambridge University Press. This paper proposes a novel dynamic stable grasping method of an arbitrary polyhedral object for a hand-arm system with hemispherical fingertips. This method makes it possible to satisfy the force/torque equilibrium condition for the immobilization of the object without knowledge of the object. Two control signals are proposed which generate grasping forces normal and tangential to an object surface in a final state. The dynamics of the overall system is modeled and analyzed theoretically. We demonstrate the stable grasping of an arbitrary polyhedral object using the proposed controller through numerical simulations and experiments using a newly developed mechanical hand-arm system.

DOI： 10.1017/S0263574712000525

Language：English   Publishing type：Research paper (scientific journal)  

Language：English   Publishing type：Research paper (international conference proceedings)  

Language：English   Publishing type：Research paper (international conference proceedings)  

Language：English   Publishing type：Research paper (international conference proceedings)  

Language：English  

Language：English  

Language：English   Publishing type：Research paper (international conference proceedings)  

Language：English   Publishing type：Research paper (scientific journal)  

Global localization is a fundamental requirement for a mobile robot. Map-based global localization is a popular technique and gives a precise position by comparing a provided geometric map and current sensory data. However, it is quite time-consuming if 3D range data is processed for 6D global localization. On the other hand, appearance-based global localization using a captured image and recorded images is simple and suitable for real-time processing. However, this technique does not work in the dark or in an environment in which the lighting conditions change remarkably. To cope with these problems, we have proposed a two-step strategy which combines map-based global localization and appearance-based global localization. Firstly, several candidate positions are selected according to an appearance-based technique, and then the optimum position is determined by a map-based technique. Instead of camera images, we use reflectance images, which are captured by a laser range finder as a by-product of range sensing. In this paper, a new technique based on this global localization technique is proposed by combining the two step algorithm and a sampling-based approach. To cope with the odometry data, a particle filter is adopted for tracking robot positions. The effectiveness of the proposed technique is demonstrated through experiments in real environments.

DOI： 10.15017/21947

Language：English   Publishing type：Research paper (scientific journal)  

The categorization of places in indoor environments is an important capability for service robots working and interacting with humans. In this paper we present a method to categorize different areas in indoor environments using a mobile robot equipped with a Kinect camera. Our approach transforms depth and grey scale images taken at each place into histograms of local binary patterns (LBPs) whose dimensionality is further reduced following a uniform criterion. The histograms are then combined into a single feature vector which is categorized using a supervised method. In this work we compare the performance of support vector machines and random forests as supervised classifiers. Finally, we apply our technique to distinguish five different place categories: Corridors, laboratories, offices, kitchens, and study rooms. Experimental results show that we can categorize these places with high accuracy using our approach. © 2012 by the authors; licensee MDPI, Basel, Switzerland.

DOI： 10.3390/s120506695

Language：English   Publishing type：Research paper (bulletin of university, research institution)  

Language：English   Publishing type：Research paper (international conference proceedings)  

Language：Japanese  

Study on CPS-SLAM : Improvement of Measurement Precision and Application for Tunnel Shape Measurement System
This paper presents a high precision 3D measurement system by multiple mobile robots. This system is composed of three mobile robots consisting of a parent robot and two child robots. The parent robot is equipped with a 3D laser scanner, attitude sensor and a total station, and the child robots are equipped with corner cubes. The parent robot moves and stops repeatedly, and measures the 3D shape using the equipped laser scanner at several positions. Meanwhile, the child robots also move and stop repeatedly, and act as mobile landmarks for the positioning of the parent robot. To improve the positioning accuracy, several devices were replaced or newly installed. The experimental results show the system achieves quite high accuracy of the 0.03&#37; of target's size. As an example of applications, we applied the system to shape measurement of tunnels under construction, and verified that the accuracy of the developed system is as high as a high-precise 3D laser scanner.

DOI： 10.7210/jrsj.30.180

Language：English   Publishing type：Research paper (scientific journal)  

Language：English   Publishing type：Research paper (international conference proceedings)  

Language：English   Publishing type：Research paper (international conference proceedings)  

Language：Japanese   Publishing type：Research paper (scientific journal)  

Autonomous Robot誌において，レーザレンジファインダを搭載した複数台の移動ロボットにより，３次元大規模環境モデルを高精度で計測するシステムに関する学術論文を発表した．

Language：English  

For the construction of 3-D shape models of large-scale architectural structures using laser range finders, a number of range images are taken from different viewpoints around the targets. Next, the obtained images are normally aligned by post-processing procedures, such as the ICP algorithm. However, to obtain convergent results in the ICP algorithm and align these range images to their proper positions, the initial position of each range image needs to be manually aligned to roughly the correct position. This paper proposes a new measurement and modeling system using a group of multiple robots and an on-board laser range finder. Each measurement position is identified by a highly precise positioning technique called the Co-operative Positioning System (CPS), which utilizes the characteristics of the multiple-robot system. Therefore, the proposed system can construct 3-D shapes of large-scale architectural structures without any post-processing procedure or manual intervention. In addition, it is possible to register range images even if the number of measurements is few and there are only a few range images, for example, due to range images containing insufficient feature shapes or overlapping regions. Measurement experiments in unknown and large indoor/outdoor environments including a large hall, a building, an urban district, and a cultural heritage have been successfully carried out using the newly developed measurement system consisting of three mobile robots named CPS-V. Path generation experiments of the mobile robots based on the partially measured 3-D model are also presented. © 2011 Springer Science+Business Media, LLC.

DOI： 10.1007/s10514-011-9256-x

Language：English  

Language：Japanese   Publishing type：Research paper (scientific journal)  

Language：Japanese   Publishing type：Research paper (scientific journal)  

Language：English   Publishing type：Research paper (international conference proceedings)  

Language：English   Publishing type：Research paper (international conference proceedings)  

Language：English   Publishing type：Research paper (international conference proceedings)  

Language：English   Publishing type：Research paper (international conference proceedings)  

Language：English   Publishing type：Research paper (international conference proceedings)  

International Symposium on Robotics Researchにおいて，著者らが実施しているロボットタウンプロジェクトについて招待講演を行った．

Language：English   Publishing type：Research paper (international conference proceedings)  

Language：Japanese   Publishing type：Research paper (scientific journal)  

Language：English   Publishing type：Research paper (international conference proceedings)  

Language：English   Publishing type：Research paper (international conference proceedings)  

Language：English   Publishing type：Research paper (international conference proceedings)  

Language：English   Publishing type：Research paper (international conference proceedings)  

Language：English   Publishing type：Research paper (international conference proceedings)  

Language：English   Publishing type：Research paper (international conference proceedings)  

Language：English   Publishing type：Research paper (international conference proceedings)  

Language：English   Publishing type：Research paper (international conference proceedings)  

Language：English   Publishing type：Research paper (international conference proceedings)  

Language：Japanese   Publishing type：Research paper (scientific journal)  

Language：Japanese   Publishing type：Research paper (international conference proceedings)  

Language：English   Publishing type：Research paper (international conference proceedings)  

Language：English   Publishing type：Research paper (scientific journal)  

We describe a decision method for effective placement of tactile elements for grasp-type recognition. Not only does the placement decided by our method require a small number of tactile elements, it also achieves recognition performance as high as placements consisting of many elements. The placement decided by the method is evaluated through experiments involving the recognition of grasp type from the two types of grasp taxonomy defined by Cutkosky and Kamakura. The proposed method is extended by applying a decision tree pruning. The pruning is useful for reducing the number of selected tactile elements without badly dropping the recognition rate. © 2009 IEEE.

DOI： 10.1109/TMECH.2009.2023647

Language：Japanese   Publishing type：Research paper (scientific journal)  

Language：English   Publishing type：Research paper (scientific journal)  

Language：Japanese   Publishing type：Research paper (scientific journal)  

Language：English   Publishing type：Research paper (scientific journal)  

Language：English   Publishing type：Research paper (other academic)  

Fire brigades and special agencies are often demanded to operate for search and aid of human lives in extremely dangerous scenarios. It is very important to first verify the safety of the environment and to obtain remotely a clear image of the scenario inside buildings or underground spaces. Several studies have been addressing the possibility of using robotic tools to carry out safe operations. This contribution presents the development of the HELIOS team, consisting of five tracked robots for Urban Search And Rescue. Two units are equipped with manipulators for the accomplishment of particular tasks, such as the handling of objects and opening doors; the other three units, equipped with cameras and laser range finders, are utilized to create virtual 3D maps of the explored environment. The three units can move autonomously while collecting the data by using a collaborative positioning system (CPS). After an overview on the specifications of the team of robots and with respect to previous publications, detailed information about the improvements of the robot mechanical design and control systems are introduced. Tests of the CPS system and HELIOS IX vehicle together with a typical mission experiment are presented and discussed. © 2009 IEEE.

DOI： 10.1109/IROS.2009.5354452

Language：English   Publishing type：Research paper (international conference proceedings)  

Language：English   Publishing type：Research paper (international conference proceedings)  

Language：English   Publishing type：Research paper (international conference proceedings)  

Language：English   Publishing type：Research paper (scientific journal)  

Language：English   Publishing type：Research paper (international conference proceedings)  

Language：English   Publishing type：Research paper (international conference proceedings)  

Language：English   Publishing type：Research paper (international conference proceedings)  

Language：English   Publishing type：Research paper (international conference proceedings)  

Language：English   Publishing type：Research paper (international conference proceedings)  

Language：English  

Language：English   Publishing type：Research paper (international conference proceedings)  

Language：English   Publishing type：Research paper (scientific journal)  

In medical diagnostic imaging, the X-ray CT scanner and the MRI system have been widely used to examine 3D shapes and internal structures of living organisms and bones. However, these apparatuses are generally large and very expensive. Since an appointment is also required before examination, these systems are not suitable for urgent fracture diagnosis in emergency treatment. However, X-ray/fluoroscopy has been widely used as traditional medical diagnosis. Therefore, the realization of the reconstruction of precise 3D shapes of living organisms or bones from a few conventional 2D fluoroscopic images might be very useful in practice, in terms of cost, labor, and radiation exposure. The present paper proposes a method by which to estimate a patient-specific 3D shape of a femur from only two fluoroscopic images using a parametric femoral model. First, we develop a parametric femoral model by the statistical analysis of 3D femoral shapes created from CT images of 56 patients. Then, the position and shape parameters of the parametric model are estimated from two 2D fluoroscopic images using a distance map constructed by the Level Set Method. Experiments using synthesized images, fluoroscopic images of a phantom femur, and in vivo images for hip prosthesis patients are successfully carried out, and it is verified that the proposed system has practical applications. © 2008 Elsevier Inc. All rights reserved.

DOI： 10.1016/j.cviu.2008.08.012

Language：Japanese   Publishing type：Research paper (scientific journal)  

Language：Japanese   Publishing type：Research paper (scientific journal)  

Language：English   Publishing type：Research paper (scientific journal)  

Language：English   Publishing type：Research paper (international conference proceedings)  

Language：English   Publishing type：Research paper (international conference proceedings)  

Language：English   Publishing type：Research paper (international conference proceedings)  

Language：English   Publishing type：Research paper (international conference proceedings)  

Language：English   Publishing type：Research paper (international conference proceedings)  

Language：English   Publishing type：Research paper (international conference proceedings)  

Language：Japanese   Publishing type：Research paper (scientific journal)  

Language：English   Publishing type：Research paper (international conference proceedings)  

Language：English   Publishing type：Research paper (international conference proceedings)  

Language：English   Publishing type：Research paper (international conference proceedings)  

Language：Japanese   Publishing type：Research paper (scientific journal)  

Language：Japanese   Publishing type：Research paper (scientific journal)  

Language：English   Publishing type：Research paper (scientific journal)  

To create a textured virtual model of a real object, the 3D geometric model that is generally measured by a laser scanner must be aligned with the 2D image that is obtained from image sensors excluding some sensors for which geometric and photometric information are obtained at the same time. One conventional registration method that has been proposed compares the contour lines of the silhouette image of the 3D geometric model and of the 2D image and estimates the relative positions of both according to iterative calculations. However, a method that uses point correspondences of contour points in the 2D image and contour points of the silhouette image must find the nearest neighboring points for each iterative calculation, and the computational cost is high. Therefore, in this paper, the authors propose a new registration algorithm of grayscale images and 3D geometric models by using 2D distance maps that are constructed by using the Fast Marching Method from the boundary of an object on the grayscale image. They also show the effectiveness of the proposed method by performing simulation experiments and experiments using actual images. © 2007 Wiley Periodicals, Inc.

DOI： 10.1002/scj.20634

Language：English   Publishing type：Research paper (international conference proceedings)  

Language：English   Publishing type：Research paper (international conference proceedings)  

Language：English  

Language：English  

Language：English   Publishing type：Research paper (scientific journal)  

This paper presents an overview of our research project on digital preservation of cultural heritage objects and digital restoration of the original appearance of these objects. As an example of these objects, this project focuses on the preservation and restoration of the Great Buddhas. These are relatively large objects existing outdoors and providing various technical challenges. Geometric models of the great Buddhas are digitally achieved through a pipeline, consisting of acquiring data, aligning multiple range images, and merging these images. We have developed two alignment algorithms: a rapid simultaneous algorithm, based on graphics hardware, for quick data checking on site, and a parallel alignment algorithm, based on a PC cluster, for precise adjustment at the university. We have also designed a parallel voxel-based merging algorithm for connecting all aligned range images. On the geometric models created, we aligned texture images acquired from color cameras. We also developed two texture mapping methods. In an attempt to restore the original appearance of historical objects, we have synthesized several buildings and statues using scanned data and a literature survey with advice from experts. © 2007 Springer Science+Business Media, LLC.

DOI： 10.1007/s11263-007-0039-y

Language：Japanese   Publishing type：Research paper (international conference proceedings)  

Language：Japanese   Publishing type：Research paper (scientific journal)  

Language：Japanese   Publishing type：Research paper (scientific journal)  

Language：Japanese   Publishing type：Research paper (international conference proceedings)  

Language：English   Publishing type：Research paper (scientific journal)  

Language：English   Publishing type：Research paper (scientific journal)  

A computer's rendering capacity cannot handle the increased amount of data when rendering high-quality, three-dimensional computer graphic images. This results in problems such as rendering not being possible depending on the subject image, or the rendering speed dropping too far. To resolve these problems, we propose a method for generating a new hierarchical three-dimensional data structure based on recursive face cluster partitioning, and a high-speed rendering data extraction algorithm using that data structure. In the proposed algorithm, local resolution control using the approximating polygon of a face cluster, and precise invisible surface culling using the bounding volume of the face cluster can be executed in parallel. High-speed rendering can be performed with little deterioration in quality, even for a large amount of data. The effectiveness of this algorithm was confirmed with a local resolution control experiment and a large-scale three-dimensional model rendering experiment. © 2007 Wiley Periodicals, Inc.

DOI： 10.1002/scj.20581

Language：English   Publishing type：Research paper (scientific journal)  

This paper presents a new registration algorithm of two-dimensional (2-D) color images and 3-D geometric models for the navigation system of a surgical robot. A 2-D-3-D registration procedure is used to precisely superimpose a tumor model on an endoscopic image and is, therefore, indispensable for the surgical navigation system. Thus, the performance of the 2-D-3-D registration procedure influences directly the usability of the surgical robot operating system. One of the typical techniques that has been developed is the use of external markers. However, the accuracy of this method is reduced by the breathing or heartbeat of the patient, as well as other unknown factors. For precise registration of 3-D models and 2-D images without external markers or special measurement devices, a new registration method is proposed that utilizes the 2-D images and their distance maps as created by the Fast Marching Method. Here, we present results of fundamental experiments performed using simulated models and actual images of the endoscopic operation. © 2007 VSP.

DOI： 10.1163/156855307780429794

Language：English   Publishing type：Research paper (international conference proceedings)  

Language：English   Publishing type：Research paper (scientific journal)  

Language：Japanese   Publishing type：Research paper (scientific journal)  

n medical imaging, X-ray CT scanner or MRI system are quite useful to acquire 3D shapes of internal organs or bones.
However, these apparatuses are generally very expensive and of large size. They also need a prior arrangement, and thus, they are unsuitable for an urgent fracture diagnosis in emergency treatment.
This paper proposes a method to estimate a 3D shape of patient's femur from only two radiographs using a parametric femoral model. Firstly, we develop the parametric femoral model utilizing statistical procedure of 3D femoral models created by CT images of 51 patients. Then, the pose and shape parameters of the parametric model are estimated from two 2D images using a distance map constructed by the Level Set Method. Experiments using synthesized images and radiographs of a phantom femur are carried out to verify the performance of the proposed technique.

Language：Japanese   Publishing type：Research paper (scientific journal)  

Language：Japanese  

Language：English  

Language：Japanese  

Language：English   Publishing type：Research paper (international conference proceedings)  

Language：English   Publishing type：Research paper (international conference proceedings)  

Publishing type：Research paper (international conference proceedings)  

Language：English   Publishing type：Research paper (international conference proceedings)  

Repository Public URL： http://hdl.handle.net/2324/5866

Language：English  

Language：English   Publishing type：Research paper (scientific journal)  

Language：English  

Language：English   Publishing type：Research paper (scientific journal)  

Texture mapping on scanned objects, that is, the method of mapping current color images on a 3D geometric model measured by a range sensor, is a key technique of photometric modeling for virtual reality. Usually range and color images are obtained from different viewing positions, through two independent range and color sensors. Thus, in order to map those color images on the geometric model, it is necessary to determine relative relations between these two viewpoints. This paper focuses on the reflectance image of the laser scanner which is given as side products of range images for most of the range sensors, and proposes a new calibration method for the texture mapping using reflectance images and the iterative pose estimation using robust M-estimator. © 2005 Wiley Periodicals, Inc.

DOI： 10.1002/scj.10392

Language：English   Publishing type：Research paper (scientific journal)  

Language：Japanese   Publishing type：Research paper (scientific journal)  

Language：Japanese   Publishing type：Research paper (scientific journal)  

Language：Japanese   Publishing type：Research paper (scientific journal)  

Language：Japanese   Publishing type：Research paper (scientific journal)  

Publishing type：Research paper (scientific journal)  

Language：English   Publishing type：Research paper (scientific journal)  

DOI： 10.1109/IROS.2005.1545447

Language：Japanese   Publishing type：Research paper (scientific journal)  

Publishing type：Research paper (international conference proceedings)  

Language：English   Publishing type：Research paper (international conference proceedings)  

Language：Japanese   Publishing type：Research paper (scientific journal)  

Language：English   Publishing type：Research paper (international conference proceedings)  

DOI： 10.1109/3DIM.2005.39

Language：English   Publishing type：Research paper (international conference proceedings)  

Language：Japanese   Publishing type：Research paper (scientific journal)  

Language：Japanese   Publishing type：Research paper (scientific journal)  

Language：English   Publishing type：Research paper (international conference proceedings)  

The level set method (LSM) has been widely used for various applications such as motion tracking and 3D geometrical modeling. However, the calculation cost of reinitialization and updating of an implicit function is considerably expensive as compared with conventional active contour models such as "Snakes". To tackle this problem, we propose an efficient algorithm of the LSM named the Fast Level Set Method(FLSM). This paper introduces some experiments based on the FLSM, including 2D real-time tracking of moving objects in video images, and 3D simultaneous motion
capture system of multiple targets using stereo range images.

Language：English  

Language：English   Publishing type：Research paper (international conference proceedings)  

DOI： 10.1109/ICPR.2004.1334606

Language：English   Publishing type：Research paper (international conference proceedings)  

Language：English   Publishing type：Research paper (international conference proceedings)  

Language：English   Publishing type：Research paper (international conference proceedings)  

We propose a new index for dynamic performance analysis of serial link manipulators named Impedance Matching Ellipsoid, or IME. Several indexes have been proposed for indicating static and dynamic performance of robot manipulators. For example, Dynamic Manipulability Ellipsoid (DME) characterizes distributions of hand acceleration produced by normalized joint torque. Manipulating-Force Ellipsoid (MFE) denotes static torque-force transmission effciency from actuators at joints to a hand. On the other hand, the proposed IME
characterizes dynamic torque-force transmission effciency from actuators at joints to a load held at the hand of the manipulator. The IME includes a wide range of concepts proposed so far as measures of manipulator's performance. The DME and the MFE are both derived from the IME as limiting forms about the load mass. In this paper, we demonstrate the IME with some numerical examples ncluding the selection of an optimal leg posture for jump robots, optimum active stiffness control, and an extension for manipulators mounted on satellites in outer space.

DOI： 10.1109/ROBOT.2004.1302478

Language：Japanese   Publishing type：Research paper (scientific journal)  

In this decade, a number of various robot contests represented by "RoboCup" have been held in various parts of the world. In some of these contests, such as RoboCup middle-size robot league, fully automated mobile robots are used to compete for their mobility and autonomy. These robots have to plan optimum moving strategy automatically without any human intervention after the game starts. One of the fundamental and crucial faculties for these autonomous robots is the ability of accurate positioning of enemy and themselves. In this paper, a robust positioning system is proposed for soccer robots equipped with an omni-directional camera and wheels. The robot identifies its own position by observing some landmarks placed around the field through an omni-directional camera, and by counting rotation of wheels. To achieve the robust positioning, the proposed method utilizes (1) LMedS method for making correct correspondence between observed directions and landmarks, (2) elimination of miss-correspondence of landmarks using a median filter, (3) position calculation considering the properties of measurement error distributions using nonlinear maximum likelihood method, and (4) data fusion of positions obtained by observing landmarks and dead reckoning using kalman filter.

Language：English  

Language：English   Publishing type：Research paper (international conference proceedings)  

Though a legged robot has high terrain adaptability as compared with a wheeled vehicle, its moving speed is considerably low in general. For attaining a high moving speed with a legged robot, a dynamically stable gait, such as running for a biped robot and a trot gait or a bound gait for a quadruped robot, is a promising solution. However, the energy efficiency of the dynamically stable gait
is generally lower than the efficiency of the stable gait such as a crawl gait. In this paper, we present an experimental study on the energy efficiency of a quaduped walking vehicle. Energy consumption of two walking patterns for a trot
gait is investigated through experiments using a quadruped walking vehicle named TITAN-VIII. The obtained results show that the 3D sway compensation trajectory proposed in our previous paper [10] has advantages in view of energy efficiency as compared with the original sway compensation trajectory.

Language：English   Publishing type：Research paper (international conference proceedings)  

Language：Japanese   Publishing type：Research paper (scientific journal)  

The level set method, introduced by S. Osher and J. A. Sethian, has attracted much attention as a topological-free active contour model. This method utilizes an implicit representation of a contour to be tracked, and is able to handle the topological change of the contour naturally. Various applications based on the level set method have been presented including motion tracking, 3D geometrical modelling, and simulation of crystallization or semiconductor growth. However, the calculation cost of reinitialization and updating of the implicit function is considerably expensive as compared with the cost of conventional active contour models such as ``Snakes''. In this paper, we propose an efficient calculation algorithm for the level set method named the Fast Level Set Method (FLSM).
Advantages of the proposed FLSM are as follows: i) the use of the extension velocity and the high speed construction of the extension velocity field using the Fast Narrow Band Method, ii) the frequent execution of the reinitializatin process of the implicit function which requires little calculation cost.
The efficiency of the proposed method is verified through computer simulations, and experiments of realtime tracking of moving objects in video images.

Language：English   Publishing type：Research paper (international conference proceedings)  

From 1970's, legged robots have attracted much attention of many researchers. In spite of this, it has been regarded that dynamically stable walking is very difficult to be tackled for any types of legged robots. For a trot gait for quadruped walking robots, we have proposed "the sway compensation trajectory". This method utilizes a lateral, longitudinal, and vertical motion of a robot body to keep a zero moment point (ZMP) on a diagonal line between support legs. In this paper, we develop the sway compensation trajectory for a biped robot, and show that dynamically stable walking is realized. This method makes it quite easy to design stable ZMP and COG (center of gravity) trajectories, which have been regarded as a very complicated and delicate problem. The effectiveness of the proposed method is verified through computer simulations and walking experiments by a humanoid robot, HOAP-1.

Language：Japanese   Publishing type：Research paper (scientific journal)  

Language：Japanese   Publishing type：Research paper (scientific journal)  

From 1970's, legged robots have attracted much attention of many researchers. In spite of this, it has been regarded that dynamically stable walking is very difficult to be tackled for any types of legged robots. For a trot gait for quadruped walking robots, we have proposed ``the sway compensation trajectory''. This method utilizes a lateral, longitudinal, and vertical motion of a robot body to keep a zero moment point (ZMP) on a diagonal line between support legs. In this paper, we develop the sway compensation trajectory for a biped robot, and show that dynamically stable walking is realized. This method makes it quite easy to design stable ZMP and COG (center of gravity) trajectories, which have been regarded as a very complicated and delicate problem. The effectiveness of the proposed method is verified through computer simulations and walking experiments by a humanoid robot HOAP-1, and YANBO-3.

Language：Japanese   Publishing type：Research paper (scientific journal)  

Language：English   Publishing type：Research paper (international conference proceedings)  

Language：English   Publishing type：Research paper (international conference proceedings)  

Texture mapping on scanned objects, that is, the method to map current color images on a 3D geometric model measured by a range sensor, is a key technique of photometric modeling for virtual reality. Usually range and color images are obtained from different viewing positions, through two independent range and color sensors. Thus, in order to map those color images on the geometric model, it is necessary to determine relative relations between these two viewpoints. In this paper, we propose a new calibration method for the texture mapping; the method utilizes reflectance images and iterative pose estimation based on a robust Mestimator. Moreover, since a 2D texture image taken from one viewing point is a partial view of an object, several images must be mapped onto the object in order to cover the entire 3D geometric model. In this paper, we propose the new simultaneous registration technique of several images and geometric model based on 2D-3D edge correspondence and the epipolar constraint between images.

Language：English   Publishing type：Research paper (scientific journal)  

Language：English  

Language：Japanese   Publishing type：Research paper (scientific journal)  

Language：Japanese   Publishing type：Research paper (scientific journal)  

Language：English   Publishing type：Research paper (international conference proceedings)  

Language：English   Publishing type：Research paper (international conference proceedings)  

To realize dynamically stable walking for a quadruped walking robot, the combination of the trajectory planning of the body and leg position (feedforward control) and the adaptive attitude control using sensory information (feedback control) is indispensable. In this paper, we initially propose a new trajectory planning for the stable trot gait named 3D sway compensation trajectory, and show that this trajectory has lower energy consumption than the conventional sway trajectory that the authors have proposed. Next, as the adaptive attitude control method used during the 2-leg supporting phase of the trot gait, we consider four methods: a) rotation of body along the diagonal line between supporting feet, b) translation of body along the perpendicular line between supporting feet, c) vertical swing motion of recovering legs, and d) horizontal swing motion of recovering legs. The stabilization efficiency of each method is verified through computer simulation and the damping experiment using a quadruped walking robot, TITAN-VIII. Furthermore, the dynamic trot gait control that combines the feedforward control based on the proposed 3D sway compensation trajectory and the adaptive feedback control using body translation and vertical motion of swing legs is developed, and the walking experiment on rough terrain using TITAN-VIII is carried out.

Language：English   Publishing type：Research paper (international conference proceedings)  

Texture mapping on scanned objects, that is, the method to map current color images on a 3D geometric model measured by a range sensor, is a key technique of photometric modeling for virtual reality. Usually range and color images are obtained from different viewing positions, through two independent range and color sensors. Thus, in order to map those color images on the geometric model, it is necessary to determine relative relations between these two viewpoints. In this paper, we propose a new calibration method for the texture mapping; the method utilizes reflectance images and iterative pose estimation based on a robust M-estimator.

Language：Japanese   Publishing type：Research paper (scientific journal)  

Language：Japanese   Publishing type：Research paper (scientific journal)  

Language：English   Publishing type：Research paper (international conference proceedings)  

Walking robots have high adaptability for terrain variation, and thus, have been expected as effective moving platform on uneven terrain, stairs, forest, marshy surface, and on ice. On the other hand, mobile robots that perform several hazardous tasks such as mine detection or the inspection of an atomic power plant are typically controlled by operators from distant places. For a remote operation system, use of visual information from a camera mounted on a robot body is very useful. However, unlike wheeled vehicles, the camera mounted on the walking robot oscillates because of the impact of walking, and the obtained unstable images cause inferior operation performance. In this paper, we introduce an image stabilization system for remote operation of walking robots using a high speed CCD camera and gyrosensors. The image stabilization is executed in two phases, that is, the estimation of the amount of oscillation by the combination of the template matching method and gyrosensors, and change of the display region. Pentium MMX instruction is used for template matching calculation, and the estimated amount of oscillation is outputted in every 12 [msec.]. Furthermore, developed image stabilization mechanism can be used an external attitude sensor from the visual information, and the damping control of the robot body while walking is also possible. Experimental results showed stabilized images that eliminates the oscillation component are taken even when the robot moves dynamically or in long distance, and verified that the performance of attitude control using the developed image stabilization system is almost same as the case using an attitude sensor.

Language：English   Publishing type：Research paper (scientific journal)  

Language：English   Publishing type：Research paper (scientific journal)  

Several position identification methods are being used for mobile robots. Dead reckoning is a popular method but due to the error accumulation from wheel slippage, its reliability is low for measurement of long distances especially on uneven surfaces. Another popular method is the landmark method, which estimates current position relative to known landmarks, but the landmark method's limitation is that it cannot be used in an uncharted environment. Thus, this paper proposes a new method called "Cooperative Positioning System (CPS)" that is able to overcome these shortcomings. The main concept of CPS is to divide the robots into two groups, A and B where group A remains stationary and acts as a landmark while group B moves and then group B stops and acts as a landmark for group A. This process is repeated until the target position is reached. Compared with dead reckoning, CPS has a far lower accumulation of positioning errors, and can also work in three dimensions. Furthermore, CPS employs inherent landmarks and therefore can be used in uncharted environments unlike the landmark method. In this paper, we introduce the basic concept of CPS and its positioning principle. Next, we outline a second prototype CPS machine model (CPS-II) and discuss the method of position estimation using the variance of positioning error and weighted least squares method. Position identification experiments using the CPS-II model give a positioning accuracy of 0.12&#37; for position and 0.32 degree for attitude after the robots traveled a distance of 21.5 m. © 2000 Kluwer Academic Publishers.

DOI： 10.1023/A:1008988801987

Language：English   Publishing type：Research paper (scientific journal)  

Language：Japanese   Publishing type：Research paper (scientific journal)  

Language：Japanese   Publishing type：Research paper (scientific journal)  

Language：English   Publishing type：Research paper (international conference proceedings)  

Several position identification methods are being used for mobile robots. Dead reckoning is a popular method but due to the accumulation error from wheel slippage, reliability is low for the measurement of long distances especially on uneven surfaces. Another popular method is the landmark method, which estimates current position relative to known landmarks, but the landmark method's limitation is that it cannot be used in an uncharted environment. Thus, this paper proposes a new method called "Cooperative Positioning System (CPS)" that is able to overcome these shortcomings. The main concept of CPS is to divide the robots into two groups, A and B respectively, group A remains stationary and acts as a landmark while group B moves and then group B stops and acts as a landmark for group A. This process is repeated until the target position is reached. Compared with dead reckoning, CPS has a far lower accumulation of positioning error, and can also work in three-dimensions. Furthermore, CPS employs inherent landmarks and therefore can be used in uncharted environments unlike the landmark method. In this paper, focus will be on the discussion of the relationship between moving configurations of CPS and its positioning accuracy for the latest prototype CPS model, CPS-III, using simulation and analytical techniques. Optimum moving strategies in order to minimize positioning error are then discussed and verified through experiments.

Language：Japanese   Publishing type：Research paper (scientific journal)  

Language：Japanese   Publishing type：Research paper (scientific journal)  

Language：English   Publishing type：Research paper (international conference proceedings)  

Several position identification methods have been used for mobile robots. Dead reckoning is a popular method, but is not reliable for long distances or uneven surfaces because of variations in wheel diameter and slippage. The landmark method, which estimates current position relative to landmarks, cannot be used in an uncharted environment. We have proposed a new method called "Cooperative Positioning System (CPS)." For CPS, we divide the robots into two groups, A and B. One group, A, remains stationary and acts as a landmark while group B moves. Group B then stops and acts as a landmark for group A. This "dance" is repeated until the target position is reached. By using the concept of "portable landmarks", CPS has a far lower accumulation of positioning error than dead reckoning, and can work in three-dimensions which is not possible with dead reckoning. CPS, therefore, can work in uncharted environments. In this paper, we outline a second prototype CPS machine model (CPS-II) and report the results of position identification experiments. Experimental results using this model give a positioning accuracy of 0.4% for position and 1.0 degree for attitude.

Language：Japanese   Publishing type：Research paper (scientific journal)  

Language：Japanese   Publishing type：Research paper (scientific journal)  

Language：English   Publishing type：Research paper (international conference proceedings)  

Several position identification methods are being used for mobile robots. Dead reckoning is a popular method but due to the accumulation error from wheel slippage, reliability is low for the measurement of long distances especially on uneven surfaces. Another popular method is the landmark method, which estimates current position relative to known landmarks, but the landmark method's limitation is that it cannot be used in an uncharted environment. Thus, this paper proposes a new method called "Cooperative Positioning System (CPS)" that is able to overcome these shortcomings. The main concept of CPS is to divide the robots into two groups, A and B respectively, group A remains stationary and acts as a landmark while group B moves and then group B stops and acts as a landmark for group A. This process is repeated until the target position is reached. Compared with dead reckoning, CPS has a far lower accumulation of positioning error, and can also work in three-dimensions. Furthermore, CPS has inherent landmarks and therefore can be used in uncharted environments unlike the landmark method. In this paper, focus will be on the discussion of the relationship between moving configurations of CPS and its positioning accuracy for the latest prototype CPS model, CPS-III, using simulation and analytical techniques. Optimum moving strategies in order to minimize positioning error are then discussed and verified through experiments.

Language：English   Publishing type：Research paper (international conference proceedings)  

,

Language：Others   Publishing type：Research paper (scientific journal)  

This paper discusses the stabilization of an inverted pendulum by a neural network. The inverted pendulum is a typical example of an unstable and nonlinear system. Fujitsu has developed a self-learning method that enables a neural network to control an inverted pendulum by using heuristic rules. A system based on this method has been developed which generates teaching data for inversion by trial and error. Then, the neural network learns the teaching data, and acquires the control rules after a period of trial and error. The method proved its effectiveness by stably inverting a pendulum in an experimental inverted pendulum system after a period of trial and error.

Publishing type：Research paper (international conference proceedings)  

Language：English   Publishing type：Research paper (other academic)  

The authors present a basic formulation of motion dynamics of a free-floating rigid-link system to establish a basis of the collision dynamics. They propose a novel concept named extended generalized inertia tensor (Ex-GIT), which is an extended version of the GIT for ground-based arms, and discuss the virtual mass concept. By means of these concepts, they formulate the collision problem focusing on a velocity relationship just before and after the collision without sensing the impact force, but considering the momentum conservation law.

DOI： 10.1109/ROBOT.1992.220182

Language：Japanese   Publishing type：Research paper (scientific journal)  

Language：Japanese   Publishing type：Research paper (scientific journal)  

Language：English   Publishing type：Research paper (international conference proceedings)  

Language：English   Publishing type：Research paper (international conference proceedings)  

Language：Others  

Institute of Industrial Science,
This paper describes a method for parallel alignment of multiple range images. There are problems of computational time and memory space in aligning a large number of range images simultaneously. We developed a parallel method to address the problems. Searching for corresponding points between two range images is time-consuming and requires considerable memory space when performed independently. However, this process can be preformed in parallel, with each corresponding pair of range images assigned to a node. Because the computation time is approximately proportional to the number of vertices, by assigning the pairs so that the number of vertices computed is equal on each node, the load on each node is effectively distributed. In order to reduce the amount of memory required on each node, a hypergraph that represents the correspondences of range images is created, and heuristic graph partitioning algorithms are applied to determine the optimal assignment of the pairs. Moreover, by rejecting redundant dependencies, it becomes possible to accelerate computation time and reduce the amount of memory required on each node. The method was tested on a 16-processor PC cluster, where it demonstrated high extendibility and improved performance. 1.

Language：Others  

Simultaneous 2D images and 3D geometric model registration for texture mapping utilizing reflectance attribute
Texture mapping on scanned objects, that is, the method to map current color images on a 3D geometric model measured by a range sensor, is a key technique of photometric modeling for virtual reality. Usually range and color images are obtained from different viewing positions, through two independent range and color sensors. Thus, in order to map those color images on the geometric model, it is necessary to determine relative relations between these two viewpoints. In this paper, we propose a new calibration method for the texture mapping; the method utilizes reflectance images and iterative pose estimation based on a robust Mestimator. Moreover, since a 2D texture image taken from one viewing point is a partial view of an object, several images must be mapped onto the object in order to cover the entire 3D geometric model. In this paper, we propose the new simultaneous registration technique of several images and geometric model based on 2D-3D edge correspondence and the epipolar constraint between images. 1

Language：Japanese   Book type：General book, introductory book for general audience

Language：Japanese   Book type：General book, introductory book for general audience

Language：Japanese   Book type：General book, introductory book for general audience

Language：Japanese   Book type：General book, introductory book for general audience