Publishing type：Research paper (scientific journal)   Publisher：IEEE Access  

Gait recognition enables the non-contact identification of individuals from a distance based on their walking patterns and body shapes. For vision-based gait recognition, covariates (e.g., clothing, baggage and background) can negatively impact identification. As a result, many existing studies extract gait features from silhouettes or skeletal information obtained through preprocessing, rather than directly from RGB image sequences. In contrast to preprocessing which relies on the fitting accuracy of models trained on different tasks, disentangled representation learning (DRL) is drawing attention as a method for directly extracting gait features from RGB image sequences. However, DRL learns to extract features of the target attribute from the differences among multiple inputs with various attributes, which means its separation performance depends on the variation and amount of the training data. In this study, aiming to enhance the variation and quantity of each subject's videos, we propose a novel data augmentation pipeline by feature swapping for RGB-based gait recognition. To expand the variety of training data, features of posture and covariates separated through DRL are paired with features extracted from different individuals, which enables the generation of images of subjects with new attributes. Dynamic gait features are extracted through temporal modeling from pose features of each frame, not only from real images but also from generated ones. The experiments demonstrate that the proposed pipeline increases both the quality of generated images and the identification accuracy. The proposed method also outperforms the RGB-based state-of-the-art method in most settings.

DOI： 10.1109/ACCESS.2024.3445415

Web of Science

Scopus

researchmap

Publishing type：Research paper (international conference proceedings)   Publisher：IEEE  

Most existing retrofit remote-control systems for backhoes are large, hard-to-install, and expensive. Therefore, we propose an easy-to-install and inexpensive teleoperation system. The proposed system comprised remote-control and sensing systems. The remote-control system retrofits robot arm-based devices to 'Cat Command', a compact embedded teleoperation system with a limited communication range, and controls these devices via a 5G commercial network to realize control from a remote office. Because this system does not require any additional modifications to the embedded control unit in the cockpit, the operator can continue working in the cockpit even if the backhoe is remotely controlled. The system enables the remote control of various devices from an extremely long distance by changing the joint parts between the robot arm and the embedded remote-control device. The sensing system estimates the posture and position of the backhoe by attaching original sensing devices to the backhoe. In addition, a 360 camera was installed in the cockpit to transmit work images from the construction site to a remote office in real time. The sensing device was smaller and lighter than conventional devices. We confirmed that the proposed system can be used to operate a construction site backhoe from a remote office, and that the system can be used to excavate soil using an actual backhoe.

DOI： 10.1109/SII58957.2024.10417625

Scopus

researchmap

Other Link： https://dblp.uni-trier.de/db/conf/sii/sii2024.html#ShibataNTMNK24

Publishing type：Research paper (international conference proceedings)   Publisher：IEEE  

Generative modeling of 3D LiDAR data is an emerging task with promising applications for autonomous mobile robots, such as scalable simulation, scene manipulation, and sparse-to-dense completion of LiDAR point clouds. While existing approaches have demonstrated the feasibility of image-based LiDAR data generation using deep generative models, they still struggle with fidelity and training stability. In this work, we present R2DM, a novel generative model for LiDAR data that can generate diverse and high-fidelity 3D scene point clouds based on the image representation of range and reflectance intensity. Our method is built upon denoising diffusion probabilistic models (DDPMs), which have shown impressive results among generative model frameworks in recent years. To effectively train DDPMs in the LiDAR domain, we first conduct an in-depth analysis of data representation, loss functions, and spatial inductive biases. Leveraging our R2DM model, we also introduce a flexible LiDAR completion pipeline based on the powerful capabilities of DDPMs. We demonstrate that our method surpasses existing methods in generating tasks on the KITTI-360 and KITTI-Raw datasets, as well as in the completion task on the KITTI-360 dataset. Our project page can be found at https://kazuto1011.github.io/r2dm.

DOI： 10.1109/ICRA57147.2024.10611480

Web of Science

Scopus

researchmap

Other Link： https://dblp.uni-trier.de/db/conf/icra/icra2024.html#NakashimaK24

Building LiDAR generative models holds promise as powerful data priors for
restoration, scene manipulation, and scalable simulation in autonomous mobile
robots. In recent years, approaches using diffusion models have emerged,
significantly improving training stability and generation quality. Despite
their success, diffusion models require numerous iterations of running neural
networks to generate high-quality samples, making the increasing computational
cost a potential barrier for robotics applications. To address this challenge,
this paper presents R2Flow, a fast and high-fidelity generative model for LiDAR
data. Our method is based on rectified flows that learn straight trajectories,
simulating data generation with significantly fewer sampling steps compared to
diffusion models. We also propose an efficient Transformer-based model
architecture for processing the image representation of LiDAR range and
reflectance measurements. Our experiments on unconditional LiDAR data
generation using the KITTI-360 dataset demonstrate the effectiveness of our
approach in terms of both efficiency and quality.

DOI： 10.48550/arXiv.2412.02241

arXiv

researchmap

Publishing type：Research paper (international conference proceedings)   Publisher：IEEE  

People depend on medical equipment to support their movements when their motor function declines. Our previous study developed a method to estimate motor function from the force applied to a vertical handrail while standing. However, the effect of the handrail direction on movement remains unclear. Additionally, the force applied to the handrail and floor reaction forces on the buttocks and feet may also change with a decline in motor function. Here, this study constructed a system with force plates and handles in both the horizontal and vertical directions to measure the forces applied to the handrails, buttocks, and feet. Furthermore, the change in accuracy of the estimation of motor function, depending on the direction of the handrails and input information, was investigated. In the experiment, healthy participants stood up using a handrail with unrestricted movement and while wearing elderly experience kits that artificially impaired their motor function. The results showed that people exert more downward force on horizontal handrails than on vertical handrails. However, people rely on the vertical handrail for a longer period of time to stabilize anterior-posterior movement. These results indicate that different directions of handrails cause different strategies of the standing-up motion. Additionally, the accuracy of the estimation of motor function improved when the horizontal handrail was used rather than the vertical handrail. This suggests that the classification accuracy could be improved by using different handrail directions, depending on the subject's condition and standing-up motion.

DOI： 10.1109/SII55687.2023.10039452

Web of Science

Scopus

researchmap

Other Link： https://dblp.uni-trier.de/db/conf/sii/sii2023.html#KiharaATINK23

Publisher：Proceedings of the International Symposium on Automation and Robotics in Construction  

Soil compaction is one of the most important basic elements in construction work because it directly affects the quality of structures. Compaction work using vibratory rollers is generally applied to strengthen ground stiffness, and the method that focuses on the number of compaction cycles is widely used to manage the ground stiffness by vibratory rollers. In contrast to this method, the continuous compaction control (CCC) using accelerometers installed on the vibratory rollers has been proposed as a quantitative evaluation method more suited to actual ground conditions. This method quantifies the distortion rate of the acceleration waveform of the vibratory roller. However, this method based on acceleration response has problems in measurement discrimination accuracy and sensor durability because the accelerometer is installed on the vibration roller, which is the source of vibration. In this paper, we propose a new ground stiffness evaluation method using multiple accelerometers installed on the ground surface. The proposed method measures the acceleration response during compaction work by vibratory rollers using multiple accelerometers on the ground surface. Experiments show the proposed method has a higher discrimination than the conventional methods.

DOI： 10.22260/ISARC2023/0037

Scopus

Publishing type：Research paper (international conference proceedings)   Publisher：IEEE  

3D LiDAR sensors are indispensable for the robust vision of autonomous mobile robots. However, deploying LiDAR-based perception algorithms often fails due to a domain gap from the training environment, such as inconsistent angular resolution and missing properties. Existing studies have tackled the issue by learning inter-domain mapping, while the transferability is constrained by the training configuration and the training is susceptible to peculiar lossy noises called ray-drop. To address the issue, this paper proposes a generative model of LiDAR range images applicable to the data-level domain transfer. Motivated by the fact that LiDAR measurement is based on point-by-point range imaging, we train an implicit image representation-based generative adversarial networks along with a differentiable ray-drop effect. We demonstrate the fidelity and diversity of our model in comparison with the point-based and image-based state-of-the-art generative models. We also showcase upsampling and restoration applications. Furthermore, we introduce a Sim2Real application for LiDAR semantic segmentation. We demonstrate that our method is effective as a realistic ray-drop simulator and outperforms state-of-the-art methods.

DOI： 10.1109/WACV56688.2023.00131

Web of Science

Scopus

researchmap

Other Link： https://dblp.uni-trier.de/db/conf/wacv/wacv2023.html#NakashimaIK23

Publishing type：Research paper (scientific journal)   Publisher：IEEE Access  

Gait recognition is a biometric identification method based on individual walking patterns. This modality is applied in a wide range of applications, such as criminal investigations and identification systems, since it can be performed at a long distance and requires no cooperation of interests. In general, cameras are used for gait recognition systems, and previous studies have utilized depth information captured by RGB-D cameras, such as Microsoft Kinect. In recent years, multi-layer LiDAR sensors, which can obtain range images of a target at a range of over 100 m in real time, have attracted significant attention in the field of autonomous mobile robots and self-driving vehicles. Compared with general cameras, LiDAR sensors have rarely been used for biometrics due to the low point cloud densities captured at long distances. In this study, we focus on improving the robustness of gait recognition using LiDAR sensors under confounding conditions, specifically addressing the challenges posed by viewing angles and measurement distances. First, our recognition model employs a two-scale spatial resolution to enhance immunity to varying point cloud densities. In addition, this method learns the gait features from two invariant viewpoints (i.e., left-side and back views) generated by estimating the walking direction. Furthermore, we propose a novel attention block that adaptively recalibrates channel-wise weights to fuse the features from the aforementioned resolutions and viewpoints. Comprehensive experiments conducted on our dataset demonstrate that our model outperforms existing methods, particularly in cross-view, cross-distance challenges, and practical scenarios.

DOI： 10.1109/ACCESS.2023.3333037

Web of Science

Scopus

researchmap

Language：English   Publisher：Institute of Electrical and Electronics Engineers (IEEE)  

3D LiDAR sensors are indispensable for the robust vision of autonomous mobile robots. However, deploying LiDAR-based perception algorithms often fails due to a domain gap from the training environment, such as inconsistent angular resolution and missing properties. Existing studies have tackled the issue by learning inter-domain mapping, while the transferability is constrained by the training configuration and the training is susceptible to peculiar lossy noises called ray-drop. To address the issue, this paper proposes a generative model of LiDAR range images applicable to the data-level domain transfer. Motivated by the fact that LiDAR measurement is based on point-by-point range imaging, we train an implicit image representation-based generative adversarial networks along with a differentiable ray-drop effect. We demonstrate the fidelity and diversity of our model in comparison with the point-based and image-based state-of-the-art generative models. We also showcase upsampling and restoration applications. Furthermore, we introduce a Sim2Real application for LiDAR semantic segmentation. We demonstrate that our method is effective as a realistic ray-drop simulator and outperforms state-of-the-art methods.

CiNii Research

Language：Japanese   Publisher：The Japan Society of Mechanical Engineers  

<p>In this study, we develop a sensor terminal with multiple and various sensors named sensor pod, which collects various environmental information at a construction site. The sensor pod is equipped with a 3D-LiDAR and a vibration sensor, which can be used to predict the surrounding hazards and evaluate the ground stiffness. In this paper, we introduce a method of evaluating ground stiffness using the waveform distortion of multi-point synchronized vibration data obtained by the distributed sensor pods.</p>

DOI： 10.1299/jsmermd.2022.1a1-e04

CiNii Research

Publishing type：Research paper (international conference proceedings)   Publisher：IEEE  

Gait recognition, which is a biometric identifier for individual walking patterns, is utilized in many applications, such as criminal investigation and identification systems, because it can be applied at a long distance and requires no explicit cooperation of the subjects. In general, cameras are used for gait recognition, and several methods in previous studies have used depth information captured by RGB-D cameras. However, RGB-D cameras are limited in terms of their measurement distance and are difficult to access outdoors. In recent years, real-time multi-layer 3D LiDAR, which can obtain 3D range images of a target at ranges of over 100 m, has attracted significant attention for use in autonomous mobile robots, serving as eyes for obstacles detection and navigation. Compared with cameras, such 3D LiDAR has rarely been used for biometrics owing to its low spatial resolution. However, considering the unique characteristics of 3D LiDAR, such as the robustness of the illumination conditions, long measurement distances, and wide-angle scanning, the approach has the potential to be applied outdoors as a biometric identifier. The present paper describes a gait recognition system, called 2V-Gait, which is robust to variations in the walking direction of a subject and the distance measured from the 3D LiDAR. To improve the performance of gait recognition, we leverage the unique characteristics of 3D LiDAR, which are not included in regular cameras. Extensive experiments on our dataset show the effectiveness of the proposed approach.

DOI： 10.1109/SII52469.2022.9708899

Web of Science

Scopus

researchmap

Other Link： https://dblp.uni-trier.de/db/conf/sii/sii2022.html#AhnNYIK22

Publishing type：Research paper (international conference proceedings)   Publisher：IEEE  

Gait recognition is a non-contact person identification method that utilizes cameras installed at a distance. However, gait images contain person-agnostic elements (covariates) such as clothing, and the removal of covariates is important for identification with high performance. Disentanglement representation learning, which separates gait-dependent information such as posture from covariates by unsupervised learning, has been attracting attention as a method to remove covariates. However, because the amount of gait data is negligible compared to other computer vision tasks, such as image recognition, the separation performance of existing methods is insufficient. In this study, we propose a gait recognition method to improve the separation performance, which augments the training data by adversarial generation based on identity features, separated by disentanglement representation learning. The proposed method first separates gait-dependent features (pose features) and appearance-related covariate features (style features) from gait videos based on disentanglement representation learning. Then, synthesized gait images are generated by exchanging pose features between gait images of the person under different walking conditions, followed by adding them to the training data. The experiments indicate that our method can improve the separation performance, and generate high-quality gait images that are effective for data augmentation.

DOI： 10.1109/SII52469.2022.9708776

Web of Science

Scopus

researchmap

Other Link： https://dblp.uni-trier.de/db/conf/sii/sii2022.html#YoshinoNAIK22

Publishing type：Research paper (international conference proceedings)   Publisher：IEEE  

Assisting patients with dementia is a significant social issue. Currently, to assist patients with dementia, a multimodal care technique called Humanitude is gaining popularity. In Humanitude, the patients are assisted through various techniques to stand up independently by utilizing their motor functions as much as possible. Humanitude care techniques encourage caregivers to increase the area of contact with patients during the sit-to-stand motion. However, Humanitude care techniques are not accurately performed by novice caregivers. Therefore, in this study, a smock-type wearable sensor was developed to measure the proximity between caregivers and care recipients during sit-to-stand motion assistance. A measurement experiment was conducted to evaluate the proximity differences between Humanitude care and simulated novice care. In addition, the effects of different care techniques on the center of mass (CoM) trajectory and muscle activity of the care recipients were investigated. The results showed that the caregivers tend to bring their top and middle trunk closer in Humanitude care compared with novice simulated care. Furthermore, it was observed that the CoM trajectory and muscle activity under Humanitude care were similar to those observed when the care recipient stands up independently. These results validate the effectiveness of Humanitude care and provide useful information for teaching techniques in Humanitude.

DOI： 10.1109/SMC53654.2022.9945156

Scopus

researchmap

Other Link： https://dblp.uni-trier.de/db/conf/smc/smc2022.html#AnTNSSK22

Language：Japanese   Publisher：The Japan Society of Mechanical Engineers  

<p>This paper presents a retrofit backhoe remote control system that is inexpensive, compact, and easy to install. The system consists of a remote control system using a teleoperation system embedded by a construction machinery manufacturer and a small robot arm, and a remote sensing system using a multi-core microcomputer.</p>

DOI： 10.1299/jsmermd.2022.1p1-c07

CiNii Research

Language：English   Presentation type：Oral presentation (general)  

Country：Japan  

Language：English   Presentation type：Oral presentation (general)  

Country：Singapore  

Language：English   Presentation type：Oral presentation (general)  

Country：Canada  

Language：English   Presentation type：Oral presentation (general)  

Country：Taiwan, Province of China  

Language：English   Presentation type：Oral presentation (general)  

Country：United States  

Language：English   Presentation type：Oral presentation (general)  

Country：Japan  

Language：English   Presentation type：Oral presentation (general)  

Country：United States  

Language：English   Presentation type：Oral presentation (general)  

Country：United States  

Language：English   Presentation type：Oral presentation (general)  

Country：Norway  

Language：English   Presentation type：Oral presentation (general)  

Country：Norway  

Language：English   Presentation type：Oral presentation (general)  

Country：Czech Republic  

Language：English   Presentation type：Oral presentation (general)  

Country：United States  

Language：English   Presentation type：Oral presentation (general)  

Country：India  

Language：English   Presentation type：Oral presentation (invited, special)  

Country：Japan  

Language：English   Presentation type：Oral presentation (general)  

Country：Viet Nam  

Generative modeling of 3D LiDAR data is an emerging task with promising
applications for autonomous mobile robots, such as scalable simulation, scene
manipulation, and sparse-to-dense completion of LiDAR point clouds. Existing
approaches have shown the feasibility of image-based LiDAR data generation
using deep generative models while still struggling with the fidelity of
generated data and training instability. In this work, we present R2DM, a novel
generative model for LiDAR data that can generate diverse and high-fidelity 3D
scene point clouds based on the image representation of range and reflectance
intensity. Our method is based on the denoising diffusion probabilistic models
(DDPMs), which have demonstrated impressive results among generative model
frameworks and have been significantly progressing in recent years. To
effectively train DDPMs on the LiDAR domain, we first conduct an in-depth
analysis regarding data representation, training objective, and spatial
inductive bias. Based on our designed model R2DM, we also introduce a flexible
LiDAR completion pipeline using the powerful properties of DDPMs. We
demonstrate that our method outperforms the baselines on the generation task of
KITTI-360 and KITTI-Raw datasets and the upsampling task of KITTI-360 datasets.
Our code and pre-trained weights will be available at
https://github.com/kazuto1011/r2dm.

DOI： 10.48550/arXiv.2309.09256

arXiv

researchmap

Other Link： http://arxiv.org/pdf/2309.09256v1

J-GLOBAL

researchmap

J-GLOBAL

researchmap

J-GLOBAL

researchmap

J-GLOBAL

researchmap

J-GLOBAL

researchmap

J-GLOBAL

researchmap

3D LiDAR sensors are indispensable for the robust vision of autonomous mobile
robots. However, deploying LiDAR-based perception algorithms often fails due to
a domain gap from the training environment, such as inconsistent angular
resolution and missing properties. Existing studies have tackled the issue by
learning inter-domain mapping, while the transferability is constrained by the
training configuration and the training is susceptible to peculiar lossy noises
called ray-drop. To address the issue, this paper proposes a generative model
of LiDAR range images applicable to the data-level domain transfer. Motivated
by the fact that LiDAR measurement is based on point-by-point range imaging, we
train an implicit image representation-based generative adversarial networks
along with a differentiable ray-drop effect. We demonstrate the fidelity and
diversity of our model in comparison with the point-based and image-based
state-of-the-art generative models. We also showcase upsampling and restoration
applications. Furthermore, we introduce a Sim2Real application for LiDAR
semantic segmentation. We demonstrate that our method is effective as a
realistic ray-drop simulator and outperforms state-of-the-art methods.

DOI： 10.48550/arXiv.2210.11750

arXiv

researchmap

Other Link： http://arxiv.org/pdf/2210.11750v1

J-GLOBAL

researchmap

J-GLOBAL

researchmap

J-GLOBAL

researchmap

J-GLOBAL

researchmap

J-GLOBAL

researchmap

J-GLOBAL

researchmap

J-GLOBAL

researchmap

Type：Peer review 

Proceedings of International Conference Number of peer-reviewed papers：1