Language：Others   Publishing type：Research paper (scientific journal)   Publisher：IEEE Transactions on Visualization and Computer Graphics  

As one of the facial expression recognition techniques for Head-Mounted Display (HMD) users, embedded photo-reflective sensors have been used. In this paper, we investigate how gaze and face directions affect facial expression recognition using the embedded photo-reflective sensors. First, we collected a dataset of five facial expressions (Neutral, Happy, Angry, Sad, Surprised) while looking in diverse directions by moving 1) the eyes and 2) the head. Using the dataset, we analyzed the effect of gaze and face directions by constructing facial expression classifiers in five ways and evaluating the classification accuracy of each classifier. The results revealed that the single classifier that learned the data for all gaze points achieved the highest classification performance. Then, we investigated which facial part was affected by the gaze and face direction. The results showed that the gaze directions affected the upper facial parts, while the face directions affected the lower facial parts. In addition, by removing the bias of facial expression reproducibility, we investigated the pure effect of gaze and face directions in three conditions. The results showed that, in terms of gaze direction, building classifiers for each direction significantly improved the classification accuracy. However, in terms of face directions, there were slight differences between the classifier conditions. Our experimental results implied that multiple classifiers corresponding to multiple gaze and face directions improved facial expression recognition accuracy, but collecting the data of the vertical movement of gaze and face is a practical solution to improving facial expression recognition accuracy.

DOI： 10.1109/TVCG.2022.3179766

Scopus

researchmap

Language：Others   Publishing type：Research paper (other academic)   Publisher：ACM  

Recent technological advances have enabled the use of AI agents to assist with human tasks and augment human cognitive abilities in a variety of contexts, including decision making. It is critical that users trust these AI agents in order to use them effectively. Given that people tend to trust other people who are similar to themselves, incorporating features of one's own face into the AI agent's face may improve one's trust in the AI agent. However, it is still unclear how impressions differ when comparing agents with the same appearance as one's own and some similarities under the same conditions. Recognizing the appropriate level of similarity when using a self-similar agent is important for establishing a trustworthy agent relationship between people and the AI agent. Therefore, we investigated the effect of the degree of self-similarity of the face of the AI agent on the user's trust in the agent. We examined users' impressions of four AI agents with different degrees of face self-similarity in different scenarios. The results showed that the AI agent, whose similarity to the user's facial feature was slightly recognizable but not obvious, received higher ratings on the feeling of closeness, attractiveness, and facial preferences. These self-similar AI agents were also more trustworthy in everyday non-serious decisions and were more likely to improve people's trustworthiness in such situations. Finally, we discuss the potential applications of our findings to design real-world AI agents.

DOI： 10.1145/3582700.3582721

Scopus

researchmap

Other Link： https://dblp.uni-trier.de/db/conf/aughuman2/ahs2023.html#NiwaMYS23

Language：Others   Publishing type：Research paper (other academic)   Publisher：ACM  

Embedding technology into day-to-day wearables and creating smart devices such as smartwatches and smart-glasses has been a growing area of interest. In this paper, we explore the interaction around face masks, a common accessory worn by many to prevent the spread of infectious diseases. Particularly, we propose a method of using the straps of a face mask as an input medium. We identified a set of plausible gestures on mask straps through an elicitation study (N = 20), in which the participants proposed different gestures for a given referent. We then developed a prototype to identify the gestures performed on the mask straps and present the recognition accuracy from a user study with eight participants. Our results show the system achieves 93.07% classification accuracy for 12 gestures.

DOI： 10.1145/3581641.3584062

Scopus

researchmap

Other Link： https://dblp.uni-trier.de/db/conf/iui/iui2023.html#YamamotoMWS23

Publishing type：Research paper (international conference proceedings)   Publisher：Proceedings of the Annual International Conference of the IEEE Engineering in Medicine and Biology Society, EMBS  

The rise in population and aging has led to a significant increase in the number of individuals affected by common causes of vision loss. Early diagnosis and treatment are crucial to avoid the consequences of visual impairment. However, in early stages, many visual problems are making it difficult to detect. Visual adaptation can compensate for several visual deficits with adaptive eye movements. These adaptive eye movements may serve as indicators of vision loss. In this work, we investigate the association between eye movement and blurred vision. By using Electrooculography (EOG) to record eye movements, we propose a new tracking model to identify the deterioration of refractive power. We verify the technical feasibility of this method by designing a blurred vision simulation experiment. Six sets of prescription lenses and a pair of flat lenses were used to create different levels of blurring effects. We analyzed binocular movements through EOG signals and performed a seven-class classification using the ResNet18 architecture. The results revealed an average classification accuracy of 94.7% in the subject-dependent model. However, the subject-independent model presented poor performance, with the highest accuracy reaching only 34.5%. Therefore, the potential of an EOG-based visual quality monitoring system is proven. Furthermore, our experimental design provides a novel approach to assessing blurred vision.

DOI： 10.1109/EMBC40787.2023.10341004

Web of Science

Scopus

PubMed

researchmap

Other Link： https://dblp.uni-trier.de/db/conf/embc/embc2023.html#WeiLPMIUK23

Publishing type：Research paper (international conference proceedings)   Publisher：Proceedings - SIGGRAPH 2023 Emerging Technologies  

Enhancing human capabilities through the use of multiple bodies has been a significant research agenda. When multiple bodies are synchronously operated in different environments, the differences in environment placement make it difficult to interact with objects simultaneously. In contrast, if automatic control is performed to complement the differences and to perform a parallel task, the mismatch between the user and robotic arm movements generates visuomotor incongruence, leading to a decline in embodiment across the body. This can lead to difficulty completing tasks or achieving goals, and may even cause frustration or anxiety. To address this issue, we have developed a system that allows a parallel operation of synchronized multiple robotic arms by assisting the arm towards which the user's gaze is not directed while maintaining the sense of embodiment over the robotic arms.

DOI： 10.1145/3588037.3595401

Scopus

researchmap

Other Link： https://dblp.uni-trier.de/db/conf/siggraph/siggraph2023et.html#KawamuraKFMKS23

Publishing type：Research paper (international conference proceedings)   Publisher：IEEE  

We propose a virtual reality (VR) baseball batting system for assessing individual decision-making skill based on swing judgement of pitch types and the underlying prediction ability by manipulating combinations of pitching motion and ball trajectory cues. Our analy-sis of data from 10 elite baseball players revealed highly accurate swing motions in conditions during which the batter made precise swing decisions. Delays in swing motion were observed in conditions during which predictive cues were mismatched. Our findings indicated that decision-making based on pitch type influences the inherent stability of decision and accuracy of hitting, and that most batters made decisions based on pitching motion cues rather than on ball trajectory.

DOI： 10.1109/VRW58643.2023.00230

Scopus

researchmap

Other Link： https://dblp.uni-trier.de/db/conf/vr/vr2023w.html#TaniKMFSK23

Language：Others   Publishing type：Research paper (other academic)   Publisher：IEEE  

Smiling plays a crucial role in human communication. It is the most frequent expression shown in daily life. Smile analysis usually employs computer vision-based methods that use data sets annotated by experts. However, cameras have space constraints in most realistic scenarios due to occlusions. Wearable electromyography is a promising alternative; however, issue of user comfort is a barrier to long-term use. Other wearable-based methods can detect smiles, but they lack consistency because they use subjective criteria without expert annotation. We investigate a wearable-based method that uses optical sensors for consistent smile intensity estimation while reducing manual annotation cost. First, we use a state-of-art computer vision method (OpenFace) to train a regression model to estimate smile intensity from sensor data. Then, we compare the estimation result to that of OpenFace. We also compared their results to human annotation. The results show that the wearable method has a higher matching coefficient (r=0.67) with human annotated smile intensity than OpenFace (r=0.56). Also, when the sensor data and OpenFace output were fused, the multimodal method produced estimates closer to human annotation (r=0.74). Finally, we investigate how the synchrony of smile dynamics among subjects and their average smile intensity are correlated to assess the potential of wearable smile intensity estimation.

DOI： 10.1109/ACII55700.2022.9953867

Scopus

researchmap

Other Link： https://dblp.uni-trier.de/db/conf/acii/acii2022.html#MasaiPSKK22

Language：Others   Publishing type：Research paper (scientific journal)   Publisher：Institute of Electrical and Electronics Engineers ({IEEE})  

Recognizing foot gestures can be useful for subtle inputs to appliances and machines in everyday life, but for a system to be useful, it must allow users to assume various postures and work in different spaces. Camera-based and pressure-based systems have limitations in these areas. In this paper, we introduce AnkleSens, a novel ankle-worn foot sensing device that estimates a variety of foot postures using photo reflective sensors. Since our device is not placed between the foot and the floor, it can predict foot posture, even if we keep the foot floating in the air. We developed a band prototype with 16 sensors that can be wrapped around the leg above the ankle. To evaluate the performance of the proposed method, we used eight foot postures and four foot states as preliminary classes. After assessing a test dataset with the preliminary classes, we integrated the eight foot postures into five. Finally, we classified the dataset with five postures in four foot states. For the resulting 20 classes, the average classification accuracy with our proposed method was 79.57% with user-dependent training. This study showed the potential of foot posture sensing as a new subtle input method in daily life.

DOI： 10.1109/ACCESS.2022.3158158

Scopus

researchmap

Language：Others   Publishing type：Research paper (scientific journal)   Publisher：Springer  

High-Speed Thermochromism Control Method Integrating Water Cooling Circuits and Electric Heating Circuits Printed with Conductive Silver Nanoparticle Ink.

DOI： 10.1007/978-3-031-05409-9_6

Scopus

J-GLOBAL

researchmap

Other Link： https://dblp.uni-trier.de/db/conf/hci/hci2022-2.html#MasuiTTSMH22

Publishing type：Research paper (international conference proceedings)   Publisher：IEEE  

Virtual reality (VR) is gaining interest as a platform for sports skills training. VR allows for information manipulation and feedback that would be difficult in reality. This is particularly useful in open skill sports where players must adjust their behavior in response to environmental factors. Auditory feedback (sonification) is constructive for sports training in VR. However, this has not been well studied in open skill-specific situations due to the difficulty of accounting for environmental factors in reality. This study focuses on a serve return, an important scene in tennis. It investigates the effects of sonification on the forehand return stroke in VR by comparing score displays and trajectory visualizations. We designed the sonification based on the difference between experienced and novice players' movements in VR. We then conducted a between-subjects experiment to investigate the effect of the sonification (N=20). The results showed that the system with sonification effectively improved the timing of hip movement for preparing a slow serve return compared to the system without sonification.

DOI： 10.1109/ISMAR-Adjunct57072.2022.00182

Scopus

researchmap

Other Link： https://dblp.uni-trier.de/db/conf/ismar/ismar2022a.html#MasaiKMSK22

Language：Others   Publishing type：Research paper (other academic)  

DOI： 10.1007/978-3-030-78468-3_40

Language：Others   Publishing type：Research paper (other academic)  

DOI： 10.1109/ismar50242.2020.00030

Language：Others   Publishing type：Research paper (other academic)  

DOI： 10.1109/ISMAR50242.2020.00064

Language：Others   Publishing type：Research paper (other academic)  

DOI： 10.1145/3384657.3384787

Language：Others   Publishing type：Research paper (other academic)  

DOI： 10.1145/3374920.3374936