researchmap

researchmap

Language：English   Publishing type：Research paper (international conference proceedings)  

We present ZEL, the first net-zero-energy lifelogging system that allows office workers to collect semi-permanent records of when, where, and what activities they perform on company premises. ZEL achieves high accuracy lifelogging by using heterogeneous energy harvesters with different characteristics. The system is based on a 192-gram nametag-shaped wearable device worn by each employee that is equipped with two comparators to enable seamless switching between system states, thereby minimizing the battery usage and enabling net-zero-energy, semi-permanent data collection. To demonstrate the effectiveness of our system, we conducted data collection experiments with 11 participants in a practical environment and found that the person-dependent (PD) model achieves an 8-place recognition accuracy level of 87.2% (weighted F-measure) and a static/dynamic activities recognition accuracy level of 93.1% (weighted F-measure). Additional testing confirmed the practical long-Term operability of the system and showed it could achieve a zero-energy operation rate of 99.6% i.e., net-zero-energy operation.

DOI： 10.1109/PerCom53586.2022.9762376

Language：English   Publishing type：Research paper (scientific journal)   Publisher：Sensors  

Information on congestion of buses, which are one of the major public transportation modes, can be very useful in light of the current COVID-19 pandemic. Because it is unrealistic to manually monitor the number of riders on all buses in operation, a system that can automatically monitor congestion is necessary. The main goal of this paper’s work is to automatically estimate the congestion level on a bus route with acceptable performance. For practical operation, it is necessary to design a system that does not infringe on the privacy of passengers and ensures the safety of passengers and the installation sites. In this paper, we propose a congestion estimation system that protects passengers’ privacy and reduces the installation cost by using Bluetooth low-energy (BLE) signals as sensing data. The proposed system consists of (1) a sensing mechanism that acquires BLE signals emitted from passengers’ mobile terminals in the bus and (2) a mechanism that estimates the degree of congestion in the bus from the data obtained by the sensing mechanism. To evaluate the effectiveness of the proposed system, we conducted a data collection experiment on an actual bus route in cooperation with Nara Kotsu Co., Ltd. The results showed that the proposed system could estimate the number of passengers with a mean absolute error of 2.49 passengers (error rate of 38.8%).

DOI： 10.3390/s22030881

Web of Science

Scopus

PubMed

researchmap

Language：English   Publishing type：Research paper (international conference proceedings)  

This paper presents an Internet of Things (IoT)-based concept called "IoT Nudge", which aims to induce continuous healthful behavior changes through interactions between human users and IoT devices, including smart sensors and actuators. More specifically, IoT Nudge systems aim to establish self-reflection loops in which our daily behaviors are reflected in the IoT devices around us. The changes in the presentations of those IoT devices promote self-Awareness and provide us with opportunities to improve our lifestyle habits. To reconfigure poor behavior into good behavior, the IoT Nudge provides three functions: (1) automatic tracking of a user's lifestyle habits using IoT sensors, (2) playfully reframing the user's daily activities into other more healthful representations based on analogies, and (3) delivering real-Time ambient feedback through reframed representations from IoT actuators that encourage self-reflection. Herein, we describe our conceptual model of IoT Nudge and introduce three prototype systems aimed at promoting health improvements: fit2plant, eat2pic, and brush2music.

DOI： 10.1145/3460418.3479280

Language：English   Publishing type：Research paper (international conference proceedings)  

Maintaining hand hygiene is the one of the most effective way to prevent the spread of germs during a pandemic. This paper focuses on encouraging people to use a hand sanitizer more frequently by applying the nudge theory to improve hand hygiene behavior in private organizations. We propose a system that recognizes hand hygiene behavior using face recognition and detects hand sanitizer use. The system responds to the user's personal hand hygiene behavior with animation of a virtual bonsai as an interactive agent. To preserve user privacy, we implemented the system on an edge device and conducted experiments for 4 case studies in 2 real-world organizations. The results showed that the system improved the hand hygiene behavior of people in a private organization.

DOI： 10.1145/3472306.3478355

Language：English   Publishing type：Research paper (scientific journal)  

The emergence of various types of commercial cameras (compact, high resolution, high angle of view, high speed, and high dynamic range, etc.) has contributed significantly to the understanding of human activities. By taking advantage of the characteristic of a high angle of view, this paper demonstrates a system that recognizes micro-behaviors and a small group discussion with a single 360 degree camera towards quantified meeting analysis. We propose a method that recognizes speaking and nodding, which have often been overlooked in existing research, from a video stream of face images and a random forest classifier. The proposed approach was evaluated on our three datasets. In order to create the first and the second datasets, we asked participants to meet physically: 16 sets of five minutes data from 21 unique participants and seven sets of 10 min meeting data from 12 unique participants. The experimental results showed that our approach could detect speaking and nodding with a macro average f1-score of 67.9% in a 10-fold random split cross-validation and a macro average f1-score of 62.5% in a leave-one-participant-out cross-validation. By considering the increased demand for an online meeting due to the COVID-19 pandemic, we also record faces on a screen that are captured by web cameras as the third dataset and discussed the potential and challenges of applying our ideas to virtual video conferences.

DOI： 10.3390/s21175719

Language：English  

DOI： 10.1016/j.jhlto.2025.100232

PubMed

Language：English  

DOI： 10.1007/s10147-025-02747-9

PubMed

Language：English  

DOI： 10.1007/s10147-024-02683-0

PubMed

Language：English  

DOI： 10.1200/PO-24-00718

PubMed

Language：English  

DOI： 10.1200/PO.24.00283

PubMed

Language：English  

DOI： 10.1200/GO-24-00644

PubMed

Language：English  

DOI： 10.1093/carcin/bgaf011

PubMed

Publisher：Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies  

This study addresses a knowledge gap in understanding intervention strategies for ethical disengagement in mobile games, focusing on approaches that prioritize player health by moderating game interactivity and aesthetics to mitigate usage. In a randomized experiment involving 84,325 participants playing Flying Gorilla over one month, we evaluated user engagement across seven conditions: a baseline, loading time delays of 1, 5, and 10 seconds, grayscale display, and combinations of grayscale with 5- and 10-second delays. The results indicated that loading time delays and grayscale display effectively reduced retention rates and average daily playtime, with the most significant impact observed when combining grayscale with longer loading time delays. Specifically, a grayscale display paired with a 10-second loading delay reduced daily playtime by up to 30.8% and retention rates by 40.4% compared to the baseline. Additionally, regional and user-type analyses suggested that the effects of these interventions may vary based on regional characteristics and user engagement patterns. These findings highlight the potential of such interventions that might help nudge players to curb excessive gaming and offer insights into designing mobile gaming experiences that consider player well-being.

DOI： 10.1145/3712281

Scopus

Language：English  

DOI： 10.1016/j.jlb.2024.100282

PubMed

Language：English  

DOI： 10.1002/acn3.52307

PubMed

Language：English  

DOI： 10.1111/jgh.16839

PubMed

Language：English  

DOI： 10.1158/1078-0432.CCR-24-2396

PubMed

Language：English  

DOI： 10.1007/s11604-025-01735-3

PubMed

Language：English  

DOI： 10.1111/ases.13407

PubMed

Language：Japanese  

DOI： 10.1254/fpj.24056

PubMed

Language：English  

DOI： 10.3390/cancers16244204

PubMed

Language：English  

DOI： 10.1007/s00595-024-02967-y

PubMed

Language：English  

DOI： 10.1212/WNL.0000000000209992

PubMed

Language：English  

DOI： 10.5551/jat.65049

PubMed

Language：English  

DOI： 10.1038/s41591-024-03254-6

PubMed

Language：English  

DOI： 10.1016/j.annonc.2024.08.2240

PubMed

Language：English  

DOI： 10.1007/s10875-024-01821-7

PubMed

Language：English  

DOI： 10.3802/jgo.2025.36.e38

PubMed

Language：English  

DOI： 10.1038/s41416-024-02834-0

Web of Science

PubMed

Language：English  

DOI： 10.1158/1078-0432.CCR-24-1651

PubMed

Language：English  

DOI： 10.1111/ases.13370

Web of Science

PubMed

Publisher：Multimodal Technologies and Interaction  

With the rise of head-mounted displays (HMDs), immersive virtual reality (IVR) for second-language learning is gaining attention. However, current methods fail to fully exploit IVR’s potential owing to the use of abstract avatars and limited human perspectives in learning experiences. This study investigated IVR’s novel potential by using non-human avatars to understand complex concepts. We developed a system for learning English vocabulary through the actions of non-human avatars, offering a unique learning perspective. This paper presents an IVR vocabulary learning environment with a dragon avatar and compares word retention rates (immediate and one-week memory tests), subjective workload, and emotional changes with traditional methods. We also examined the vocabulary ranges that are teachable using this system by varying the number of avatars. The results showed that the proposed method significantly reduced forgotten English words after one week compared to traditional methods, indicating its effectiveness in the long term.

DOI： 10.3390/mti8100093

Scopus

Language：English  

DOI： 10.1016/j.jphs.2024.07.007

Web of Science

PubMed

Language：English  

DOI： 10.1007/s00345-024-05240-6

PubMed

Language：English  

DOI： 10.1093/brain/awae185

Web of Science

PubMed

Language：English  

DOI： 10.1038/s44276-024-00073-7

PubMed

DOI： 10.1016/j.physa.2024.129846

Web of Science

Web of Science

Web of Science

Web of Science

Web of Science

Web of Science

Publishing type：Research paper (scientific journal)   Publisher：Multimodal Technologies and Interaction  

Subtitles play a crucial role in facilitating the understanding of visual content when watching films and television programs. In this study, we propose a method for presenting subtitles in a way that considers cognitive load when viewing video content in a non-native language. Subtitles are generally displayed at the bottom of the screen, which causes frequent eye focus switching between subtitles and video, increasing the cognitive load. In our proposed method, we focused on the position, display time, and amount of information contained in the subtitles to reduce the cognitive load and to avoid disturbing the viewer’s concentration. We conducted two experiments to investigate the effects of our proposed subtitle method on gaze distribution, comprehension, and cognitive load during English-language video viewing. Twelve non-native English-speaking subjects participated in the first experiment. The results show that participants’ gazes were more focused around the center of the screen when using our proposed subtitles compared to regular subtitles. Comprehension levels recorded using LightSub were similar, but slightly inferior to those recorded using regular subtitles. However, it was confirmed that most of the participants were viewing the video with a higher cognitive load using the proposed subtitle method. In the second experiment, we investigated subtitles considering connected speech form in English with 18 non-native English speakers. The results revealed that the proposed method, considering connected speech form, demonstrated an improvement in cognitive load during video viewing but it remained higher than that of regular subtitles.

DOI： 10.3390/mti8060051

Web of Science

Scopus

researchmap

Language：English  

DOI： 10.1007/s10147-024-02493-4

Web of Science

PubMed

Language：English  

DOI： 10.1038/s44276-024-00049-7

PubMed

Language：English   Publishing type：Research paper (scientific journal)  

Federated learning (FL) has gained widespread attention as a distributed machine learning (ML) technique that offers data protection when training on local devices. Unlike conventional centralized training in traditional ML, FL incorporates privacy and security measures as it does not share raw data between the client and server, thereby safeguarding potentially sensitive information. However, there are still vulnerabilities in the FL field, and commonly used approaches, such as encryption and blockchain technologies, often result in significant computational and communication costs, making them impractical for devices with restricted resources. To tackle this challenge, we present a privacy-preserving FL system specifically designed for resource-constrained devices, leveraging compressive sensing and differential privacy (DP) techniques. We implemented the weight-pruning-based compressive sensing method with an adaptive compression ratio based on resource availability. In addition, we employ DP to introduce noise to the gradient before sending it to a central server for aggregation, thereby protecting the gradient's privacy. Evaluation results demonstrate that our proposed method achieves slightly better accuracy when compared to state-of-the-art methods like DP-federated averaging, DP-FedOpt, and adaptive Gaussian clipping-DP (AGC-DP) for the MNIST, Fashion-MNIST, and Human Activity Recognition data sets. Furthermore, our approach achieves this higher accuracy with a lower total communication cost and training time than the current state-of-the-art methods. Moreover, we comprehensively evaluate our method's resilience against poisoning attacks, revealing its better resistance than existing state-of-the-art approaches.

DOI： 10.1109/JIOT.2023.3347552

DOI： 10.1039/d3ta06486g

Web of Science

Language：English  

DOI： 10.2188/jea.JE20220154

Web of Science

PubMed

Language：English  

DOI： 10.1111/cas.16020

Web of Science

PubMed

Language：English  

DOI： 10.1126/sciadv.ade2780

Web of Science

PubMed

DOI： 10.1200/JCO.2024.42.3_suppl.544

Web of Science

DOI： 10.1200/JCO.2024.42.3_suppl.538

Web of Science

Language：English  

DOI： 10.1093/plcell/koad320

Web of Science

PubMed

Web of Science

Publishing type：Research paper (international conference proceedings)   Publisher：2024 8th International Conference on Robotics, Control and Automation, ICRCA 2024  

Dense video captioning tasks are used to detect interesting events and provide descriptive text for these events from untrimmed videos. This technology has the potential to be used in security surveillance and human care applications. However, current methods often overlook the relationships between objects in the video, which limits their applicability and makes it challenging to adapt them to specific domains, such as video summarization for indoor human activities. In these scenarios, human activities are closely intertwined with the objects in the scene. In this paper, we propose a plug-and-play module designed to enhance existing dense video captioning methods with spatial information. Specifically, we extract spatial information about the interesting objects using Red-Green-Blue-Depth (RGB-D) images and the results of image segmentation. We then integrate this information into the captions generated by the Dense Video Captioning (DVC) method using a fine-tuned Large Language Model (LLM). We evaluate the performance of our model on a custom dataset and demonstrate that our system provides a convenient and effective approach for obtaining space-enhanced captions.

DOI： 10.1109/ICRCA60878.2024.10649311

Scopus

researchmap

Publishing type：Research paper (international conference proceedings)   Publisher：2024 IEEE International Conference on Pervasive Computing and Communications Workshops and other Affiliated Events, PerCom Workshops 2024  

In recent years, with the trend towards shorter working hours, the quality of meetings has increasingly impacted these hours. Consequently, meeting quality and value become more important. To achieve better results in limited time, everyone needs to conduct meetings effectively. This requires skills such as facilitating the meeting, listening to others' opinions, accurately expressing one's own views, and using body language. However, these skills are currently only correlated with meeting effectiveness and lack comprehensive qualitative and quantitative assessments, leaving specific methods for improvement unclear. Additionally, the type of meeting can lead to participant stress or disinterest, making it essential to understand their psychological safety and engagement for more effective meetings. Measuring these psychological states poses significant challenges due to privacy and compliance concerns, particularly when using cameras or wearable sensors. This paper broadly addresses these issues and reports on our approach to detecting non-verbal cues, specifically nodding, from chair movements as a potential solution.

DOI： 10.1109/PerComWorkshops59983.2024.10502817

Web of Science

Scopus

researchmap

Publishing type：Research paper (scientific journal)   Publisher：IEEE Access  

The development of smart sensors and appliances can provide a lot of services. Nevertheless, the act of aggregating data containing sensitive information related to privacy in a single location poses significant issues. Such information can be misused by a malicious attacker. Also, some previous studies attempted to apply privacy mechanisms, but they decreased data utility. In this paper, we propose privacy protection mechanisms to preserve privacy-sensitive sensor data generated in a smart home. We leverage Rényi differential privacy (RDP) to preserve privacy. However, the preliminary result showed that using only RDP still significantly decreases the utility of data. Thus, a novel scheme called feature merging anonymization (FMA) is proposed to preserve privacy while maintaining data utility by merging feature dataframes of the same activities from other homes before applying RDP. Also, the expected trade-off is defined so that data utility should be greater than the privacy preserved. To evaluate the proposed techniques, we define privacy preservation and data utility as inverse accuracy of person identification (PI) and accuracy of activity recognition (AR), respectively.We trained the AR and PI models for two cases with and without FMA, using 2 smart-home open datasets i.e. the HIS and Toyota dataset. As a result, we could lower the accuracy of PI in the HIS and Toyota dataset to 73.85% and and 41.18% with FMA respectively compared to 100% without FMA, while maintaining the accuracy of AR at 94.62% and 87.3% with FMA compared to 98.58% and 89.28% without FMA in the HIS and Toyota dataset, respectively. Another experiment was conducted to explore the feasibility of implementing FMA in a local server by partially merging frames of the original activity with frames of other activities at different merging ratios. The results show that the local server can still satisfy the expected trade-off at some ratios.

DOI： 10.1109/ACCESS.2024.3390039

Web of Science

Scopus

researchmap

Publishing type：Research paper (international conference proceedings)   Publisher：MOBISYS 2024 - Proceedings of the 2024 22nd Annual International Conference on Mobile Systems, Applications and Services  

Since office work has become large-scale and diversified in companies or organizations, work engagement and efficiency have been always an important index of a team's or group's evaluation because it is directly connected to their outcomes. In order to identify the group work context, we first need to recognize for what and how long the individual members are spending their time at their desks, but without privacy concerns and underestimation of their actual work. In this paper, we propose and evaluate the base system of personal desk activity recognition by using a low-cost compact WiFi node and its WiFi channel state information (CSI), which can lead to a lightweight group work context identification system. As a result, we achieved 94.2% desk activity recognition accuracy using the on-desk receiver, in recognizing five different classes.

DOI： 10.1145/3643832.3661452

Web of Science

Scopus

researchmap

Publishing type：Research paper (scientific journal)   Publisher：IEEE Access  

Machine learning's exponential growth has transformed healthcare, with Federated Learning (FL) playing a pivotal role. Despite its significance, FL is vulnerable to privacy attacks. In response, researchers have integrated differential privacy (DP) into FL. Nevertheless, incorporating DP introduces challenges such as increased total communication costs and computational overheads due to the introduction of noise. This drawback renders FL with DP less viable for healthcare systems, characterized by numerous low-resource devices and network bandwidth constraints. To overcome this limitation, we propose integrating a Discrete Fourier Transform (DFT) aggregator post-noise addition to transform the gradient generated by local training before sending it to the central server. This process reduces the gradient size and provides rudimentary encryption. The evaluation results reveal the superior performance of our proposed method, demonstrating an enhanced accuracy ranging from 0.2% to 2% compared to existing differential privacy techniques, including RDP, DP-SGD, ZcDP, LDP-Fed, and DP-AdapClip. Our approach substantially reduces the total communication costs (ranging from 6% to 43% across different privacy budgets) with faster training times in healthcare datasets such as the PIMA Indian database and Breast Cancer Histopathology Images.

DOI： 10.1109/ACCESS.2024.3418016

Web of Science

Scopus

researchmap

Publishing type：Research paper (international conference proceedings)   Publisher：International Conference on Information Networking  

As the high-speed wireless network spreads, the usage of shared Wi-Fi in places such as offices, coworking spaces, and homes has been increased. Along with the development and diversification of digital content, the amount of data consumed by individual devices has also grown. In situations where multiple users share limited network resources, some content that consumes a large amount of bandwidth like video streaming can potentially degrade the Quality of Experience (QoE) for other users nearby. Consequently, a need for persuasive intervention systems that encourage prosocial behavior is rising taking into consideration the QoE of other network users. To address this issue, this study proposes a "Persuasive Public Wi-Fi"that employs Wi-Fi to incrementally convince users towards more considerate usage of network resources. To be specific, we suppose that the persuasive public Wi-Fi must include three different modes: 1) a mode of normal networking, 2) a mode that can intervene with individual users through a captive portal, and 3) a mode that intentionally limits bandwidth, i.e., Quality of Service (QoS) control. This work demonstrates the design and implementation of the proposed system and presents the results of operational verification using a prototype system.

DOI： 10.1109/ICOIN59985.2024.10572181

Scopus

researchmap

Publishing type：Research paper (international conference proceedings)   Publisher：MOBISYS 2024 - Proceedings of the 2024 22nd Annual International Conference on Mobile Systems, Applications and Services  

We present a secure decentralized learning framework suitable for resource-constrained devices within a cluster environment. Our approach focuses on enhancing privacy preservation during model aggregation by utilizing Differential Privacy. This technique adds random noise to gradients obtained from local training on edge devices before sending them for aggregation. This noise addition ensures that sensitive information within the gradients remains distorted, thus safeguarding user privacy. We showcase the implementation of our system on a cluster system employing Raspberry Pi 4 Model B devices, illustrating its feasibility and effectiveness in real-world scenarios. Through this demonstration, we highlight the practical applicability of our system in enabling secure decentralized learning within resource-constrained environments.

DOI： 10.1145/3643832.3661843

Web of Science

Scopus

researchmap

Publishing type：Research paper (international conference proceedings)   Publisher：CEUR Workshop Proceedings  

In an era where digital technology profoundly influences our daily habits, we explore the potential of color psychology as a key element in the design of persuasive technology to promote health behavior change. This paper presents a strategy that uses color psychology to both encourage positive behaviors and discourage negative ones. We present two different applications: "eat2pic," a system designed to promote healthier eating habits by transforming meal selection into an engaging activity of adding color to images, and "color-wall," an application designed to reduce digital distractions by applying a grayscale filter to non-essential digital content. Through these demonstrations, we explore the efficacy and potential of color psychology in persuasive technology to influence user behavior in a health context and offer promising solutions to improve dietary choices and concentration while using computers.

Scopus

researchmap

Publishing type：Research paper (scientific journal)   Publisher：Institute of Electrical and Electronics Engineers (IEEE)  

Sugary beverages are a significant contributor to sugar consumption, and their excessive consumption is associated with increased risks of elevated blood glucose levels and diabetes. Many individuals have a strong preference for sugary beverages and often find beverages with lower sugar content to be less satisfying. Attempts to switch to less sugary options are frequently short-lived, leading to a return to higher-sugar beverages. Recognizing that 75 - 95% of taste perception is influenced by scent, we investigated a scent-based approach to reduce sugar intake while preserving the perception of sweetness. This study introduces an olfactory interface in the form of a mug named 'Aromug,' designed to emit a sweet scent in sync with the drinking action. Aromug incorporates motion sensing and scent presentation functions to enhance the perceived sweetness of a beverage, thereby encouraging a gradual reduction in sugar intake. Our experiments, involving 33 participants, demonstrated that the combined scents of sugar-free coffee and chocolate increased the perception of sweetness (p = 1.641 × 10-2). The study also found that the simultaneous presentation of scent and visual cues improved taste satisfaction and sweetness perception. Additionally, we observed variations in sweetness preference related to age and frequency of coffee consumption. It was particularly observed that people in their 20s and those who frequently drink coffee tend to perceive the taste of beverages as sweeter. This suggests a potential for Aromug to customize the scent experience based on individual preferences, offering a novel way to encourage healthier beverage choices.

DOI： 10.1109/ACCESS.2024.3401392

Web of Science

Scopus

researchmap

Publishing type：Research paper (international conference proceedings)   Publisher：MOBISYS 2024 - Proceedings of the 2024 22nd Annual International Conference on Mobile Systems, Applications and Services  

Indoor activity recognition using WiFi sensing is expected to have a wide range of applications, such as monitoring the elderly and home security. The state of radio wave propagation is called Channel State Information (CSI) and can be obtained using specific devices. By collecting CSI and applying machine learning, it is possible to recognize activities. However, CSI is sensitive to changes in the environment, so whenever the arrangement of furniture or the layout of the room changes, it is necessary to re-collect sample data and retrain the model. Retraining a model requires annotation work, which is costly in terms of time and effort. To address this issue, this paper proposes an annotation system that uses backscatter tags to reduce the cost of data collection and model training. In this system, a backscatter tag that generates a frequency shift depending on its angle is attached to a person during data collection, and activity recognition is performed by detecting the presence of the frequency shift. The backscatter tag-based recognition results are then used as pseudo-ground truth for model update.

DOI： 10.1145/3643832.3661451

Web of Science

Scopus

researchmap

Language：English  

DOI： 10.12938/bmfh.2023-104

PubMed

Publishing type：Research paper (scientific journal)  

DOI： 10.48550/arXiv.2405.06252

researchmap

Publisher：Sensors and Materials  

Dense video captioning aims to extract every goal event and its corresponding period, and has gathered significant attention owing to its potential and valuable applications in smart homes, human care, security monitoring, and more. However, current methods do not sufficiently reduce ambiguity in the generated captions and have a limited field of view, making it difficult to integrate the relationship between people and their surrounding environment into the captions. The limitations restrict their applicability in smart home or indoor security systems. These systems require clear distinctions between normal and abnormal human actions, as similar actions can have different interpretations depending on the surrounding environment. For instance, in indoor security systems, ambiguous captions might fail to distinguish between harmless activities such as a person walking and potentially concerning behaviors such as unauthorized access attempts. In this article, we propose a retrieval-augmented generation (RAG)-based system to enhance existing methods, making them suitable for recording human activity indoors. Our key ideas are as follows. First, we collect information about the house environment using a 3D reconstruction and 3D detection process to build a knowledge base. Second, we design a RAG procedure to extract the relevant environmental context. Third, we develop a spatial information enhancement query based on human detection results from RGB and depth image pairs. We utilize the summarization and reasoning capabilities of a large language model to fuse all information, thereby obtaining spatially enhanced dense captions of human indoor activities. We evaluate our method by comparing it with PDVC (end-to-end Dense Video Captioning with Parallel Decoding), GVL (Learning Grounded Vision-Language Representation for Versatile Understanding in Untrimmed Videos), and SG-PDVC (Scene Graph-enhanced PDVC) on a custom video dataset collected from two houses with eight different camera positions, using Recall, Precision, Bilingual Evaluation Understudy (BLEU), Metric for Evaluation of Translation with Explicit ORdering (METEOR), Recall-Oriented Understudy for Gisting Evaluation (ROUGE), and Consensus-based Image Description Evaluation (CIDEr) as metrics. Our method outperforms the three compared methods in BLEU-3, METEOR, CIDEr, and Precision metrics, but shows a small decline in ROUGE-L and Recall metrics compared with GVL. These results demonstrate that our method effectively incorporates spatial information and reduces ambiguity for indoor human activity caption applications.

DOI： 10.18494/SAM5238

Scopus

Language：English   Publishing type：Research paper (international conference proceedings)   Publisher：ACM International Conference Proceeding Series  

In this paper, we introduce the concept of a spoken dialogue system, which aims to recognize user eating behaviors during meals and promotes healthier eating habits by engaging users in an interactive co-dining experience. To provide an intuitive and more natural access, this information is integrated into a natural conversation with the user. Therefore, we conducted a preliminary study to explore how a human-like virtual avatar's eating speed affects the eating pace of a co-dining user. The results indicate that dining together with an avatar reduces the user's feeling of loneliness and enhances the overall satisfaction and enjoyment of the meal. Therefore, this work represents an important step toward realizing engaging co-dining experiences with human-like avatars and provides new insights into the future design of IoT systems to support solitary dining and health recommendations.

DOI： 10.1145/3627050.3630728

Scopus

researchmap

Other Link： https://dblp.uni-trier.de/db/conf/iot/iot2023.html#MayumiN0AYM23

Language：English   Publishing type：Research paper (international conference proceedings)   Publisher：ACM International Conference Proceeding Series  

To support the decision-making process for selecting upcoming tourist spots (PoI: Points of Interest), it is necessary to share the tourist destinations' current status. Methods for sharing the current status of PoIs could include installing live cameras or having tourists share real-Time photos. Still, there are challenges in terms of installation and maintenance costs and privacy concerns. In this paper, a tourism support system is proposed to aid in the decision-making of which PoI to visit next by automatically generating PoI current status images from template images and contextual information of tourist destinations while preserving privacy and presenting them to users. In the proposed system, semantic segmentation is used to divide template images into categories such as sky class, trees/grasses class, crowd class, and overall class. Based on the separately collected contextual information (weather, condition of trees/grass, crowd density, time, etc.), appropriate image transformations (using style transfer, etc) are applied to each category. The proposed system consists of an automated method for generating PoI status images and an application that presents these images to users. As part of a tourism experiment using the proposed system, 11 individuals aged 20 to 30 participated in a roughly 2-hour tour of Nara City in Japan. The results showed that compared to presenting template images, the proposed method yielded a decision-making score improvement of 0.27 and a similarity (to the current status) score improvement of 0.45 (out of five), both of which were statistically significant.

DOI： 10.1145/3627050.3627068

Scopus

researchmap

Other Link： https://dblp.uni-trier.de/db/conf/iot/iot2023.html#KawanakaNSY23

Language：English   Publishing type：Research paper (international conference proceedings)   Publisher：ACM International Conference Proceeding Series  

In this demo, we will show our proposed system that can count nodding without either a camera or any sensor attached to the person. Our proposed system capitalizes on the fact that the upper body moves in conjunction with nodding and that this body motion slightly shakes the chair. We explore the challenge of recognizing nodding from the extremely subtle sway of a chair. To recognize nods in real-Time, we employed a supervised learning approach using acceleration data from sensors attached to the chair's backrest. Ultimately, the Support Vector Machine (SVM) achieved a nodding recognition accuracy of 0.990. Further testing of the accuracy of nodding frequency measurements yielded an accuracy of 0.947, suggesting that the optimal position for the accelerometer is the backrest. These results indicate that simply placing the accelerometer on the backrest can effectively quantify the frequency of nods from seated participants.

DOI： 10.1145/3627050.3630740

Scopus

researchmap

Other Link： https://dblp.uni-trier.de/db/conf/iot/iot2023.html#HayashidaNCA23

DOI： 10.1016/j.annonc.2023.09.123

Web of Science

Language：English  

DOI： 10.3389/fimmu.2023.1223331

Web of Science

PubMed

Language：English  

DOI： 10.1038/s41416-023-02378-9

Web of Science

PubMed

DOI： 10.1016/j.annonc.2023.09.1749

Web of Science

DOI： 10.1016/j.annonc.2023.09.1782

Web of Science

DOI： 10.1186/s40623-023-01890-9

Web of Science

Web of Science

Language：English  

DOI： 10.1016/j.tjnut.2023.05.027

Web of Science

PubMed

Language：English  

DOI： 10.2188/jea.JE20220341

Web of Science

PubMed

Language：English  

DOI： 10.1093/bjd/ljad114

Web of Science

PubMed

DOI： 10.3390/agronomy13071791

Web of Science

Web of Science

Language：English   Publishing type：Research paper (scientific journal)   Publisher：MDPI AG  

Litter has become a social problem. To prevent litter, we consider urban planning, the efficient placement of garbage bins, and interventions with litterers. In order to carry out these actions, we need to comprehensively grasp the types and locations of litter in advance. However, with the existing methods, collecting the types and locations of litter is very costly and has low privacy. In this research, we have proposed the conceptual design to estimate the types and locations of litter using only the sensor data from a smartwatch worn by the user. This system can record the types and locations of litter only when a user raps on the litter and picks it up. Also, we have constructed a sound recognition model to estimate the types of litter by using sound sensor data, and we have carried out experiments. We have confirmed that the model built with other people’s data enabled to estimate the F-measure of 80.2% in a noisy environment through the experiment with 12 participants.

DOI： 10.3390/su151310079

Web of Science

Scopus

researchmap

Language：English  

DOI： 10.1016/j.esmoop.2023.101558

Web of Science

PubMed

Language：English  

DOI： 10.2188/jea.JE20210155

Web of Science

PubMed

DOI： 10.1029/2022JA031248

Web of Science

DOI： 10.1038/s41588-023-01375-1

Web of Science

PubMed

Language：English  

DOI： 10.1038/s41416-023-02189-y

Web of Science

PubMed

Language：English  

DOI： 10.1007/s13340-023-00618-0

Web of Science

PubMed

Language：English  

DOI： 10.2188/jea.JE20210142

Web of Science

PubMed

DOI： 10.1557/s43578-022-00886-7

Web of Science

Language：English  

DOI： 10.1016/j.clcc.2022.12.001

Web of Science

PubMed

DOI： 10.1103/PhysRevC.107.024306

Web of Science

Web of Science

Web of Science

Language：English  

DOI： 10.1016/j.resinv.2022.08.009

Web of Science

PubMed

Language：English  

DOI： 10.1038/s41591-022-02115-4

Web of Science

PubMed

Publishing type：Research paper (international conference proceedings)   Publisher：2023 IEEE International Conference on Pervasive Computing and Communications Workshops and other Affiliated Events, PerCom Workshops 2023  

In this paper, we present a point cloud streaming method of real-space objects such as humans and animals for real-time 3D reconstruction in VR space. The system uses a depth camera to scan a human or animal, divides the point cloud into parts of the body, and then controls the quality of the point cloud (i.e., resolution and frame rate) for each part in real-time according to the object's motion and context. This enables point cloud streaming with limited resources (computational and network resources) and maximizes the user's quality of experience (QoE). We have implemented and evaluated a series of systems incorporating the proposed method to enhance the user experience in realistic environments and scenarios while maintaining interactivity in VR-based online communication. The results show that the proposed system is feasible under resource-constrained environments without significantly affecting the user's QoE.

DOI： 10.1109/PerComWorkshops56833.2023.10150337

Scopus

researchmap

Other Link： https://dblp.uni-trier.de/db/conf/percom/percomw2023.html#IshimaruNFSY23

Web of Science

Publishing type：Research paper (international conference proceedings)   Publisher：2023 14th International Conference on Mobile Computing and Ubiquitous Network, ICMU 2023  

Internet use is expanding along with the spread of smartphones and the promotion of digital use, but information security literacy is required to use the Internet safely. This paper addresses these challenges by focusing on the potential of microlearning and nudge theory to support continuous learning to solve these problems and presents a novel learning system that utilizes a Wi-Fi landing portal that appears as a dialog or a browser. To assess the effectiveness of the proposed system, we conducted a comparative analysis of information security test scores before and after the operation of the learning system and questionnaire-based evaluation. The results showed that the analysis of answer scores before and after the operation of the learning system was significant (\mathrm{p} < 0.05), indicating an improvement in scores. It was also found that using interface information with nudges effectively motivated participants to engage in repeated learning behaviors. However, it should be noted that frequent interventions might reduce acceptability, and the psychological burden arising from task interruptions needs to be taken into account.

DOI： 10.23919/ICMU58504.2023.10412210

Scopus

researchmap

Other Link： https://dblp.uni-trier.de/db/conf/icmu/icmu2023.html#HiraokaMAN23

Web of Science

Publishing type：Research paper (international conference proceedings)   Publisher：2023 14th International Conference on Mobile Computing and Ubiquitous Network, ICMU 2023  

Nowadays more and more people are wearing smart-watches in their daily lives. The various sensors embedded in smartwatches bring the ability to evaluate users' status as well as the risk of privacy issues. For example, if users are typing on key-boards while wearing smartwatches, the attacker could know the typing contents from the sensor data collected by the malicious applications that are installed on the targets' smartwatches. In this paper, we propose WatchLogger, the framework using audio and accelerometer signals to recognize the English words being typed, for demonstrating how to implement the smartwatch-based side-channel attack. Different from the previous studies that focused on the recognition of each key or pair of keys being pressed, WatchLogger aims to perform recognition on the scale of words. To achieve this goal, WatchLogger exploits the audio signals for segmentation and the accelerometer signals for classification. In addition, we propose an ensemble classification model to deal with the problem caused by too many words. At last, we build the dataset WTW-100 with 100 classes of words and 100 samples for each class, and we conduct experiments on the dataset. The experimental results show an accuracy of 98.5 % for keystroke recognition and 91.5 % for word classification, showing a considerable performance of WatchLogger.

DOI： 10.23919/ICMU58504.2023.10412218

Scopus

researchmap

Other Link： https://dblp.uni-trier.de/db/conf/icmu/icmu2023.html#LiANCFW23

Web of Science

Publishing type：Research paper (international conference proceedings)   Publisher：Proceedings - International Conference on Tools with Artificial Intelligence, ICTAI  

In this paper, we propose ToonMeet, a hybrid frame-work for high-resolution and style-controllable online meeting toonification that ensures real-time operation speed. ToonMeet applies video frame interpolation to traditional portrait toonification pipelines, allowing for the synthesis of intermediate frames between adjacent toonified keyframes, significantly accelerating the overall process and saving computational resources. However, this approach brings a new problem, where prevailing flow-based video frame interpolation methods tend to cause more ghost and blur artifacts in toonified scenes compared to non-toonified scenes, especially when fast-moving objects exist. We study this previously undiscussed problem and explore its causes. To address this, we introduce a new dataset called TM3B (Toonified Multi-modal Meeting Behaviors), offering high-resolution and cross-platform multi-modal stylized meeting data of Japanese youth in various scenarios. Then, we fine-tune ToonMeet on these tailored data and the resulting model presents improved optical flow estimation ability on toonified videos. Extensive experiments demonstrate that ToonMeet can achieve great spatiotemporal performance and perform high-quality toonification of online meetings with real-time operation speed.

DOI： 10.1109/ICTAI59109.2023.00013

Web of Science

Scopus

researchmap

Other Link： https://dblp.uni-trier.de/db/conf/ictai/ictai2023.html#ChenFNA23

Publishing type：Research paper (international conference proceedings)   Publisher：Springer  

In this paper, we propose a purchasing support system for offline stores that can present information to users without being evasive by interactively presenting a small amount of product information according to their interest in the product as predicted by the system. We design and implement a proof-of-concept system for an actual use case scenario: a snack purchasing area in an office. We evaluated the effectiveness of the proposed system through a snack purchasing experiment with 11 participants. Experimental results suggest that the proposed method can induce user interest and encourage viewing and that some users may receive and consider the information presented while making a purchase.

DOI： 10.1007/978-3-031-30933-5_8

Web of Science

Scopus

researchmap

Other Link： https://dblp.uni-trier.de/db/conf/persuasive/persuasive2023.html#TokunagaMSOYNA23

Publishing type：Research paper (international conference proceedings)   Publisher：2023 IEEE International Conference on Pervasive Computing and Communications Workshops and other Affiliated Events, PerCom Workshops 2023  

3D applications such as VR and AR are attracting increasing commercial attention, and point cloud video is expected to be one of the most suitable representations for real-time applications due to its simplicity and versatility. However, point cloud data is large in size and difficult to stream in a mobile network environment with limited bandwidth. Therefore, a method for streaming point clouds with low bandwidth consumption while maintaining the quality of the user experience is needed. In this paper, we present a point cloud streaming method of real-space objects such as humans and animals for real-time 3D reconstruction in VR space. The system uses a depth camera to scan a human or animal, divides the point cloud into parts of the body, and then controls the quality of the point cloud (i.e., resolution and frame rate) for each part in real-time according to the object's motion and context. This enables point cloud streaming with limited resources (computational and network resources) and maximizes the user's quality of experience. We exhibit a series of systems that enhance the user experience in remote communication in realistic environments and scenarios while maintaining interactivity between real-space objects and remote users.

DOI： 10.1109/PerComWorkshops56833.2023.10150405

Scopus

researchmap

Other Link： https://dblp.uni-trier.de/db/conf/percom/percomw2023.html#IshimaruNFSY23a

Publishing type：Research paper (international conference proceedings)   Publisher：2023 IEEE International Conference on Pervasive Computing and Communications Workshops and other Affiliated Events, PerCom Workshops 2023  

There are two nasal pathways through which we perceive odors: 'orthonasal smell,' which comes from the tip of the nose, and 'retronasal smell,' which comes from the mouth and exits through the nose. In particular, the retronasal smell is known to be more critical in influencing perceived taste than the orthonasal smell. In this study, we propose a smart lid called Kaolid that changes the taste of beverages by presenting scent information as retronasal smell. Kaolid achieves retronasal smell by equipping a small, lightweight straw- or cup-type olfactory device with a sensor that senses drinking behavior and the amount of water in the cup. The straw-type device injects the scent into the tube through the middle of the straw, delivering the scent to the user's mouth simultaneously as the beverage is being consumed. The cup-type device injects the scent into the cup through the lid and delivers the scent to the user's mouth at the beginning of drinking to achieve in-mouth scenting. The concept of Kaolid is to promote water consumption, considered insipid, by altering the sense of taste through a scent presented when drinking the beverage.

DOI： 10.1109/PerComWorkshops56833.2023.10150412

Scopus

researchmap

Other Link： https://dblp.uni-trier.de/db/conf/percom/percomw2023.html#MayumiNMUMY23

Publishing type：Research paper (international conference proceedings)   Publisher：Proceedings - IEEE International Conference on Mobile Data Management  

Federated learning has gained popularity as a distributed machine learning approach that provides security and privacy for data trained on local devices. However, vulnerabilities still exist in this approach, and common solutions such as encryption and blockchain techniques often suffer from high computation and communication costs, making them impractical for resource-constrained devices. To solve this problem, we propose a privacy-preserving federated learning system that leverages compressive sensing and differential privacy, specifically designed for devices with limited computational resources. In this paper, we demonstrate the capabilities of our proposed system in resource-limited environments. We outline the features, infrastructure, and algorithm of our proposed system, and simulate its performance using image datasets on a Raspberry Pi 4 and an Android smartphone in a cloud environment. Our approach offers a practical solution for secure and privacy-preserving federated learning in resource-constrained scenarios, with potential applications in various domains such as healthcare, IoT, and edge computing.

DOI： 10.1109/MDM58254.2023.00038

Scopus

researchmap

Other Link： https://dblp.uni-trier.de/db/conf/mdm/mdm2023.html#HidayatNA23

Publishing type：Research paper (international conference proceedings)   Publisher：2023 IEEE International Conference on Pervasive Computing and Communications Workshops and other Affiliated Events, PerCom Workshops 2023  

Many services based on human activity recognition (HAR) have been developed; however, user activity data include a large amount of private information. Although privacy protection is important in activity recognition, it has not been sufficiently explored. Therefore, we propose a privacy-preserving mechanism for HAR services that uses differential privacy. The proposed method reduces the user recognition accuracy to a level that satisfies the privacy requirements by adding weighted noise to the features in the learning model construction and then improves the activity recognition accuracy (service usefulness). The results indicate that when the privacy requirement is defined as less than the probability of a user being identified by chance, the proposed method improves the activity recognition accuracy by approximately 10 % compared to the conventional method.

DOI： 10.1109/PerComWorkshops56833.2023.10150239

Scopus

researchmap

Other Link： https://dblp.uni-trier.de/db/conf/percom/percomw2023.html#FujimotoNA23

Publishing type：Research paper (international conference proceedings)   Publisher：2023 14th International Conference on Mobile Computing and Ubiquitous Network, ICMU 2023  

In this paper, we propose a method to measure human head motion, especially nodding, without attaching any sensors to the person. Our proposed system focuses on the fact that the upper body moves along with nodding and that the body motion slightly shakes the chair. We challenge the problem of whether it is possible to recognize a nodding from the extremely slight sway of a chair. To reveal the optimal position of sensors, we collected data by attaching multiple accelerometers to various positions on a chair, including the backrest, the seat's underside, and the legs. Using a supervised learning approach, we determined the best positions and combinations of sensors for recognizing nodding more collectively. The Support Vector Machine (SVM) achieved a nodding recognition accuracy of 0.990. Further testing of the accuracy of nodding frequency measurements resulted in an accuracy of 0.947, suggesting that the best position for the accelerometer is the backrest. These results suggest that simply placing the accelerometer on the backrest can effectively quantify the nod frequency of seated participants.

DOI： 10.23919/ICMU58504.2023.10412249

Scopus

researchmap

Other Link： https://dblp.uni-trier.de/db/conf/icmu/icmu2023.html#HayashidaNCA23

Web of Science

Publishing type：Research paper (international conference proceedings)   Publisher：CEUR Workshop Proceedings  

In this study, the target behavior of prosocial behavior was defined as garbage disposal (behavior of taking out the trash), and an intervention method using an inducement was proposed and evaluated. Specifically, we conducted an intervention in which the saturation of a trash can was visualized using a colored light that could be changed to red, yellow, and green. In addition, questionnaire surveys were administered before and after the intervention to 28 undergraduate and graduate students and faculty members, asking about their attitudes and feelings toward garbage disposal behavior, and personality traits. The results of a survey on the frequency of trash disposal be-fore and after the experiment showed that the time that the saturated trash can was left unattended was reduced by 81%. The questionnaire survey confirmed that the intervention increased the positive attitude toward trash disposal behavior, indicating that the intervention had an impact on trash disposal behavior. In addition, we were able to confirm the characteristics of individual personality traits and susceptibility to intervention.

Scopus

researchmap

Other Link： https://dblp.uni-trier.de/rec/conf/persuasive/2023bcss

Publisher：Proceedings - IEEE International Conference on Mobile Data Management  

Federated learning provides techniques for training algorithms using mobile or decentralized devices, in contrast to traditional machine learning in which algorithm training is performed on centralized devices. In addition, federated learning provides privacy and security features, as the client and server do not share raw data, which may contain confidential information. A number of studies have shown, however, that using federated learning alone is not enough to protect data privacy in certain situations. To overcome this problem, differential privacy is proposed, which is a technique in which artificial noise is added to the raw data. By implementing this method, a high level of privacy protection can be obtained, however this added noise also reduces model accuracy. To address this issue, this paper proposes a new approach to implement differential privacy in federated learning using adaptive Gaussian clipping. We implemented the method by tightening the privacy budget, and introducing dynamic sampling probability, adaptive clipping based on hyperparameters, and a new privacy loss calculation. Our method's main objective is to adaptively change the amount of noise given to the model, thereby maximizing the model's accuracy performance, while maintaining privacy protection levels. Evaluation results show that our proposed method presents slightly better accuracy when compared to other existing differential privacy variants such as RDP, DP-SGD, and ZcDP, for both balanced (i.i.d.) and unbalanced datasets (non-i.i.d.), for a lower total communication cost than some variants.

DOI： 10.1109/MDM58254.2023.00042

Scopus

Language：English  

DOI： 10.1507/endocrj.EJ22-0585

Web of Science

PubMed

Publishing type：Research paper (international conference proceedings)   Publisher：2023 International Conference on Consumer Electronics - Taiwan, ICCE-Taiwan 2023 - Proceedings  

Human activity recognition (HAR) using the deep learning method has caught the attention of researchers thanks to its automatic feature extraction and accurate prediction capabilities. However, for applications based on a wearable sensor, such as an inertial measurement unit (IMU), the process of collecting and hand-labeling large amounts of data is complicated and labor-intensive, meaning that there is a limited amount of data available for model training. Therefore, there is a need to propose and develop data augmentation approaches to generate high quality data for the growth of HAR research. We propose a head-mounted virtual gyroscope signal generator to alleviate the problems caused by the lack of data in head movement-related applications. Unlike previous work, our system only generates head-motion related gyroscope data, minimizing system complexity. We trained a deep-learning model in a head motion-based application with different generated sensor data ratios, and show the viability of our proposed data generation method.

DOI： 10.1109/ICCE-Taiwan58799.2023.10227010

Scopus

researchmap

Other Link： https://dblp.uni-trier.de/db/conf/icce-tw/icce-tw2023.html#LuCDNA23

DOI： 10.1200/PO.22.00653

Web of Science

PubMed

Language：English  

DOI： 10.1371/journal.pone.0279169

PubMed

Language：English  

DOI： 10.1200/PO.22.00688

Web of Science

PubMed

Language：English  

DOI： 10.1186/s12885-022-10398-6

Web of Science

PubMed

DOI： 10.1063/5.0124299

Web of Science

Language：English  

DOI： 10.1016/j.sleh.2022.08.010

Web of Science

PubMed

DOI： 10.1142/S1088424622500456

Web of Science

Language：English  

DOI： 10.1016/j.sleep.2022.09.020

Web of Science

PubMed

Language：English  

In recent years, with the spread of mobile devices, “participatory sensing,” in which users are asked to contribute information, such as their surrounding environment, via their smartphones, has attracted increasing attention. However, in active participatory sensing, which asks users to input text or upload photos, respondents often try to complete the request quickly and effortlessly, and consequently, not always accurately. In this study, we propose a method of expressing intention to contribute (EIC) for suppressing careless responses in participatory sensing tasks. We implemented a prototype system that requests two types of EIC method (tap the button, shake the phone), and conducted the experiment over two weeks with 20 participants. Through the statistical tests, we found that proposed EIC methods significantly suppressed the number of careless responses compared with the normal situation.

DOI： 10.1007/978-3-030-94822-1_50

Language：English  

DOI： 10.1111/cas.15504

Web of Science

PubMed

Language：English  

DOI： 10.1038/s41598-022-22108-0

Web of Science

PubMed

DOI： 10.1016/j.tafmec.2022.103460

Web of Science

Language：English  

DOI： 10.1111/bjd.21276

Web of Science

PubMed

Language：English  

DOI： 10.1097/MD.0000000000030580

Web of Science

PubMed

Language：English  

DOI： 10.3389/fcimb.2022.925444

Web of Science

PubMed

Language：English  

DOI： 10.1016/j.biopsych.2022.04.003

Web of Science

PubMed

Language：English   Publishing type：Research paper (international conference proceedings)   Publisher：UbiComp/ISWC 2022 Adjunct - Proceedings of the 2022 ACM International Joint Conference on Pervasive and Ubiquitous Computing and Proceedings of the 2022 ACM International Symposium on Wearable Computers  

As the Attention Economy has grown, numerous digital contents are made with the intention of capturing users' attention and spending more of their time by the use of attractive graphics and color psychology. In this paper, we propose a novel defense strategy, called a color-wall, to block digital distraction brought on by the consumption of visually seductive and addictive digital content. When it detects that a user is viewing media or using applications unrelated to the main task, it removes colors from the screen (gray-scaling) to make them less aesthetically pleasing and directs the user away from looking at them. We implemented a mac OS software as a proof-of-concept for the color-wall and conducted user testing (N=14) to assess the effectiveness of the proposed strategy. The evaluation results suggest that the color-wall is an effective strategy for reducing unwanted digital distractions such as browsing restricted content (YouTube, Twitter, etc.) while working on a PC.

DOI： 10.1145/3544793.3563403

Scopus

researchmap

Other Link： https://dblp.uni-trier.de/db/conf/huc/ubicomp2022ap.html#NakamuraTA22

Language：English   Publishing type：Research paper (international conference proceedings)   Publisher：UbiComp/ISWC 2022 Adjunct - Proceedings of the 2022 ACM International Joint Conference on Pervasive and Ubiquitous Computing and Proceedings of the 2022 ACM International Symposium on Wearable Computers  

Reducing the amount of sugar consumed from daily beverages is essential for well-being. However, since a sudden reduction in intake is mentally painful for those who prefer sweetened beverages, solutions are needed to help them gradually become accustomed to non-sweetened drinks. In this paper, we propose Aromug, a mug-type olfactory interface that amplifies the user's perceived sweetness by providing a sweet scent along with the action of drinking. Through user studies, we investigated the effect of the information presented by Aromug on the taste of iced coffee. The experiment results showed that the sugar-free coffee and chocolate aroma might enhance the perceived sweetness compared to the no-scent case. In addition, it was suggested that there were differences in preference for sweetness depending on age and frequency of coffee drinking.

DOI： 10.1145/3544793.3563402

Scopus

researchmap

Other Link： https://dblp.uni-trier.de/db/conf/huc/ubicomp2022ap.html#MayumiN0MY22

Language：English  

DOI： 10.1136/bmjno-2022-000291

Web of Science

PubMed

Language：English  

DOI： 10.1136/bmjresp-2022-001337

Web of Science

PubMed

Language：English  

DOI： 10.1111/1346-8138.16422

Web of Science

PubMed

DOI： 10.1016/j.jallcom.2022.165272

Web of Science

Language：English   Publishing type：Research paper (scientific journal)   Publisher：MYU K.K.  

Short stick exercises have been attracting attention from the viewpoint of preventing falls and improving the health of elderly people and are generally performed under the guidance of instructors and nursing staff at nursing homes. However, in situations such as the COVID-19 pandemic, where people should refrain from unnecessary outings, it is advisable that individuals perform short stick exercises at home and record their exercise implementation status. In this paper, we propose an inertial measurement unit (IMU)-sensor-based short stick exercise tracking method that can automatically record the types and amounts of exercises performed using a short stick equipped with an IMU sensor. The proposed method extracts time-domain and frequency-domain features from linear acceleration and quaternion time-series data obtained from the IMU sensor and classifies the type of exercise using an inference model based on machine learning algorithms. To evaluate the proposed method, we collected sensor data from 21 young subjects (in their 20s) and 14 elderly subjects (79-95 years old), where the participants performed three sets (10 times per set) of eight basic types of short stick exercises (five types for elderly people). As a result of evaluating the proposed method using this data set, we confirmed that when LightGBM was used as the learning algorithm, it achieved F values of 90.0 and 86.6% for recognizing the type of exercise for young and elderly people, respectively.

DOI： 10.18494/sam3968

Web of Science

Scopus

researchmap

Language：English  

DOI： 10.1080/08820139.2021.1973491

Web of Science

PubMed

DOI： 10.1016/j.annonc.2022.05.067

Web of Science

Web of Science

Web of Science

DOI： 10.1021/acs.cgd.2c00145

Web of Science

Web of Science

Web of Science

DOI： 10.1016/j.annonc.2022.04.091

Web of Science

Language：English   Publishing type：Research paper (international conference proceedings)   Publisher：IEEE  

Stick exercises, which have been attracting attention for improving the health of the elderly, are usually performed in nursing homes under the guidance of nursing staff. However, in the current pandemic in which the elderly are advised to refrain from going out unnecessarily, it is desirable for each individual to be able to perform the stick exercises alone. In this study, we aim to develop a stick exercise support system that can automatically record the number of times an elderly person performs each type of stick exercise and provide feedback to improve the movement for each exercise. As a first step toward the realization of this stick exercise support system, we investigated a method for recognizing exercise movements using inertial measurement unit (IMU) sensors. In the evaluation experiment, 21 subjects performed 3 sets (10 times per set) of eight basic stick exercises. The exercise movements were classified based on the linear acceleration and quaternion data obtained from the IMU. As a result, 90% of F-measure was achieved when using Light GBM as the learning algorithm.

DOI： 10.1109/cphs56133.2022.9804564

Scopus

researchmap

Language：English   Publishing type：Research paper (international conference proceedings)   Publisher：IEEE  

In many of the world's major cities, commuter trains provide vital transportation support and thus play an essential role in our daily lives. Therefore, it has become necessary to estimate the degree of congestion in each train car, both to improve passenger comfort levels and, more recently, to prevent worsening the COVID-19 pandemic infection rate. However, it is difficult to estimate the degree of congestion within a train without violating passenger privacy. The same issues are true for busses, which is noteworthy because we have previously developed and evaluated a system that can estimate the degree of congestion within a bus while protecting passenger privacy by using Bluetooth Low Energy (BLE) signals. In this paper, we report on our efforts to extend that system to railway use, which were conducted on actual trains in cooperation with Kintetsu Railway Co., Ltd. During this trial, we collected BLE signals and used the data to estimate congestion levels in each car using an ML regression model. The results show that the mean absolute error (MAE) and the mean absolute percentage error (MAPE) could be estimated at accuracy levels of 5.56 and 0.27, respectively.

DOI： 10.1109/di-cps56137.2022.00007

Web of Science

Scopus

researchmap

Web of Science

Language：English  

DOI： 10.3390/cancers14102438

Web of Science

PubMed

Language：English  

DOI： 10.1200/PO.21.00535

Web of Science

PubMed

Language：English  

DOI： 10.1111/cas.15226

Web of Science

PubMed

DOI： 10.1109/LRA.2022.3148799

Web of Science

Language：English   Publishing type：Research paper (international conference proceedings)  

Due to the pandemic of COVID-19, product sampling at supermarkets has been restricted, and the disposal of unsold vegetables and other perishable foods has become a problem. To explore alternatives to product sampling, this study focuses on the “aroma” generated during product sampling and explores the effect of aroma nudges on purchasing behavior. Specifically, we focused on the scenario of promoting the purchase of Yamato-maru eggplant, a traditional vegetable of Nara Prefecture in Japan. We conducted an experiment in a supermarket for two months, from June to July of 2021, the season of the Yamato-maru eggplant. We compared the number of visits to the sales booth, the time spent in the booth, and the sales volume under four conditions: (1) no presentation, (2) presentation of paper media, (3) presentation of the paper and video media, and (4) presentation of a paper, video, and olfactory media. The experiment results suggest that the inclusion of aroma nudges has a positive potential to attract consumer interest and positively influence their purchasing decisions.

Language：English   Publishing type：Research paper (international conference proceedings)   Publisher：IEEE  

In this paper, we propose a method for estimating psychological states such as hesitation and interest during survey responses based on touch operation behavior (touch log). In addition, we propose two enhanced web survey RUIs (Response User Interfaces), (a) a slidebar type RUI and (b) a magnifier type RUI, to expand the operational differences due to hesitation and interest for more accurate estimation. To evaluate the proposed RUIs, we implemented Augmented Web Survey (AWS) as an extension of LimeSurvey and conducted two experiments. Responses were collected from 8 subjects in the experiment to evaluate (a) a slidebar type RUI and from 21 subjects to evaluate (b) a magnifier type RUI. We measured the degree of hesitation by evaluating the subjective hesitation to the questions on a 5-point scale. For the measurement of the degree of interest, the participants were asked to select their top two favorite images among the six images, and the true value was taken as the top two. As a result of comparing the operation logs of the conventional RUI and the slidebar type RUI, it was confirmed that there was a significant difference depending on the degree of hesitation. In addition, more magnification operations were obtained for the magnifier type RUI than the standard pinch-out operation suggesting that it may help estimate psychological states such as hesitation.

DOI： 10.1109/LifeTech53646.2022.9754964

Scopus

researchmap

Other Link： https://dblp.uni-trier.de/db/conf/lifetech/lifetech2022.html#NakagawaAN22

Language：English   Publishing type：Research paper (international conference proceedings)   Publisher：Springer International Publishing  

To promote the face-to-face communication reduced by COVID-19, we proposed and implemented a context-aware Slack chatbot based on Cyber-Physical sensing that helps colleagues meet more often in the same place. Our system periodically collects the user’s internal context through Slack (cyber sensing) and uses small BLE beacons distributed to colleagues and beacon scanners installed in a laboratory to sense physical attendance (physical sensing). In addition, the system notifies the user of recommended actions, such as lunch or coffee break, depending on the context determined by the Cyber-Physical sensors. We deployed the proposed system in a laboratory environment and conducted an initial experiment for six weeks. Experimental results confirmed that our system can encourage serendipitous face-to-face communication during periods when the frequency of attending school and going to work dropped due to COVID-19. It was also found that in an environment such as a laboratory, where a certain level of trust has already been established, the openness of the collected information can further motivate users to participate in the system.

DOI： 10.1007/978-3-030-98438-0_18

Web of Science

Scopus

researchmap

Other Link： https://dblp.uni-trier.de/db/conf/persuasive/persuasive2022.html#TanakaMNA22

Language：English   Publishing type：Research paper (international conference proceedings)  

In this study, we propose a behavior change support system that encourages people to use stairs instead of elevators. Using the stairs is not only good for one's health, but nowadays it also plays a role in dense avoidance from the perspective of COVID-19. Although many people are waiting for the elevator, we thought that if we knew how many people were on the elevator, more people could change their behavior. However, there are few buildings where the number of people in the elevator is displayed on each floor. So, we have developed a headcount measurement system in the cargo and a visualization system, those can be easily appended to the current elevator. As a method to measure the number of people in an elevator, we propose a method to detect BLE (Bluetooth Low Energy) signals transmitted from the terminals of elevator users who have installed COCOA, an application for confirming contact with the COVID-19 in Japan, and to measure in real time the number of detected BLE signals with a received signal strength (RSSI) exceeding a certain value. We propose a method to measure the number of BLE signals detected in real time. In this paper, we have designed and developed the system, verified the accuracy of the detection of the continuous operation time and the number of passengers, and estimated the behavior pattern and waiting time of elevator users using the system. The evaluation of the transformation of the decision making of the elevator users brought about by this system is out of the scope of this paper and will be discussed in the future.

Language：English   Publishing type：Research paper (international conference proceedings)   Publisher：{IEEE}  

In this paper, we propose FedTour, a federated learning-based method for training tourism object recognition models, which utilizes short-distance direct communication between user devices and maximizes the model performance within a limited number of updates. In FedTour, whenever two user devices are within range, they first exchange metadata including the learning degree (e.g., recognition accuracy) of their models, and determine whether it is effective to integrate the peer model by using a regressor trained with various pairs of models with different accuracy to predict the accuracy of the merged model. Once it is deemed effective, the model parameters are exchanged and the model is updated using FedAvg (averaging weights of two models of user devices). By carefully setting the threshold of whether FedAvg is applied or not, model performance is improved within a limited number of model parameter exchanges resulting in lower power consumption of user devices. We conducted a simulation using mobile phone trace data of actual users in a real sightseeing area and evaluated the improvement in accuracy of a CNN model that recognizes 10 objects while limiting the number of model parameter exchanges to only 40. Results show FedTour increased the initial model accuracy by 112%, while the baseline gossip-based method achieved 69%.

DOI： 10.1109/PerComWorkshops53856.2022.9767391

Web of Science

Scopus

researchmap

Other Link： https://dblp.uni-trier.de/db/conf/percom/percomw2022.html#TomitaTNSY22

Language：English   Publishing type：Research paper (international conference proceedings)  

DOI： 10.1109/percom53586.2022.9762376

Publishing type：Research paper (international conference proceedings)   Publisher：CEUR-WS.org  

In this study, we propose a behavior change support system that encourages people to use stairs instead of elevators. Using the stairs is not only good for one's health, but nowadays it also plays a role in dense avoidance from the perspective of COVID-19. Although many people are waiting for the elevator, we thought that if we knew how many people were on the elevator, more people could change their behavior. However, there are few buildings where the number of people in the elevator is displayed on each floor. So, we have developed a headcount measurement system in the cargo and a visualization system, those can be easily appended to the current elevator. As a method to measure the number of people in an elevator, we propose a method to detect BLE (Bluetooth Low Energy) signals transmitted from the terminals of elevator users who have installed COCOA, an application for confirming contact with the COVID-19 in Japan, and to measure in real time the number of detected BLE signals with a received signal strength (RSSI) exceeding a certain value. We propose a method to measure the number of BLE signals detected in real time. In this paper, we have designed and developed the system, verified the accuracy of the detection of the continuous operation time and the number of passengers, and estimated the behavior pattern and waiting time of elevator users using the system. The evaluation of the transformation of the decision making of the elevator users brought about by this system is out of the scope of this paper and will be discussed in the future.

Scopus

researchmap

Other Link： https://dblp.uni-trier.de/rec/conf/persuasive/2022a

DOI： 10.1029/2022JA030312

Web of Science

DOI： 10.1016/j.neulet.2022.136467

Web of Science

PubMed

DOI： 10.1021/acs.jpcc.1c08739

Web of Science

DOI： 10.1200/JCO.2022.40.4_suppl.168

Web of Science

Language：English  

DOI： 10.1136/thoraxjnl-2020-216263

Web of Science

PubMed

DOI： 10.1016/j.jtct.2021.10.021

Web of Science

PubMed

Language：English  

DOI： 10.1136/bmjdrc-2021-002479

Web of Science

PubMed

Language：English  

DOI： 10.1002/jhbp.903

Web of Science

PubMed

Language：English  

DOI： 10.1002/jhbp.1081

Web of Science

PubMed

Language：English  

DOI： 10.1002/jhbp.1071

Web of Science

PubMed

DOI： 10.3390/biom12010018

Web of Science

PubMed

Publishing type：Research paper (international conference proceedings)   Publisher：CEUR Workshop Proceedings  

Due to the pandemic of COVID-19, product sampling at supermarkets has been restricted, and the disposal of unsold vegetables and other perishable foods has become a problem. To explore alternatives to product sampling, this study focuses on the “aroma” generated during product sampling and explores the effect of aroma nudges on purchasing behavior. Specifically, we focused on the scenario of promoting the purchase of Yamato-maru eggplant, a traditional vegetable of Nara Prefecture in Japan. We conducted an experiment in a supermarket for two months, from June to July of 2021, the season of the Yamato-maru eggplant. We compared the number of visits to the sales booth, the time spent in the booth, and the sales volume under four conditions: (1) no presentation, (2) presentation of paper media, (3) presentation of the paper and video media, and (4) presentation of a paper, video, and olfactory media. The experiment results suggest that the inclusion of aroma nudges has a positive potential to attract consumer interest and positively influence their purchasing decisions.

Scopus

researchmap

DOI： 10.1109/CPHS56133.2022.00007

Web of Science

Publishing type：Research paper (international conference proceedings)   Publisher：Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)  

Understanding user emotion is essential for Human-AI Interaction (HAI). Thus far, many approaches have been studied to recognize emotion from signals of various physiological modalities such as cardiac activity and skin conductance. However, little attention has been paid to the fact that physiological signals are influenced by and reflect various factors that have little or no association with emotion. While emotion is a cross-modal factor that triggers responses across multiple physiological modalities, features used in existing approaches also reflect modality-specific factors that affect only a single modality and have little association with emotion. To address this, we propose an approach to extract features that exclusively reflect cross-modal factors from multimodal physiological signals. Our approach introduces a multilayer RNN with two types of layers: multiple Modality-Specific Layers (MSLs) for modeling physiological activity in individual modalities and a single Cross-Modal Layer (CML) for modeling the process by which emotion affects physiological activity. By having all MSLs update their hidden states using the CML hidden states, our RNN causes the CML to learn cross-modal factors. Using real physiological signals, we confirmed that the features extracted by our RNN reflected emotions to a significantly greater extent than the features of existing approaches.

DOI： 10.1007/978-981-99-7019-3_40

Scopus

researchmap

Other Link： https://dblp.uni-trier.de/db/conf/pricai/pricai2023-1.html#IshikawaKNNIMA22

Publishing type：Research paper (international conference proceedings)   Publisher：IEEE  

The Internet is an effective tool for learners to gain new knowledge. Often, people use search engines (e.g., Google) rather than accessing websites directly. People have their search techniques to find specific information. In particular, people with domain knowledge tend to search more efficiently than novices. By understanding the gap between people with domain knowledge and novices, the novice can understand the path to becoming an expert. Therefore, in this study, we wanted to know what differences exist in search and programming behavior with and without domain knowledge. In this experiment, we asked a group with and without domain knowledge to solve ten programming problems and collected search logs (input knowledge) and compilation logs (output knowledge). Specifically, the first dataset consisted of 13 participants who had taken a university programming class. The second dataset consisted of 20 participants who had not taken a programming class and had no domain knowledge. We examined differences in search and compilation behavior based on participants' domain knowledge from this data. Since we observed a difference between each group when referring to the correlation coefficient, we performed a binary classification of novice and experienced participants using Random Forest, and achieved an average precision of 0.95, indicating that there were different trends in behavior with and without domain knowledge.

DOI： 10.1109/iThings-GreenCom-CPSCom-SmartData-Cybermatics55523.2022.00034

Scopus

researchmap

Other Link： https://dblp.uni-trier.de/db/conf/ithings/ithings2022.html#WatanabeMNAI22

Publisher：CEUR Workshop Proceedings  

In this study, we propose a behavior change support system that encourages people to use stairs instead of elevators. Using the stairs is not only good for one's health, but nowadays it also plays a role in dense avoidance from the perspective of COVID-19. Although many people are waiting for the elevator, we thought that if we knew how many people were on the elevator, more people could change their behavior. However, there are few buildings where the number of people in the elevator is displayed on each floor. So, we have developed a headcount measurement system in the cargo and a visualization system, those can be easily appended to the current elevator. As a method to measure the number of people in an elevator, we propose a method to detect BLE (Bluetooth Low Energy) signals transmitted from the terminals of elevator users who have installed COCOA, an application for confirming contact with the COVID-19 in Japan, and to measure in real time the number of detected BLE signals with a received signal strength (RSSI) exceeding a certain value. We propose a method to measure the number of BLE signals detected in real time. In this paper, we have designed and developed the system, verified the accuracy of the detection of the continuous operation time and the number of passengers, and estimated the behavior pattern and waiting time of elevator users using the system. The evaluation of the transformation of the decision making of the elevator users brought about by this system is out of the scope of this paper and will be discussed in the future.

Scopus

Publishing type：Research paper (scientific journal)  

DOI： 10.48550/arXiv.2211.06696

researchmap

DOI： 10.1109/SII52469.2022.9708891

Web of Science

Language：English  

DOI： 10.1200/PO.21.00383

Web of Science

PubMed

Language：English   Publishing type：Research paper (international conference proceedings)  

DOI： 10.23919/ICMU50196.2021.9638884

Language：English   Publishing type：Research paper (international conference proceedings)  

For the COVID-19 Contact Confirming Application (COCOA) to be fully effective, it is important that many people install COCOA on their smartphones. However, because the current COCOA implements only a minimum number of functions, it is difficult to motivate people to install it. Therefore, we focus on an extrinsic approach that motivates the installation of COCOA by adding fun functions through interaction with a ubiquitous system on the environment side. In this paper, we introduce an interactive system that nudges users to install COCOA by reframing the act of installing COCOA as getting a ticket to participate in fun experiences.

DOI： 10.1145/3460418.3479284

Language：English   Publishing type：Research paper (international conference proceedings)  

In this paper, we introduce a food-tech design as a healthy nudge called eat2pic that encourages people to slow down their eating pace and encourage healthier eating habits. The eat2pic system is composed of a sensor-equipped chopstick (one of a pair) and components that use context-aware digital canvases. The eat2pic system achieves (1) automatic tracking of what and how fast the user consumed each mouthful of food, (2) real-time visual feedback that indicates good or bad eating behavior. The key concept of eat2pic involves extending the relationship between humans and paintings in order to establish a closed-loop in which daily eating behavior is reflected in pictures on the canvas near us and feedback from the canvas leads our lifestyles towards healthier diets. The eat2pic system provides a novel experience in which users can come to see eating as a playful task that involves coloring landscape pictures, and not simply taking nutrition into their bodies.

DOI： 10.1109/PerComWorkshops51409.2021.9431002

Language：English   Publishing type：Research paper (international conference proceedings)  

In this paper, we propose ProceThings, a new middleware platform to provide smart community services by utilizing computational resources of IoT devices in a target community area. To realize ProceThings, we address three key challenges: (1) dynamic load balance management among numerous IoT devices; (2) distributed task assignment/execution over the IoT devices taking into account fault-Tolerance; and (3) fulfillment of a service level agreement (SLA) for each service. For (1), ProceThings employs a heuristic monitoring mechanism which hierarchically aggregates load and resource conditions from all IoT devices in the target area (or belonging to a service). For (2), ProceThings employs a cluster-based architecture where proximity IoT devices are grouped into clusters with a fail over function, where they are allocated processing tasks from user queries. For (3), ProceThings employs demand-Aware in-situ resource provisioning which dynamically predicts and assigns a sufficient amount of computational resources within the area where the service is provided to meet the SLA while preventing over-provisioning of resources. We have implemented a prototype of ProceThings running on commodity small computers consisting of Raspberry Pis and Intel NUCs and confirmed that the above mechanisms can properly work satisfying the corresponding SLAs when running smart community services.

DOI： 10.1145/3427477.3429275

Language：English   Publishing type：Research paper (international conference proceedings)  

In response to the Coronavirus 2019 (COVID-19) pandemic, the World Health Organization (WHO) has published preventive measures such as performing hand hygiene frequently, wearing a medical mask, trying to avoid touching face and so on. This paper presents a nudge-based system to promote hand hygiene in a private organization. The proposed system consists of a hand sanitizer station equipped with a magnetic sensor to sense user presses. We conducted 4 case studies to compare the effects of nudging on the frequency of hand sanitizer use: no nudging, traditional nudging, non-personalized nudging, and personalized nudging. The results reveal that using nudge-based methods offer a significant increase in the frequency hand sanitizer use.

DOI： 10.1145/3384419.3430595

Language：English   Publishing type：Research paper (international conference proceedings)  

Recently smart homes equipped with many sensors and IoT devices are widespread. However, when smart home users receive smart home services like elderly monitoring, they need to upload their privacy sensitive data to potentially untrusted cloud servers where the service quality (user's benefit) depends on the amount/frequency of the uploaded data. In this paper, aiming to minimize the risk of privacy leakage and maximize users' benefit obtained through services, we propose a novel privacy-aware data management method that works on a smart-home system composed of smart homes with sensors, edge computing servers, and a cloud server. We formulate a combinatorial optimization problem which determines the best choice of data type (raw or activity label recognized at the edge) and upload frequency in each time slot taking into account the constraints of edge server resources and users' budgets as well as the k-anonymity of activities and users' preferences. Since the target problem is NP-hard, we propose a heuristic algorithm to derive semi-optimal solutions by determining choices with better objective function values in a greedy manner. Through experiments using smart-home open dataset, we confirmed that the proposed method outperforms the conventional methods using only a cloud server.

DOI： 10.1109/SMARTCOMP50058.2020.00048

Language：English   Publishing type：Research paper (scientific journal)  

Recently, due to the drastic increase in foreign tourists coming to Japan, there has been a demand to provide smart tourism services that enable inbound tourists to comfortably enjoy sightseeing. To provide satisfactory experiences for tourists, it is desirable to provide tourist information in a timely manner by considering dynamic information, which is information that changes over time, such as current congestion information in destination spots and travel route information, in addition to static information, such as the preferences and profiles of tourists. However, in many existing systems, serious problems occur, such as (1) a lack of support for on-site use, (2) a lack of consideration of dynamic information, and (3) heavy burden on tourists. In this paper, we propose a novel system that can provide tourism plans for tourism spots in a timely manner. The proposed system consists of the following two key mechanisms: (A) A mechanism for acquiring preference information from tourists (including preference on dynamic information); (B) a curation mechanism for realizing on-site tourism. To demonstrate the effectiveness of the proposed system, we carried out evaluation experiments utilizing real tourism spots and simulations. As a result, we obtained the following primary findings: (1) On-site tourism spot recommendation is effective for tourists who do not make detailed tourism plans before sightseeing; (2) preference information for participants can be reflected in the tourism spot recommendation while massively reducing the burden on participants; (3) it is possible to obtain a higher satisfaction level than is achieved with model courses, which are often used for sightseeing.

DOI： 10.3390/smartcities3020013

Language：English   Publishing type：Research paper (scientific journal)  

In the field of sports, there are increasing opportunities to use inertial measurement units (IMUs) to enhance the training process and improve the performance of athletes. We focus on kendo, a traditional martial art using shinai (bamboo swords) in Japan, and propose methods for detecting and recognizing strike activities using IMUs towards realizing a kendo skill improvement support system. We used a sensor data set of strike activities obtained from 14 participants (seven kendo-experienced and seven inexperienced persons). We attached four IMUs to the participants’ right wrist, waist, and shinai (tsuba and saki-gawa). First, to detect the strike activity, we calculated the dynamic time warping (DTW) distance between the training data and the time series data, and detected the strike activity sections. The proposed method detected strike activities with a high accuracy of 95.0&#37;. Next, to recognize the strike activity, we recognized five types (Center-Men, Right-Men, Left-Men, Dō, and Kote). In the person-dependent (PD) case, we achieved an accuracy of 89.5&#37; using data of the right wrist. In the person-independent (PI) case, we achieved an accuracy of 54.9&#37; using IMUs attached to the three positions. These results clarified the points to be improved in the proposed method to realize the support system.

DOI： 10.18494/SAM.2020.2615

Language：English   Publishing type：Research paper (scientific journal)  

Changing behavior related to improper lifestyle habits has attracted attention as a solution to prevent lifestyle diseases, such as diabetes, heart disease, arteriosclerosis, and stroke. To drive health behavior changes, wearable devices are needed, and they must not only provide accurate sensing and visualization functions but also effective intervention functions. In this paper, we propose a health support system, WaistonBelt X, that consists of a belt-type wearable device with sensing and intervention functions and a smartphone application. WaistonBelt X can automatically measure a waistline with a magnetometer that detects the movements of a blade installed in the buckle, and monitor the basic activities of daily living with inertial sensors. Furthermore, WaistonBelt X intervenes with the user to correct lifestyle habits by using a built-in vibrator. Through evaluation experiments, we confirmed that our proposed device achieves measurement of the circumference on the belt position (mean absolute error of 0.93 cm) and basic activity recognition (F1 score of 0.95) with high accuracy. In addition, we confirmed that the intervention via belt vibration effectively improves the sitting posture of the user.

DOI： 10.3390/s19204600

Language：English   Publishing type：Research paper (international conference proceedings)  

In searching for a rental property, information such as a degree of quietness and sunshine on the rental property is important. Sensing devices are necessary to get that information. However, data collection devices cannot use the Internet and power from the outlet because the target is vacant rental property. Therefore, it is necessary to construct an IoT device capable of sensing environmental information without internet and power. In this paper, we develop an environmental information sensing device under the constraint that there is no electricity in the vacant rental property and a no Internet environment. As a result of the demonstration experiment based on the cooperation of real estate agents, we clarified that the proposed device can collect data.

DOI： 10.1109/GCCE46687.2019.9015533

Language：English   Publishing type：Research paper (international conference proceedings)  

A wearable based long-term lifelogging system is desirable for the purpose of reviewing and improving users lifestyle habits. Energy harvesting (EH) is a promising means for realizing sustainable lifelogging. However, present EH technologies suffer from instability of the generated electricity caused by changes of environment, e.g., the output of a solar cell varies based on its material, light intensity, and light wavelength. In this paper, we leverage this instability of EH technologies for other purposes, in addition to its use as an energy source. Specifically, we propose to determine the variation of generated electricity as a sensor for recognizing "places" where the user visits, which is important information in the lifelogging system. First, we investigate the amount of generated electricity of selected energy harvesting elements in various environments. Second, we design a system called EHAAS (Energy Harvesters As A Sensor) where energy harvesting elements are used as a sensor. With EHAAS, we propose a place recognition method based on machine-learning and implement a prototype wearable system. Our prototype evaluation confirms that EHAAS achieves a place recognition accuracy of 88.5&#37; F-value for nine different indoor and outdoor places. This result is better than the results of existing sensors (3-axis accelerometer and brightness). We also clarify that only two types of solar cells are required for recognizing a place with 86.2&#37; accuracy.

DOI： 10.1109/PERCOM.2019.8767385

Language：English   Publishing type：Research paper (international conference proceedings)  

In this paper, we focus on Kendo, which is a traditional sport in Japan, and propose a strikes-thrusts activity recognition method using a wrist sensor towards a pervasive Kendo support system. We collected the inertial sensor data set from 6 subjects. We attached 3 inertial sensor units (IMUs) on the subjects body, and 2 IMUs on the Shinai (bamboo sword used for Kendo). On the body, IMUs were placed on the Right Wrist, Waist and Right Ankle. On the Shinai, they were placed on the Tsuba and Saki-Gawa. We first classified strikes-thrusts activities consisting of 4 general types, Men, Tsuki, Do, and Kote, followed by further classification into 8 detailed types. We achieved 90.0&#37; of F-measure in the case of 4-type classification and 82.6&#37; of F-measure in the case of 8-type classification when learning and testing the same subjects data for only Right Wrist. Further, when adding data of sensors attached to the Waist and Right Ankle, we achieved 97.5&#37; of F-measure for 4-type classification and 91.4&#37; of F-measure for 8-type classification. As a result of leave-one-person-out cross-validation from 6 subjects to confirm generalized performance, in the case of 4-type classification, we achieved 77.5&#37; of F-measure by using only 2 IMUs (Right Wrist and Shinai Tsuba).

DOI： 10.1109/PERCOMW.2019.8730861

Language：English   Publishing type：Research paper (international conference proceedings)  

With the number of IoT devices expected to exceed 50 billion in 2023, edge and fog computing paradigms are beginning to attract attention as a way to process the massive amounts of raw data being generated. However, these paradigms do not consider the processing capabilities of the existing commodity IoT devices in the wild. In order to solve this challenge, we are developing a new middleware platform called IFoT, which processes various sensor data while considering Quality of Service (QoS) by utilizing the computational resources of heterogeneous IoT devices within an area. This allows smart services to be created and processed in parallel by various IoT devices. In this paper, we show the effectiveness of the IFoT based approach of constructing services. We designed and implemented a workspace context recognition service, utilizing environmental sensor data processed in a distributed manner according to the IFoT framework. We evaluate the QoS of IFoT middleware and its feasibility when used on commodity devices such as the Raspberry Pi, through the service.

DOI： 10.1109/PERCOMW.2019.8730693

Language：English   Publishing type：Research paper (scientific journal)  

Evacuation Center Determination Method Considering Bias of Congestion Degree on Areas

Language：English   Publishing type：Research paper (international conference proceedings)  

In an era where billions of IoT devices are deployed, edge/fog computing paradigms are attracting attention for their ability to reduce processing delays and mitigate waste of communication resources. However, since the computing system assumed by edge/fog paradigms have heterogeneity (in terms of the computing power of devices, network performance between devices, device density, etc.), provisioning computational resources according to computational demand becomes a challenging constrained optimization problem. In this paper, we propose in-situ resource provisioning method consisting of insitu resource area selection with adaptive scale out and in-situ task scheduling based on tabu search algorithm. We conducted a simulation study in a target regional area where 2,000 IoT devices and 10 IoT services are deployed to evaluate the effectiveness of the proposed algorithm. The simulation results show that our proposed algorithm can obtain higher user QoS compared to conventional resource provisioning algorithms.

DOI： 10.1109/IWQoS.2018.8624127

Language：English   Publishing type：Research paper (international conference proceedings)  

In an era where billions of IoT devices have been deployed, edge/fog computing paradigms are attracting attention for their ability to reduce processing delays and communication overhead. In order to improve Quality of Experience (QoE) of regional IoT services that create timely geo-spatial information in response to users’ queries, it is important to efficiently allocate sufficient resources based on the computational demand of each service. However since edge/fog devices are assumed to be heterogeneous (in terms of their computational power, network performance to other devices, deployment density, etc.), provisioning computational resources according to computational demand becomes a challenging constrained optimization problem. In this paper, we formulate a delay constrained regional IoT service provisioning (dcRISP) problem. dcRISP assigns computational resources of devices based on the demand of the regional IoT services in order to maximize users’ QoE. We also present dcRISP+, an extension of dcRISP, that enables resource selection to extend beyond the initial area in order to satisfy increasing computational demands. We propose a provisioning algorithm, in-situ resource area selection with adaptive scale out and in-situ task scheduling based on a tabu search, to solve the dcRISP+ problem. We conducted a simulation study of a tourist area in Kyoto where 4,000 IoT devices and 3 types of IoT services were deployed. Results show that our proposed algorithms can obtain higher user QoE compared to conventional resource provisioning algorithms.

DOI： 10.1109/SEC.2018.00022

Language：English   Publishing type：Research paper (international conference proceedings)  

Recently, there is a growing demand to know con- gestion information on sightseeing spots in real-time to provide a satisfactory tour plan to tourists. Many studies on a congestion estimation have been conducted so far. However, most of them suffer from high deployment/operation costs and/or rely on contributions by users with smartphones/sensors. In this paper, we propose a novel system that estimates congestion of sightseeing spots in real-time without attaching any device to tourists by observing the distribution of per-RSSI intensity occurrences in each time window when beacon signals are periodically sent between BLE (Bluetooth Low Energy) transceivers installed in sightseeing spots. In other words, our system can estimate the congestion degree in sightseeing spots simply by using the property of RSSI intensity which dramatically changes depending on the number of people. Therefore, the proposed system is simple and low cost and can estimate the congestion easily without any special devices attached to tourists. In the demonstration, we show the congestion degree in three levels (low, medium, and high) changing in real-time depending on the number of audience in the demonstration site.

DOI： 10.1109/PERCOMW.2018.8480395

Language：English   Publishing type：Research paper (international conference proceedings)  

A body weight training (BWT) means the training which utilizes the self-weight instead of the weight machine. The feedback of form and proper training menu recommendation is important for maximizing the effect of BWT. The objective of this study is to realize a novel support system which allows beginners to perform effective BWT alone, under wearable computing environment. To make an effective feedback, it is necessary to recognize BWT type with high accuracy. However, since the accuracy is greatly affected by the position of wearable sensors, we need to know the sensor position which achieves the high accuracy in recognizing the BWT type. We investigated 10 types BWT recognition accuracy for each sensor position. We found that waist is the best position when only 1 sensor is used. When 2 sensors are used, we found that the best combination is of waist and wrist. We conducted an evaluation experiment to show the effectiveness of sensor position. As a result of leave-one-person-out cross-validation from 13 subjects to confirm validity, we calculated the F-measure of 93.5&#37; when sensors are placed on both wrist and waist.

DOI： 10.1145/3267305.3267504

Language：English   Publishing type：Research paper (international conference proceedings)  

In this paper, we, Ubi-NUTS Japan, introduce a multi-stage activity inference method that can recognize a user's mode of locomotion and transportation using mobile device sensors. We use the Sussex-Huawei Locomotion-Transportation (SHL) dataset to tackle the SHL recognition challenge, where the goal is to recognize 8 modes of locomotion and transportation (still, walk, run, bike, car, bus, train, and subway) activities from the inertial sensor data of a smartphone. We adopt a multi-stage approach where the 8 class classification problem is divided into multiple sub-problems considering the similarity of each activity. Multimodal sensor data collected from a mobile phone are inferred using a proposed pipeline that combines feature extraction and 4 different types of classifiers generated using the random forest algorithm. We evaluated our method using data from over 271 hours of daily activities of 1 participant and the 5-fold cross-validation. Evaluation results demonstrate that our method clearly recognizes the 8 types of activities with an average F1-score of 97&#37;.

DOI： 10.1145/3267305.3267526

Language：English   Publishing type：Research paper (international conference proceedings)  

In this paper, we propose a battery-free place recognition system that utilizes solar-cells as a sensor for localization. Our system combines multiple solar cells having different characteristics against the light environment. As an initial work, we select five kinds of solar cells available in the market and investigate the characteristics of them. Then, we show a potential for estimating the place based on the variation of electricity amounts generated from solar cells. Finally, we show that our proposed system distinguish nine places with 88.0&#37; accuracy.

DOI： 10.1145/3267305.3267624

Language：English   Publishing type：Research paper (international conference proceedings)  

Information and communication technologies (ICTs) has enabled growth in developed countries and urban cities through improvements in communication systems, devices and applications. In rural areas, especially in developing countries, ICT penetration is not as high, often due to lack of available infrastructure and funding. With the increasing availability of Internet-of-Things (IoT) devices, low-cost large-scale deployments have become possible even in rural areas. We design, develop and implement, Near Cloud, a cloud-less platform that allows users and IoT devices to communicate and share information. This is built on top of a wireless mesh network (WMN) of low-cost, low-power IoT devices and deployed in areas where there is little to no Internet connectivity. To inject ICT and help bridge the digital divide in rural areas, Near Cloud provides functionalities such as web servers on nodes, accessibility to all users via Wi-Fi, and various data processing including image processing and machine learning. We will show applicability of Near Cloud in improving rural education, health care facilities, disaster response and agriculture.

DOI： 10.1109/COMPSAC.2018.10307

Language：English   Publishing type：Research paper (scientific journal)  

We propose a comprehensive sensing platform called SenStick, which is composed of hardware (ultra tiny all-in-one sensor board), software (iOS, Android, and PC), and 3D case data. The platform aims to allow all the researchers to start IoT research, such as activity recognition and context estimation, easily and efficiently. The most important contribution is the hardware that we have designed. Various sensors often used for research are embedded in an ultra tiny board with the size of 50 mm (W) × 10 mm (H) × 5 mm (D) and weight around 3 g including a battery. Concretely, the following sensors are embedded on this board: acceleration, gyro, magnetic, light, UV, temperature, humidity, and pressure. In addition, this board has BLE (Bluetooth low energy) connectivity and capability of a rechargeable battery. By using 110 mAh battery, it can run more than 15 hours. The most different point from other similar boards is that our board has a large flash memory for logging all the data without a smartphone. By using SenStick, all the users can collect various data easily and focus on IoT data analytics. In this paper, we introduce SenStick platform and some case studies. Through the user study, we confirmed the usefulness of our proposed platform.

DOI： 10.1155/2017/6308302

Language：English   Publishing type：Research paper (international conference proceedings)  

A System for Collecting and Curating Sightseeing Information toward Satisfactory Tour Plan Creation

Language：English   Publishing type：Research paper (international conference proceedings)  

In this paper, we propose a novel rapid prototyping platform called SenStick, which is composed of both hardware and software. The main purpose of our platform is to sensorize our personal belongings easily and smart for recognizing our living activities. The most impressive point is its size. The size is 75mm(W) x 10mm(H) x 5mm(D) and its weight is around 3 (g) including a battery. On this tiny board, 8 sensors (acceleration, gyro, magnetic, light, UV, temperature, humidity, and pressure), flash memory, BLE, and battery are embedded in high density. The battery life in stand-alone mode is more than 12 hours. Second interesting point is the support software for iOS and Android OS. It can monitor the sensing data as well as can record the ground truth video simultaneously and synchronously. Furthermore, 3D CAD data of various case designs will be open for users in SenStick community site. As a result, SenStick enables everyone to sense every activities easily and smart.

DOI： 10.1109/ICMU.2016.7742087

Language：English   Publishing type：Research paper (international conference proceedings)  

Thanks to rapid advance and penetration of IoT devices, it is becoming possible to sense almost every information of real-world. This urges us to utilize data streams continuously generated from IoT devices in real-Time. In this paper, aiming to locally process data streams by using computational resources of IoT devices, we propose middleware for IoT devices where the devices process data streams in real-Time and in a distributed manner. The proposed middleware provides four functions: (1) distribution of tasks issued by application software into sub-Tasks and distributed execution of the sub-Tasks over multiple IoT devices, (2) distribution of data streams over IoT devices, (3) real-Time analysis of the data streams, and (4) seamless integration of sensors and actuators. We have implemented a prototype of the proposed middleware for Raspberry Pi and show its basic performance.

DOI： 10.1109/ICDCSW.2016.37

Language：English   Publishing type：Research paper (international conference proceedings)  

In this paper, we report our efforts to recognize human context through the things sensorized by our sensor platform As you know, many things wearing on your body have alread become a sensor to understand our activities. Mos famous things are watches and glasses. In the future, w suppose that unwearable things, such as cutlery, sport gea and instruments, would be also a sensor for recognizing ou activities. Compared with our movements, the movement o unwearable things will have more variation. Also, suitabl sensor and proper algorithm would be different according t the things. To enhance the senserization of various things we have proposed a platform, called SenStick, that consist of ultra tiny all-in-one sensor board and sophisticate mobile app. If using it, every researcher can try embeddin various sensor into various things and can focus on dat analysis. We explain the possibility of smart experimen platform with introducing various sensorized things on th platform.

DOI： 10.1145/2968219.2968268

Language：English   Publishing type：Research paper (international conference proceedings)  

In this demo, we present a novel rapid prototyping platform called SenStick2, which is all-in-one sensor with wireless charging. The main purpose of our platform is to sensorize our personal belongings easily and smartly for recognizing our living activities. The size of SenStick2 hardware (board) is 50mm(W) 10mm(H) 5mm(D) and its weight is around 3 (g) including a battery. On this tiny board, 8 sensors (acceleration, gyro, magnetic, light, UV, temperature, humidity, and pressure), flash memory, BLE, and battery are embedded in high density. The battery life in stand-Alone mode is more than 15 hours. For SenStick platform, we have developed the support software for iOS and Android OS. It can monitor the sensing data and also record the ground truth video simultaneously and synchronously. Furthermore, 3D CAD data of various case designs will be open for users in SenStick community site. By providing 3D case data, all the researcher can measure every tiny activities in a daily life easily and smartly.

DOI： 10.1145/2968219.2971399

Language：English   Publishing type：Research paper (international conference proceedings)  

Smart Experiment Platform for Wearable Computing

Language：English   Publishing type：Research paper (international conference proceedings)  

EdgeComputing and Fog Computing are new paradigms where data processing is executed in or on the edge of networks to mitigate cloud server load. However, EdgeComputing and Fog Computing still need powerful servers on the edge of networks which impose additional costs for deployments. We proposed a platform called IFoT (Information Flow of Things) that efficiently performs distributed processing as well as distribution and analysis of data streams near their sources based on "Process On Our Own (PO3)" concept. In IFoT, processing of tasks for cloud servers is delegated to an ad-hoc distributed system consisting of proximity IoT devices for distributed real-time stream processing. In this demonstration, we show a face recognition system for person tracking developed on top of IFoT middleware which locally processes video streams in real-time and in a distributed manner by using computational resources of IoT devices.

DOI： 10.1109/ICDCS.2016.101

Language：English   Publishing type：Research paper (international conference proceedings)  

Sensor Network technologies are expected to be used in various kinds of application systems. Most of them, however, have been developed to work in a dedicated system. It is desired that various types of sensor networks could be operable in a same application system. To meet the requirement, we have studied middleware architecture named CMSAS (Common Middleware in Sensors, Actuators and Storages), which aims to realize virtual communications between various devices such as heterogeneous sensor networks, actuators and storages. In this paper, we propose the concept of CMSAS based on RT middleware, which was developed by National Institute of Advanced Industrial Science and Technology (AIST) to integrate components in robot technologies. We also discuss implementation design of CMSAS in wireless sensor networks to collect acceleration data.

DOI： 10.1145/2645884.2645888

Language：English   Publishing type：Research paper (international conference proceedings)  

DOI： 10.1117/12.2044929

In recent years, with the trend towards shorter working hours, the quality of meetings has increasingly impacted these hours. Consequently, meeting quality and value become more important. To achieve better results in limited time, everyone needs to conduct meetings effectively. This requires skills such as facilitating the meeting, listening to others' opinions, accurately expressing one's own views, and using body language. However, these skills are currently only correlated with meeting effectiveness and lack comprehensive qualitative and quantitative assessments, leaving specific methods for improvement unclear. Additionally, the type of meeting can lead to participant stress or disinterest, making it essential to understand their psychological safety and engagement for more effective meetings. Measuring these psychological states poses significant challenges due to privacy and compliance concerns, particularly when using cameras or wearable sensors. This paper broadly addresses these issues and reports on our approach to detecting non-verbal cues, specifically nodding, from chair movements as a potential solution.

researchmap

Since office work has become large-scale and diversified in companies or organizations, work engagement and efficiency have been always an important index of a team's or group's evaluation because it is directly connected to their outcomes. In order to identify the group work context, we first need to recognize for what and how long the individual members are spending their time at their desks, but without privacy concerns and underestimation of their actual work. In this paper, we propose and evaluate the base system of personal desk activity recognition by using a low-cost compact WiFi node and its WiFi channel state information (CSI), which can lead to a lightweight group work context identification system. As a result, we achieved 94.2% desk activity recognition accuracy using the on-desk receiver, in recognizing five different classes.

researchmap

Indoor activity recognition using WiFi sensing is expected to have a wide range of applications, such as monitoring the elderly and home security. The state of radio wave propagation is called Channel State Information (CSI) and can be obtained using specific devices. By collecting CSI and applying machine learning, it is possible to recognize activities. However, CSI is sensitive to changes in the environment, so whenever the arrangement of furniture or the layout of the room changes, it is necessary to re-collect sample data and retrain the model. Retraining a model requires annotation work, which is costly in terms of time and effort. To address this issue, this paper proposes an annotation system that uses backscatter tags to reduce the cost of data collection and model training. In this system, a backscatter tag that generates a frequency shift depending on its angle is attached to a person during data collection, and activity recognition is performed by detecting the presence of the frequency shift. The backscatter tag-based recognition results are then used as pseudo-ground truth for model update.

researchmap

Understanding user emotion is essential for Human-AI Interaction (HAI). Thus far, many approaches have been studied to recognize emotion from signals of various physiological modalities such as cardiac activity and skin conductance. However, little attention has been paid to the fact that physiological signals are influenced by and reflect various factors that have little or no association with emotion. While emotion is a cross-modal factor that triggers responses across multiple physiological modalities, features used in existing approaches also reflect modality-specific factors that affect only a single modality and have little association with emotion. To address this, we propose an approach to extract features that exclusively reflect cross-modal factors from multimodal physiological signals. Our approach introduces a multilayer RNN with two types of layers: multiple Modality-Specific Layers (MSLs) for modeling physiological activity in individual modalities and a single Cross-Modal Layer (CML) for modeling the process by which emotion affects physiological activity. By having all MSLs update their hidden states using the CML hidden states, our RNN causes the CML to learn cross-modal factors. Using real physiological signals, we confirmed that the features extracted by our RNN reflected emotions to a significantly greater extent than the features of existing approaches.

researchmap

Other Link： https://dblp.uni-trier.de/rec/conf/pricai/2023-1

There are two nasal pathways through which we perceive odors: 'orthonasal smell,' which comes from the tip of the nose, and 'retronasal smell,' which comes from the mouth and exits through the nose. In particular, the retronasal smell is known to be more critical in influencing perceived taste than the orthonasal smell. In this study, we propose a smart lid called Kaolid that changes the taste of beverages by presenting scent information as retronasal smell. Kaolid achieves retronasal smell by equipping a small, lightweight straw- or cup-type olfactory device with a sensor that senses drinking behavior and the amount of water in the cup. The straw-type device injects the scent into the tube through the middle of the straw, delivering the scent to the user's mouth simultaneously as the beverage is being consumed. The cup-type device injects the scent into the cup through the lid and delivers the scent to the user's mouth at the beginning of drinking to achieve in-mouth scenting. The concept of Kaolid is to promote water consumption, considered insipid, by altering the sense of taste through a scent presented when drinking the beverage.

researchmap

Other Link： https://dblp.uni-trier.de/rec/conf/percom/2023w

In this paper, we introduce Kaolid, an olfactory interface that uses a lid mechanism to augment the flavor of beverages by delivering scents as retronasal smell. Kaolid aims to promote the consumption of healthier beverages by intensifying their perceived taste through the release of scents during drinking. The system features a compact olfactory display and an IMU sensor, triggering scents in response to drinking movements. It comes in two models: A straw-Type for cold beverages and a cup-Type for hot drinks. We tested the interface using sparkling and hot water and measured its efficacy in enhancing perceived sweetness when paired with scents. Results showed significant enhancements in all evaluation metrics (taste satisfaction, perceived sweetness, and preference) with the straw-Type device. Notably, the perceived sweetness increased by an amount equivalent to about 2.88 grams of sugar when a retronasal smell was introduced compared to when no scent was present. This innovative interface holds promise in elevating the flavor of sugar-free drinks and could support those aiming to limit sugar consumption. Furthermore, this research contributes to the future of IoT systems for health support by harnessing the power of scent, opening avenues for novel approaches in sensory-driven well-being advancements.

researchmap

Other Link： https://dblp.uni-trier.de/rec/conf/iot/2023

The Internet is an effective tool for learners to gain new knowledge. Often, people use search engines (e.g., Google) rather than accessing websites directly. People have their search techniques to find specific information. In particular, people with domain knowledge tend to search more efficiently than novices. By understanding the gap between people with domain knowledge and novices, the novice can understand the path to becoming an expert. Therefore, in this study, we wanted to know what differences exist in search and programming behavior with and without domain knowledge. In this experiment, we asked a group with and without domain knowledge to solve ten programming problems and collected search logs (input knowledge) and compilation logs (output knowledge). Specifically, the first dataset consisted of 13 participants who had taken a university programming class. The second dataset consisted of 20 participants who had not taken a programming class and had no domain knowledge. We examined differences in search and compilation behavior based on participants' domain knowledge from this data. Since we observed a difference between each group when referring to the correlation coefficient, we performed a binary classification of novice and experienced participants using Random Forest, and achieved an average precision of 0.95, indicating that there were different trends in behavior with and without domain knowledge.

researchmap

Other Link： https://dblp.uni-trier.de/rec/conf/ithings/2022

Human activity recognition (HAR) using the deep learning method has caught the attention of researchers thanks to its automatic feature extraction and accurate prediction capabilities. However, for applications based on a wearable sensor, such as an inertial measurement unit (IMU), the process of collecting and hand-labeling large amounts of data is complicated and labor-intensive, meaning that there is a limited amount of data available for model training. Therefore, there is a need to propose and develop data augmentation approaches to generate high quality data for the growth of HAR research. We propose a head-mounted virtual gyroscope signal generator to alleviate the problems caused by the lack of data in head movement-related applications. Unlike previous work, our system only generates head-motion related gyroscope data, minimizing system complexity. We trained a deep-learning model in a head motion-based application with different generated sensor data ratios, and show the viability of our proposed data generation method.

researchmap

Other Link： https://dblp.uni-trier.de/rec/conf/icce-tw/2023

In this paper, we propose FedTour, a federated learning-based method for training tourism object recognition models, which utilizes short-distance direct communication between user devices and maximizes the model performance within a limited number of updates. In FedTour, whenever two user devices are within range, they first exchange metadata including the learning degree (e.g., recognition accuracy) of their models, and determine whether it is effective to integrate the peer model by using a regressor trained with various pairs of models with different accuracy to predict the accuracy of the merged model. Once it is deemed effective, the model parameters are exchanged and the model is updated using FedAvg (averaging weights of two models of user devices). By carefully setting the threshold of whether FedAvg is applied or not, model performance is improved within a limited number of model parameter exchanges resulting in lower power consumption of user devices. We conducted a simulation using mobile phone trace data of actual users in a real sightseeing area and evaluated the improvement in accuracy of a CNN model that recognizes 10 objects while limiting the number of model parameter exchanges to only 40. Results show FedTour increased the initial model accuracy by 112%, while the baseline gossip-based method achieved 69%.

researchmap

Other Link： https://dblp.uni-trier.de/rec/conf/percom/2022w

In many of the world's major cities, commuter trains provide vital transportation support and thus play an essential role in our daily lives. Therefore, it has become necessary to estimate the degree of congestion in each train car, both to improve passenger comfort levels and, more recently, to prevent worsening the COVID-19 pandemic infection rate. However, it is difficult to estimate the degree of congestion within a train without violating passenger privacy. The same issues are true for busses, which is noteworthy because we have previously developed and evaluated a system that can estimate the degree of congestion within a bus while protecting passenger privacy by using Bluetooth Low Energy (BLE) signals. In this paper, we report on our efforts to extend that system to railway use, which were conducted on actual trains in cooperation with Kintetsu Railway Co., Ltd. During this trial, we collected BLE signals and used the data to estimate congestion levels in each car using an ML regression model. The results show that the mean absolute error (MAE) and the mean absolute percentage error (MAPE) could be estimated at accuracy levels of 5.56 and 0.27, respectively.

researchmap

Federated learning has gained popularity as a distributed machine learning approach that provides security and privacy for data trained on local devices. However, vulnerabilities still exist in this approach, and common solutions such as encryption and blockchain techniques often suffer from high computation and communication costs, making them impractical for resource-constrained devices. To solve this problem, we propose a privacy-preserving federated learning system that leverages compressive sensing and differential privacy, specifically designed for devices with limited computational resources. In this paper, we demonstrate the capabilities of our proposed system in resource-limited environments. We outline the features, infrastructure, and algorithm of our proposed system, and simulate its performance using image datasets on a Raspberry Pi 4 and an Android smartphone in a cloud environment. Our approach offers a practical solution for secure and privacy-preserving federated learning in resource-constrained scenarios, with potential applications in various domains such as healthcare, IoT, and edge computing.

researchmap

Other Link： https://dblp.uni-trier.de/rec/conf/mdm/2023

Federated learning has gained popularity as a distributed machine learning approach that provides security and privacy for data trained on local devices. However, vulnerabilities still exist in this approach, and common solutions such as encryption and blockchain techniques often suffer from high computation and communication costs, making them impractical for resource-constrained devices. To solve this problem, we propose a privacy-preserving federated learning system that leverages compressive sensing and differential privacy, specifically designed for devices with limited computational resources. In this paper, we demonstrate the capabilities of our proposed system in resource-limited environments. We outline the features, infrastructure, and algorithm of our proposed system, and simulate its performance using image datasets on a Raspberry Pi 4 and an Android smartphone in a cloud environment. Our approach offers a practical solution for secure and privacy-preserving federated learning in resource-constrained scenarios, with potential applications in various domains such as healthcare, IoT, and edge computing.

researchmap

Other Link： https://dblp.uni-trier.de/rec/conf/mdm/2023

researchmap

Language：Japanese  

観光地において訪日観光客が求める情報に合わせて観光案内をするなど，訪日観光客の属性を判別して効果的に情報を発信できるサービスの実現が期待されている．特に，デジタルサイネージなどの時間に応じて表示コンテンツを変えられる媒体では，イベントや観光情報を周知する手段である言語情報をその利用者に応じて変えることは重要であると考える．これにより多様な地域から集まる観光地において，訪日観光客の国籍ごとに言語を変えることができ，販売促進やイベント周知を効果的に行えるようになる．本研究では，空港や駅などに設置してある無料 Wi-Fi が提供する DNS サーバから取得できる DNS クエリログを用いてリアルテイムに国籍を判定する手法を提案する．具体的には，DNS クエリログに含まれる問い合わせドメインから，各国を識別するためのドメインである ccTLD や，RIR に基づいて国コードを算出し，それらを特徴量として分析することで実現した．日本人 6 人，中国人 6 人，その他国籍 4 人に対して評価実験を行った結果，15 秒間のクエリログを使用して 61% の精度で国籍判定ができることを明らかにした．

researchmap

Many services based on human activity recognition (HAR) have been developed; however, user activity data include a large amount of private information. Although privacy protection is important in activity recognition, it has not been sufficiently explored. Therefore, we propose a privacy-preserving mechanism for HAR services that uses differential privacy. The proposed method reduces the user recognition accuracy to a level that satisfies the privacy requirements by adding weighted noise to the features in the learning model construction and then improves the activity recognition accuracy (service usefulness). The results indicate that when the privacy requirement is defined as less than the probability of a user being identified by chance, the proposed method improves the activity recognition accuracy by approximately 10 % compared to the conventional method.

researchmap

Other Link： https://dblp.uni-trier.de/rec/conf/percom/2023w

In this paper, we introduce the concept of a spoken dialogue system, which aims to recognize user eating behaviors during meals and promotes healthier eating habits by engaging users in an interactive co-dining experience. To provide an intuitive and more natural access, this information is integrated into a natural conversation with the user. Therefore, we conducted a preliminary study to explore how a human-like virtual avatar's eating speed affects the eating pace of a co-dining user. The results indicate that dining together with an avatar reduces the user's feeling of loneliness and enhances the overall satisfaction and enjoyment of the meal. Therefore, this work represents an important step toward realizing engaging co-dining experiences with human-like avatars and provides new insights into the future design of IoT systems to support solitary dining and health recommendations.

researchmap

Other Link： https://dblp.uni-trier.de/rec/conf/iot/2023

As the high-speed wireless network spreads, the usage of shared Wi-Fi in places such as offices, coworking spaces, and homes has been increased. Along with the development and diversification of digital content, the amount of data consumed by individual devices has also grown. In situations where multiple users share limited network resources, some content that consumes a large amount of bandwidth like video streaming can potentially degrade the Quality of Experience (QoE) for other users nearby. Consequently, a need for persuasive intervention systems that encourage prosocial behavior is rising taking into consideration the QoE of other network users. To address this issue, this study proposes a "Persuasive Public Wi-Fi"that employs Wi-Fi to incrementally convince users towards more considerate usage of network resources. To be specific, we suppose that the persuasive public Wi-Fi must include three different modes: 1) a mode of normal networking, 2) a mode that can intervene with individual users through a captive portal, and 3) a mode that intentionally limits bandwidth, i.e., Quality of Service (QoS) control. This work demonstrates the design and implementation of the proposed system and presents the results of operational verification using a prototype system.

researchmap

In this paper, we propose a method to measure human head motion, especially nodding, without attaching any sensors to the person. Our proposed system focuses on the fact that the upper body moves along with nodding and that the body motion slightly shakes the chair. We challenge the problem of whether it is possible to recognize a nodding from the extremely slight sway of a chair. To reveal the optimal position of sensors, we collected data by attaching multiple accelerometers to various positions on a chair, including the backrest, the seat's underside, and the legs. Using a supervised learning approach, we determined the best positions and combinations of sensors for recognizing nodding more collectively. The Support Vector Machine (SVM) achieved a nodding recognition accuracy of 0.990. Further testing of the accuracy of nodding frequency measurements resulted in an accuracy of 0.947, suggesting that the best position for the accelerometer is the backrest. These results suggest that simply placing the accelerometer on the backrest can effectively quantify the nod frequency of seated participants.

researchmap

Other Link： https://dblp.uni-trier.de/rec/conf/icmu/2023

In this study, we propose a behavior change support system that encourages people to use stairs instead of elevators. Using the stairs is not only good for one's health, but nowadays it also plays a role in dense avoidance from the perspective of COVID-19. Although many people are waiting for the elevator, we thought that if we knew how many people were on the elevator, more people could change their behavior. However, there are few buildings where the number of people in the elevator is displayed on each floor. So, we have developed a headcount measurement system in the cargo and a visualization system, those can be easily appended to the current elevator. As a method to measure the number of people in an elevator, we propose a method to detect BLE (Bluetooth Low Energy) signals transmitted from the terminals of elevator users who have installed COCOA, an application for confirming contact with the COVID-19 in Japan, and to measure in real time the number of detected BLE signals with a received signal strength (RSSI) exceeding a certain value. We propose a method to measure the number of BLE signals detected in real time. In this paper, we have designed and developed the system, verified the accuracy of the detection of the continuous operation time and the number of passengers, and estimated the behavior pattern and waiting time of elevator users using the system. The evaluation of the transformation of the decision making of the elevator users brought about by this system is out of the scope of this paper and will be discussed in the future.

researchmap

Other Link： https://dblp.uni-trier.de/rec/conf/persuasive/2022a

We present a secure decentralized learning framework suitable for resource-constrained devices within a cluster environment. Our approach focuses on enhancing privacy preservation during model aggregation by utilizing Differential Privacy. This technique adds random noise to gradients obtained from local training on edge devices before sending them for aggregation. This noise addition ensures that sensitive information within the gradients remains distorted, thus safeguarding user privacy. We showcase the implementation of our system on a cluster system employing Raspberry Pi 4 Model B devices, illustrating its feasibility and effectiveness in real-world scenarios. Through this demonstration, we highlight the practical applicability of our system in enabling secure decentralized learning within resource-constrained environments.

researchmap

In this demo, we will show our proposed system that can count nodding without either a camera or any sensor attached to the person. Our proposed system capitalizes on the fact that the upper body moves in conjunction with nodding and that this body motion slightly shakes the chair. We explore the challenge of recognizing nodding from the extremely subtle sway of a chair. To recognize nods in real-Time, we employed a supervised learning approach using acceleration data from sensors attached to the chair's backrest. Ultimately, the Support Vector Machine (SVM) achieved a nodding recognition accuracy of 0.990. Further testing of the accuracy of nodding frequency measurements yielded an accuracy of 0.947, suggesting that the optimal position for the accelerometer is the backrest. These results indicate that simply placing the accelerometer on the backrest can effectively quantify the frequency of nods from seated participants.

researchmap

Other Link： https://dblp.uni-trier.de/rec/conf/iot/2023

To promote the face-to-face communication reduced by COVID-19, we proposed and implemented a context-aware Slack chatbot based on Cyber-Physical sensing that helps colleagues meet more often in the same place. Our system periodically collects the user’s internal context through Slack (cyber sensing) and uses small BLE beacons distributed to colleagues and beacon scanners installed in a laboratory to sense physical attendance (physical sensing). In addition, the system notifies the user of recommended actions, such as lunch or coffee break, depending on the context determined by the Cyber-Physical sensors. We deployed the proposed system in a laboratory environment and conducted an initial experiment for six weeks. Experimental results confirmed that our system can encourage serendipitous face-to-face communication during periods when the frequency of attending school and going to work dropped due to COVID-19. It was also found that in an environment such as a laboratory, where a certain level of trust has already been established, the openness of the collected information can further motivate users to participate in the system.

researchmap

Other Link： https://dblp.uni-trier.de/rec/conf/persuasive/2022

As the Attention Economy has grown, numerous digital contents are made with the intention of capturing users' attention and spending more of their time by the use of attractive graphics and color psychology. In this paper, we propose a novel defense strategy, called a color-wall, to block digital distraction brought on by the consumption of visually seductive and addictive digital content. When it detects that a user is viewing media or using applications unrelated to the main task, it removes colors from the screen (gray-scaling) to make them less aesthetically pleasing and directs the user away from looking at them. We implemented a mac OS software as a proof-of-concept for the color-wall and conducted user testing (N=14) to assess the effectiveness of the proposed strategy. The evaluation results suggest that the color-wall is an effective strategy for reducing unwanted digital distractions such as browsing restricted content (YouTube, Twitter, etc.) while working on a PC.

researchmap

Other Link： https://dblp.uni-trier.de/rec/conf/huc/2022ap

In an era where digital technology profoundly influences our daily habits, we explore the potential of color psychology as a key element in the design of persuasive technology to promote health behavior change. This paper presents a strategy that uses color psychology to both encourage positive behaviors and discourage negative ones. We present two different applications: "eat2pic," a system designed to promote healthier eating habits by transforming meal selection into an engaging activity of adding color to images, and "color-wall," an application designed to reduce digital distractions by applying a grayscale filter to non-essential digital content. Through these demonstrations, we explore the efficacy and potential of color psychology in persuasive technology to influence user behavior in a health context and offer promising solutions to improve dietary choices and concentration while using computers.

researchmap

Language：Japanese  

本稿では，先行研究で提案したイヤラブルデバイスを用いた食事中の音から食品推定と咀嚼検出をシステムへの組み込み例として，我々の研究グループで提案している eat2pic への適用を紹介する．従来の eat2pic システムでは，センサを搭載した箸からユーザが一口ずつ何をどれくらいのスピードで食べたかを自動追跡し，食行動の良し悪しをキャンバス上に描かれた絵を用いてリアルタイムの視覚的フィードバックを提供する．しかし，eat2pic のセンシングアプローチは，カメラと IMU を搭載した専用の箸型センサを使用する必要があるため，日常生活での利便性や社会全体への普及可能性という観点で課題が残っている．そこで本稿では，センシングアプローチとして市販のワイヤレスイヤホンを用いた手法を提案する．提案手法では，市販のワイヤレスイヤホンとスマートフォンのみで完結するため，従来の eat2pic と比較して普及可能性の観点で大きく優れており，ユーザの噛む動作に応じて絵を描くという新たなインタラクションを実現することが可能となる．最近ではワイヤレスイヤホンで音楽を聴きながら食事を行う人も増えてきているため，ごく自然な形でセンシングを行うことができる．

researchmap

Language：English  

In this study, we develop a mobile application that collects push notifications from text messengers, social media, and email apps, then prioritizes the messages based on their sender, content, and application name using ChatGPT. By this application, users can check the messages in a single interface without opening multiple applications and displaying read receipts. Additionally, this application provides the capability to sort messages by priority, thus users can view more important content and filter advertisements. Moreover, this application supports content analyzation in multiple languages, allowing simultaneous prioritization of messages in different languages. We found that our application can give the higher priority to work-related messages, contrastively, the lower priority to advertisements.

researchmap

In this study, the target behavior of prosocial behavior was defined as garbage disposal (behavior of taking out the trash), and an intervention method using an inducement was proposed and evaluated. Specifically, we conducted an intervention in which the saturation of a trash can was visualized using a colored light that could be changed to red, yellow, and green. In addition, questionnaire surveys were administered before and after the intervention to 28 undergraduate and graduate students and faculty members, asking about their attitudes and feelings toward garbage disposal behavior, and personality traits. The results of a survey on the frequency of trash disposal be-fore and after the experiment showed that the time that the saturated trash can was left unattended was reduced by 81%. The questionnaire survey confirmed that the intervention increased the positive attitude toward trash disposal behavior, indicating that the intervention had an impact on trash disposal behavior. In addition, we were able to confirm the characteristics of individual personality traits and susceptibility to intervention.

researchmap

Other Link： https://dblp.uni-trier.de/rec/conf/persuasive/2023bcss

Language：Japanese  

加速度やジャイロセンサから取得できる人の行動データにはそのユーザのプライバシー情報を多く含んでいる．そのため，それらを特定しようとする外部の脅威から行動データを守る必要がある．また，プライバシー情報を守るためには，それらが推定できないような形式で利用・保存することが求められる．つまり，そのデータが利用されるアプリケーションでの機能を提供するための有用性は保ちつつ，プライバシーリスクを最小限にするためのデータ処理が求められる．そこで，本研究では，Adaptive Differential Pulse Code（ADPCM, 適応的差分パルス符号変調）とAnonymizing AutoEncoder（AAE, 匿名化オートエンコーダ）を組み合わせた行動データの匿名化手法であるCompressive-AAE（C-AAE）を提案し，実際に匿名化したデータのプライバシーリスクとアプリケーションでの有用性を評価した．結果としては先行研究のAAEと比較し，よりプライバシーリスクを抑えた上で精度を維持した行動認識を行うことに成功した．

researchmap

researchmap

To support the decision-making process for selecting upcoming tourist spots (PoI: Points of Interest), it is necessary to share the tourist destinations' current status. Methods for sharing the current status of PoIs could include installing live cameras or having tourists share real-Time photos. Still, there are challenges in terms of installation and maintenance costs and privacy concerns. In this paper, a tourism support system is proposed to aid in the decision-making of which PoI to visit next by automatically generating PoI current status images from template images and contextual information of tourist destinations while preserving privacy and presenting them to users. In the proposed system, semantic segmentation is used to divide template images into categories such as sky class, trees/grasses class, crowd class, and overall class. Based on the separately collected contextual information (weather, condition of trees/grass, crowd density, time, etc.), appropriate image transformations (using style transfer, etc) are applied to each category. The proposed system consists of an automated method for generating PoI status images and an application that presents these images to users. As part of a tourism experiment using the proposed system, 11 individuals aged 20 to 30 participated in a roughly 2-hour tour of Nara City in Japan. The results showed that compared to presenting template images, the proposed method yielded a decision-making score improvement of 0.27 and a similarity (to the current status) score improvement of 0.45 (out of five), both of which were statistically significant.

researchmap

Other Link： https://dblp.uni-trier.de/rec/conf/iot/2023

In this paper, we propose a method for estimating psychological states such as hesitation and interest during survey responses based on touch operation behavior (touch log). In addition, we propose two enhanced web survey RUIs (Response User Interfaces), (a) a slidebar type RUI and (b) a magnifier type RUI, to expand the operational differences due to hesitation and interest for more accurate estimation. To evaluate the proposed RUIs, we implemented Augmented Web Survey (AWS) as an extension of LimeSurvey and conducted two experiments. Responses were collected from 8 subjects in the experiment to evaluate (a) a slidebar type RUI and from 21 subjects to evaluate (b) a magnifier type RUI. We measured the degree of hesitation by evaluating the subjective hesitation to the questions on a 5-point scale. For the measurement of the degree of interest, the participants were asked to select their top two favorite images among the six images, and the true value was taken as the top two. As a result of comparing the operation logs of the conventional RUI and the slidebar type RUI, it was confirmed that there was a significant difference depending on the degree of hesitation. In addition, more magnification operations were obtained for the magnifier type RUI than the standard pinch-out operation suggesting that it may help estimate psychological states such as hesitation.

researchmap

Other Link： https://dblp.uni-trier.de/rec/conf/lifetech/2022

Language：Japanese  

普段何気なく飲んでいる飲料には多くの糖分が含まれている．砂糖入り飲料は私たちの生活の中で主な糖分摂取源となっており，過剰な糖分摂取は血糖値上昇や糖尿病リスクを上昇させることが知られている．そのため，糖分摂取量を低減する必要がある．これまで糖分摂取量を削減する方向に後押しする方法がいくつか提案されているが，短期的なアプローチに留まっており，習慣的に糖分が控えめな飲料の選択を促す有効的な解決策は見つかっていない．本研究では，飲むという行為に香りを付与することで，ユーザの知覚する甘さを増幅する「Aromug」を提案する．実際の糖分摂取量を低減しつつ，コンテキストアウェアな香り情報を提供することで，知覚される甘さを増幅し，日常的に飲料から摂取する糖分量を低減するシステムの実現を目指す．本稿では，アイスコーヒーと各種フレーバーの組み合わせの官能評価を行い，香りによって知覚する甘さ増幅の有効性を評価した．実験の結果，無糖コーヒーとチョコレートの香りの組み合わせは，香りなしの場合に比べ有意差が確認された．また味の甘さに関して，年齢やコーヒーを飲む頻度で嗜好に差が見られ，コンテキスト・アウェアな香り情報提示のための知見を得た．

researchmap

Reducing the amount of sugar consumed from daily beverages is essential for well-being. However, since a sudden reduction in intake is mentally painful for those who prefer sweetened beverages, solutions are needed to help them gradually become accustomed to non-sweetened drinks. In this paper, we propose Aromug, a mug-type olfactory interface that amplifies the user's perceived sweetness by providing a sweet scent along with the action of drinking. Through user studies, we investigated the effect of the information presented by Aromug on the taste of iced coffee. The experiment results showed that the sugar-free coffee and chocolate aroma might enhance the perceived sweetness compared to the no-scent case. In addition, it was suggested that there were differences in preference for sweetness depending on age and frequency of coffee drinking.

researchmap

Other Link： https://dblp.uni-trier.de/rec/conf/huc/2022ap

Due to the pandemic of COVID-19, product sampling at supermarkets has been restricted, and the disposal of unsold vegetables and other perishable foods has become a problem. To explore alternatives to product sampling, this study focuses on the “aroma” generated during product sampling and explores the effect of aroma nudges on purchasing behavior. Specifically, we focused on the scenario of promoting the purchase of Yamato-maru eggplant, a traditional vegetable of Nara Prefecture in Japan. We conducted an experiment in a supermarket for two months, from June to July of 2021, the season of the Yamato-maru eggplant. We compared the number of visits to the sales booth, the time spent in the booth, and the sales volume under four conditions: (1) no presentation, (2) presentation of paper media, (3) presentation of the paper and video media, and (4) presentation of a paper, video, and olfactory media. The experiment results suggest that the inclusion of aroma nudges has a positive potential to attract consumer interest and positively influence their purchasing decisions.

researchmap

Aiming to evaluate the concentration and engagement level in offline group discussion scenarios, we seek to capture the interaction information among meeting participants. In this paper, we focus on the facial orientation and present an end-To-end hybrid system capable of 2D overhead facial orientation estimation. Compared to common perspectives from which personal identity information is exposed, we explore approaches to perform overhead facial orientation estimation due to the privacy problem. The proposed system is a neural network-based architecture and composed of three modules. First we leverage and train from scratch the YOLOv8 model on the HollywoodHeads dataset, which allows full-perspective head detection in visual scenes. With the obtained head location, we then generate the corresponding head masks from the visual information within the bounding box. Finally, we fit them to ellipse so that 2D overhead facial orientation of all the meeting participants in the visual scenes can be estimated with the ellipse properties. Experiments demonstrate that our system achieves an MAE of 12.47 on the test dataset.

researchmap

Other Link： https://dblp.uni-trier.de/rec/conf/icmu/2023

researchmap

Other Link： https://dblp.uni-trier.de/rec/conf/mdm/2023

Federated learning provides techniques for training algorithms using mobile or decentralized devices, in contrast to traditional machine learning in which algorithm training is performed on centralized devices. In addition, federated learning provides privacy and security features, as the client and server do not share raw data, which may contain confidential information. A number of studies have shown, however, that using federated learning alone is not enough to protect data privacy in certain situations. To overcome this problem, differential privacy is proposed, which is a technique in which artificial noise is added to the raw data. By implementing this method, a high level of privacy protection can be obtained, however this added noise also reduces model accuracy. To address this issue, this paper proposes a new approach to implement differential privacy in federated learning using adaptive Gaussian clipping. We implemented the method by tightening the privacy budget, and introducing dynamic sampling probability, adaptive clipping based on hyperparameters, and a new privacy loss calculation. Our method's main objective is to adaptively change the amount of noise given to the model, thereby maximizing the model's accuracy performance, while maintaining privacy protection levels. Evaluation results show that our proposed method presents slightly better accuracy when compared to other existing differential privacy variants such as RDP, DP-SGD, and ZcDP, for both balanced (i.i.d.) and unbalanced datasets (non-i.i.d.), for a lower total communication cost than some variants.

researchmap

Other Link： https://dblp.uni-trier.de/rec/conf/mdm/2023

Language：English  

researchmap

Dense video captioning tasks are used to detect interesting events and provide descriptive text for these events from untrimmed videos. This technology has the potential to be used in security surveillance and human care applications. However, current methods often overlook the relationships between objects in the video, which limits their applicability and makes it challenging to adapt them to specific domains, such as video summarization for indoor human activities. In these scenarios, human activities are closely intertwined with the objects in the scene. In this paper, we propose a plug-and-play module designed to enhance existing dense video captioning methods with spatial information. Specifically, we extract spatial information about the interesting objects using Red-Green-Blue-Depth (RGB-D) images and the results of image segmentation. We then integrate this information into the captions generated by the Dense Video Captioning (DVC) method using a fine-tuned Large Language Model (LLM). We evaluate the performance of our model on a custom dataset and demonstrate that our system provides a convenient and effective approach for obtaining space-enhanced captions.

researchmap

In this paper, we present a point cloud streaming method of real-space objects such as humans and animals for real-time 3D reconstruction in VR space. The system uses a depth camera to scan a human or animal, divides the point cloud into parts of the body, and then controls the quality of the point cloud (i.e., resolution and frame rate) for each part in real-time according to the object's motion and context. This enables point cloud streaming with limited resources (computational and network resources) and maximizes the user's quality of experience (QoE). We have implemented and evaluated a series of systems incorporating the proposed method to enhance the user experience in realistic environments and scenarios while maintaining interactivity in VR-based online communication. The results show that the proposed system is feasible under resource-constrained environments without significantly affecting the user's QoE.

researchmap

Other Link： https://dblp.uni-trier.de/rec/conf/percom/2023w

Language：Japanese  

ナッジとは，人々の選択の自由を奪うことなく，特定の方向に微妙にかつ効果的に促すことを目的とした，行動の促進や意思決定の支援手段の総称である．その核心は，人々が常に合理的な選択をするわけではなく，しばしば予測可能な方法で非合理的な決定を下すという人間の行動の側面に着目している点にある．つまり，人々の思考のクセを捉えて，環境や情報提示の仕方を工夫することで，人々がより良い選択をするよう後押しすることが可能であると考えられてきた．HCIの分野では，このアイディアが早い段階から採用され，さまざまなインタラクティブシステムの研究開発に役立てられてきた．本稿では，HCI領域におけるナッジの活用事例を体系的にレビューした論文を取り上げ，特定された23種類のナッジ・メカニズムを概説する．

researchmap

J-GLOBAL

researchmap

Language：Japanese  

観光地において訪日観光客が求める情報に合わせて観光案内をするなど，訪日観光客の属性を判別して効果的に情報を発信できるサービスの実現が期待されている．特に，デジタルサイネージなどの時間に応じて表示コンテンツを変えられる媒体では，イベントや観光情報を周知する手段である言語情報をその利用者に応じて変えることは重要であると考える．これにより多様な地域から集まる観光地において，訪日観光客の国籍ごとに言語を変えることができ，販売促進やイベント周知を効果的に行えるようになる．本研究では，空港や駅などに設置してある無料 Wi-Fi が提供する DNS サーバから取得できる DNS クエリログを用いてリアルテイムに国籍を判定する手法を提案する．具体的には，DNS クエリログに含まれる問い合わせドメインから，各国を識別するためのドメインである ccTLD や，RIR に基づいて国コードを算出し，それらを特徴量として分析することで実現した．日本人 6 人，中国人 6 人，その他国籍 4 人に対して評価実験を行った結果，15 秒間のクエリログを使用して 61% の精度で国籍判定ができることを明らかにした．

CiNii Research

J-GLOBAL

researchmap

Language：Japanese  

カメラやマイクをミュートにしたオンライン会議で参加者の反応を知ることは困難である．対話相手からのフィードバックが少ない状況では，話者は話しづらさを感じ，聴者は会議への積極性が低下するといった課題がある．我々は，オンライン会議で不足している対話相手からのフィードバックを補填するために，参加者の表情認識から推定された感情を絵文字としてリアルタイムにフィードバックするシステム「REmotion」の開発を行っている．我々の目的は，本システムをオンライン会議で用いることで，話者の話しづらさを解消することと聴者の会議への積極性を向上させることである．そのために，参加者が対話相手を意識して表情を変化させることが重要だと考える．本論文では，開発したシステムを用いて実験を行い，参加者がカメラやマイクをミュートにして参加するオンライン会議で本システムを用いると，参加者が自分の表情を意識的に変化させることを明らかにした．

Language：English  

CiNii Research

researchmap

Language：English  

CiNii Research

researchmap

Language：Japanese  

オンライン購買システムは，ページ遷移やクリックの履歴から購買者が商品の購買に至るまでの過程を記録することができる．これにより，興味がある商品や比較検討した商品がわかり，購買者のニーズに近く購買意欲を高めるような商品推薦が可能である．一方，実店舗においては商品の購買以外の情報は，取得と活用の両面における難しさからあまり記録されていない．そのため，購買者がどのような商品に興味を持っており何で悩んでいるのかがわからないという問題がある．こうした問題を解決するために，我々はLiDAR（Light Detection And Ranging）からの点群データとDepth（深度）カメラからの視線情報を用いた購買時における候補商品推定システムを構築し，その初期評価を行った．推定の上では，興味のある商品を購買候補商品とした．評価の結果，購買時の視線は候補商品と強く結びつき，視線の情報は候補商品を推定する上で非常に重要な特徴となり得ることがわかった．一方で，視線の評価を踏まえ点群データを使用した候補商品の予測を行った結果，予測対象の人物を含んだ状態で学習したモデルは49&#37;と12クラス分類であることを考えると一定の精度を出すことができた．一方でLOPOでは精度が最大22&#37;と，モデルの一般化はまだまだ課題が残ることがわかった．今後は，データ数を増やし，モデルをより適したものに変更・改善していく必要がある．

Language：Japanese  

本研究では，行動変容を誘発する要素技術として，ゲーム等で利用される ''抽選'' がもたらす高揚感に着目し，ゲーム内で使用される抽選器を構成するどの要素が人の高揚感に影響を与えるかについて調査した．我々は，スロットを止めるボタンを押すという行為や手回しで抽選器を操作する行為が自己決定感を生み出し，さらに制御幻想を生じさせているのではないかと仮説を立て，抽選器として，手回し式の抽選器，自動でリールが停止するスロット，手動でリールを停止するスロットの 3 つの種類の抽選器を作成した．そして，実際のゲーム上で不特定多数の挙動を見るオンライン実験（アンケート回答者 428 名）と，表情筋や心拍など生理現象を定量的に計測するオフライン実験（被験者 1 名）を行った．その結果，スロットを自分で止めるという操作が制御幻想を生み出し，自己決定感が高揚感の向上に関係している可能性が示唆された．また，リーチによるニアミスなど当たりに近いハズレの存在も，ユーザの高揚感や表情の変化に影響を与える重要な要因であることが明らかとなった．

Language：Japanese  

IoT Cutting Board and Spatial Presentation Method for High-Reality Remote Co-Cooking

Language：Japanese  

Exploring the Effects of Aroma Nudges on Shopping Behavior in a Supermarket

J-GLOBAL

researchmap

Language：Japanese  

普段何気なく飲んでいる飲料には多くの糖分が含まれている．砂糖入り飲料は私たちの生活の中で主な糖分摂取源となっており，過剰な糖分摂取は血糖値上昇や糖尿病リスクを上昇させることが知られている．そのため，糖分摂取量を低減する必要がある．これまで糖分摂取量を削減する方向に後押しする方法がいくつか提案されているが，短期的なアプローチに留まっており，習慣的に糖分が控えめな飲料の選択を促す有効的な解決策は見つかっていない．本研究では，飲むという行為に香りを付与することで，ユーザの知覚する甘さを増幅する「Aromug」を提案する．実際の糖分摂取量を低減しつつ，コンテキストアウェアな香り情報を提供することで，知覚される甘さを増幅し，日常的に飲料から摂取する糖分量を低減するシステムの実現を目指す．本稿では，アイスコーヒーと各種フレーバーの組み合わせの官能評価を行い，香りによって知覚する甘さ増幅の有効性を評価した．実験の結果，無糖コーヒーとチョコレートの香りの組み合わせは，香りなしの場合に比べ有意差が確認された．また味の甘さに関して，年齢やコーヒーを飲む頻度で嗜好に差が見られ，コンテキスト・アウェアな香り情報提示のための知見を得た．

CiNii Research

J-GLOBAL

researchmap

Language：Japanese  

Language：Japanese  

Language：Japanese  

Language：Japanese  

Language：Japanese  

Language：Japanese  

Language：Japanese  

Language：Japanese  

Language：Japanese  

Language：Japanese  

Language：Japanese  

Language：Japanese  

Language：Japanese  

Language：Japanese  

Language：Japanese  

Language：Japanese  

Language：Japanese  

Language：Japanese  

Language：Japanese  

Language：Japanese  

Language：Japanese  

Promoting Installations of COCOA with Nugdes
The Ministry of Health, Labour and Welfare (MHLW) has released the COVID-19 Contact- Confirming Application (COCOA) to prevent the spread of COVID-19. However, it is currently underuti- lized and its effectiveness is questionable. Therefore, we propose the nudge system that promotes people's voluntary installation and aims to increase the utilization of COCOA. The proposed system motivates users to install COCOA by reframing the installation as the acquisition of tickets to experience three attractive events: (1) paint, (2) vote, and (3) fortune slip. In this paper, we propose the nudge system promoting the installation and assume scenarios for the use of the system, taking into account the appropriate environment for each event.

DOI： 10.11517/jsaisigtwo.2020.sai-039_10

Language：Japanese  

Language：Japanese  

Language：Japanese  

Language：Japanese  

Language：Japanese  

Language：Japanese  

Language：Japanese  

Language：Japanese  

Language：Japanese  

Language：Japanese  

Investigation of a new IoT Sensing Device for Rental Property Search Index

Language：Japanese  

Language：Japanese  

Language：Japanese  

Language：Japanese  

Language：Japanese  

Language：Japanese  

Language：Japanese  

Language：Japanese  

Language：Japanese  

Language：Japanese  

災害時の混雑情報を考慮した避難場所決定手法の提案

Language：Japanese  

災害時の混雑情報を考慮した避難所決定手法の提案,

Language：Japanese  

混雑情報を考慮した災害発生時の避難場所決定手法に関する一検討

Language：Japanese  

Language：Japanese  

観光動画キュレーションの実現に向けたハイライト抽出手法の検討

Language：Japanese  

Language：Japanese  

Language：Japanese  

Personalized Gamification on Location-based Service:
Practical Survey of Habituation and Weariness
through Stamp Rally Event

Language：Japanese  

IoTデータ流を実時間で分散処理するためのIoTデバイス向け共通ミドルウェアの設計と評価

Language：English  

Palaeotsunami deposits are the primary sources of information on past large tsunami events and thereby are critical for earthquake and tsunami hazard assessments. They usually form sandy layers preserved in coastal sediments and contain indicators of marine origins, such as microfossils (e.g., diatoms and foraminifera) and geochemical signals of saltwater. However, these indicators are often modified or erased with time. To address this issue, we present the first application of a palaeogenetic approach to a series of up to approximately 2000 year-old sandy palaeotsunami deposits from a coastal wetland on Hokkaido Island (Japan). We found the DNA of marine foraminiferal species preserved in palaeotsunami deposits, in the absence of foraminifera tests. The recovery and analysis of the ancient sedimentary DNA of marine organisms preserved in coastal zone sediments for millennia represents a new and promising tool for the identification of past tsunamis. (C) 2016 Elsevier B.V. All rights reserved.

DOI： 10.1016/j.margeo.2016.08.006

Language：Japanese  

Considering challenge of CGM curation for realtime trip planning system

Language：Japanese  

A Design of Dynamic Hazard Map Based on Real-Time Precipitation Information

Language：English  

Language：English  

Language：Japanese  

Language：Japanese  

Language：Japanese  

Language：Japanese  

Language：Japanese  

Research on the Geographically distributed Ambient interface utilizing local Information Flow of Things (GAIFoT)

Language：Japanese  

Implementation of Test-bed for Disaster Resilience Mesh-Network using NerveNet
Several types of systems for disaster communications and networks have been studied to comprehend damage situations immediately in a heavy disaster. There are, however, various issues in the systems to put into practical use in a society, e.g., expense for implementation and operation, or system operations in ordinary states. Especially, matching the results studied by research institutes and requirements of local governments is required. In this study, we introduced the disaster resilience mesh-networks: NerveNet, developed by the National Institute of Information and Communications Technology (NICT), to the education system of Hakodate National College of Technology (HNCT), and promote to implement the system in practical use by educational activities. We show a test-bed for the disaster resilience network, which was installed in HNCT and implemented the NerveNet. The test-bed mounts some applications for safety information in a disaster, file and message exchange functions, and delivering video images. Through this educational activity, we expect the institute is capable of supporting disaster communications and the damage assessment system.