工学部専攻教育科目「パターン認識」に対する表彰．

科研費の第２段審査（合議審査）に有意義な審査意見を付した第１段審査（書面審査）委員を選考し表彰

学術雑誌"Pattern Recognition Letters"の2008-2012年の査読委員の中で，最もクオリティが高い査読を行ったもの26名を表彰

研究又は産学官連携活動に関し九州大学の研究の活性化と財務上の貢献が特に顕著だったことに対する受賞

同会議におけるInvited Session "Document Analysis and Knowledge Science"の企画＆開催に対する授賞

論文発表「非マルコフ的制約を導入した最適弾性マッチング」に対する授賞．同シンポジウムは，画像認識における国内最高峰の学術集会．

第12回手書きパターン認識に関する国際会議（The 12th International Conference on Frontiers in Handwriting Recognition，ICFHR2010）において発表した次の論文に対する最優秀論文賞： Kazumasa Iwata, Koichi Kise, Masakazu Iwamura, Seiichi Uchida and Shinichiro Omachi, Tracking and Retrieval of Pen Tip Positions for an Intelligent Camera Pen

電子情報通信学会情報・システムソサイエティ論文誌の編集活動における査読者としての顕著な業績

第９回文書解析と認識に関する国際会議（The 9th International Conference on Document Analysis and Recognition，ICDAR2007）において発表した次の論文に対する最優秀論文賞： Seiichi Uchida, Megumi Sakai, Masakazu Iwamura, Shinichiro Omachi, Koichi Kise, Extraction of Embedded Class Information from Universal Character Pattern

1次元パターンの解析的DPマッチング

制約を緩めた弾性マッチングにおける固有変形利用の効果

カテゴリ毎の変形特性を組み込んだ弾性マッチングによる手書き文字認識

手書き文字の変形特性の抽出と利用,

区分線形2次元ワープを用いた画像中からの物体検出

Language：English  

DOI： 10.1002/anie.202404972

Web of Science

PubMed

Publishing type：Research paper (international conference proceedings)   Publisher：IEEE  

In pseudo-labeling (PL), which is a type of semi-supervised learning, pseudo-labels are assigned based on the confidence scores provided by the classifier; therefore, accurate confidence is important for successful PL. In this study, we propose a PL algorithm based on an energy-based model (EBM), which is referred to as the energy-based PL (EBPL). In EBPL, a neural network-based classifier and an EBM are jointly trained by sharing their feature extraction parts. This approach enables the model to learn both the class decision boundary and input data distribution, enhancing confidence calibration during network training. The experimental results demonstrate that EBPL outperforms the existing PL method in semi-supervised image classification tasks, with superior confidence calibration error and recognition accuracy.

DOI： 10.1109/ijcnn60899.2024.10650805

Web of Science

Scopus

researchmap

Publishing type：Research paper (international conference proceedings)   Publisher：IEEE  

Layout-aware text-to-image generation is a task to generate multi-object images that reflect layout conditions in addition to text conditions. The current layout-aware text-to-image diffusion models still have several issues, including mismatches between the text and layout conditions and quality degradation of generated images. This paper proposes a novel layout-aware text-to-image diffusion model called NoiseCollage to tackle these issues. During the denoising process, NoiseCollage independently estimates noises for individual objects and then crops and merges them into a single noise. This operation helps avoid condition mismatches; in other words, it can put the right objects in the right places. Qualitative and quantitative evaluations show that NoiseCollage outperforms several state-of-the-art models. These successful results indicate that the crop-and-merge operation of noises is a reasonable strategy to control image generation. We also show that NoiseCollage can be integrated with ControlNet to use edges, sketches, and pose skeletons as additional conditions. Experimental results show that this integration boosts the layout accuracy of ControlNet. The code is available at https://github.com/univ-esuty/noisecollage.

DOI： 10.1109/cvpr52733.2024.00852

Web of Science

Scopus

researchmap

Language：English   Publishing type：Research paper (scientific journal)   Publisher：Knee  

BACKGROUND: This study proposed an automatic surgical planning system for high tibial osteotomy (HTO) using deep learning-based artificial intelligence and validated its accuracy. The system simulates osteotomy and measures lower-limb alignment parameters in pre- and post-osteotomy simulations. METHODS: A total of 107 whole-leg standing radiographs were obtained from 107 patients who underwent HTO. First, the system detected anatomical landmarks on radiographs. Then, it simulated osteotomy and automatically measured five parameters in pre- and post-osteotomy simulation (hip knee angle [HKA], weight-bearing line ratio [WBL ratio], mechanical lateral distal femoral angle [mLDFA], mechanical medial proximal tibial angle [mMPTA], and mechanical lateral distal tibial angle [mLDTA]). The accuracy of the measured parameters was validated by comparing them with the ground truth (GT) values given by two orthopaedic surgeons. RESULTS: All absolute errors of the system were within 1.5° or 1.5%. All inter-rater correlation confidence (ICC) values between the system and GT showed good reliability (>0.80). Excellent reliability was observed in the HKA (0.99) and WBL ratios (>0.99) for the pre-osteotomy simulation. The intra-rater difference of the system exhibited excellent reliability with an ICC value of 1.00 for all lower-limb alignment parameters in pre- and post-osteotomy simulations. In addition, the measurement time per radiograph (0.24 s) was considerably shorter than that of an orthopaedic surgeon (118 s). CONCLUSION: The proposed system is practically applicable because it can measure lower-limb alignment parameters accurately and quickly in pre- and post-osteotomy simulations. The system has potential applications in surgical planning systems.

DOI： 10.1016/j.knee.2024.03.008

Web of Science

Scopus

PubMed

researchmap

Language：English   Publishing type：Research paper (scientific journal)   Publisher：Springer Science and Business Media LLC  

Abstract

Cell states are regulated by the response of signaling pathways to receptor ligand-binding and intercellular interactions. High-resolution imaging has been attempted to explore the dynamics of these processes and, recently, multiplexed imaging has profiled cell states by achieving a comprehensive acquisition of spatial protein information from cells. However, the specificity of antibodies is still compromised when visualizing activated signals. Here, we develop Precise Emission Canceling Antibodies (PECAbs) that have cleavable fluorescent labeling. PECAbs enable high-specificity sequential imaging using hundreds of antibodies, allowing for reconstruction of the spatiotemporal dynamics of signaling pathways. Additionally, combining this approach with seq-smFISH can effectively classify cells and identify their signal activation states in human tissue. Overall, the PECAb system can serve as a comprehensive platform for analyzing complex cell processes.

DOI： 10.1038/s41467-024-47989-9

Web of Science

Scopus

PubMed

researchmap

Other Link： https://www.nature.com/articles/s41467-024-47989-9

Language：English   Publisher：John Wiley & Sons Australia, Ltd  

潰瘍性大腸炎(UC)に対する既存の内視鏡スコアでは、内視鏡的所見の有無に基づき疾患の重症度を客観的に分類する。したがって、各カテゴリー内の疾患における重症度の差異を反映しない可能性がある。炎症性腸疾患(IBD)を専門とする内視鏡医は重症度を判定し、炎症の程度の全体的な印象を診断する。本研究の目的は、IBDを専門とする内視鏡医によるUCの内視鏡的重症度評価を正確に反映する人工知能(AI)システムを構築することである。ランキング畳み込みニューラルネットワーク(ranking-CNN)を、IBDを専門とする内視鏡医が作成した13826ペアの内視鏡画像のUC重症度に関する比較情報を用いて訓練した。訓練されたranking-CNNを用いて、UC内視鏡階調スケール(UCEGS)により重症度を判定した。AIによって診断されたUCEGSを用いた重症度評価と、Mayo内視鏡サブスコアとの間に不一致がないことを確認する為に、相関係数を計算した。4名のIBDを専門とする内視鏡医と、AIによるUCEGSを用いた画像評価の平均の相関係数も求めた。AIにより診断されたUCEGSとMayo内視鏡サブスコアとのSpearman相関係数は約0.89であった。IBD専門の内視鏡医とAIによる評価の結果の相関係数はすべて0.95超であった(P<0.01)。以上より、本研究で構築されたAIはUCの内視鏡的重症度をIBD専門の内視鏡医と同等に診断できることが示された。

DOI： 10.1075/itl.22018.uch

Web of Science

Language：English   Publishing type：Research paper (international conference proceedings)  

researchmap

Publisher：Proceedings - 2024 IEEE Winter Conference on Applications of Computer Vision, WACV 2024  

In this work, we consider the typography generation task that aims at producing diverse typographic styling for the given graphic document. We formulate typography generation as a fine-grained attribute generation for multiple text elements and build an autoregressive model to generate diverse typography that matches the input design context. We further propose a simple yet effective sampling approach that respects the consistency and distinction principle of typography so that generated examples share consistent typographic styling across text elements. Our empirical study shows that our model successfully generates diverse typographic designs while preserving a consistent typographic structure.

DOI： 10.1109/WACV57701.2024.00713

Web of Science

Scopus

Language：English   Publishing type：Research paper (other academic)  

Language：Others   Publishing type：Research paper (scientific journal)   Publisher：Springer Science and Business Media LLC  

Deep neural networks (DNNs) suffer from noisy-labeled data because of the risk of overfitting. To avoid the risk, in this paper, we propose a novel DNN training method with sample selection based on adaptive k-set selection, which selects k (< n, where n is the number of training samples) samples with a small noise-risk from the whole n noisy training samples at each epoch. It has the strong advantage of guaranteeing the performance of the selection theoretically. Roughly speaking, a regret, which is defined by the difference between the actual selection and the best selection, of the proposed method is theoretically bounded, even though the best selection is unknown until the end of all epochs. The experimental results on multiple noisy-labeled datasets demonstrate that our sample selection strategy works effectively in the DNN training; in fact, the proposed method achieved the best or the second-best performance among state-of-the-art methods, while requiring a significantly lower computational cost.

DOI： 10.1007/s10994-023-06478-8

Web of Science

Scopus

researchmap

Other Link： https://link.springer.com/article/10.1007/s10994-023-06478-8/fulltext.html

Publishing type：Part of collection (book)   Publisher：Springer Nature Switzerland  

This study analyzes the relationship between non-verbal information (e.g., genres) and text design (e.g., font style, character color, etc.) through the classification of book genres using text design on book covers. Text images have both semantic information about the word itself and other information (non-semantic information or visual design), such as font style, character color, etc. When we read a word printed on some materials, we receive impressions and other information from both the word itself and the visual design. In other words, we can understand verbal information from semantic information, i.e., the words themselves; however, we can consider that text design is helpful for understanding other additional information (i.e., non-verbal information), such as impressions, genre, etc. To investigate the effect of text design, we analyze text design using words printed on book covers and their genres in two scenarios. First, we attempted to understand the importance of visual design for determining the genre (i.e., non-verbal information) of books by analyzing the differences in the relationship between semantic information/visual design and genres. In the experiment, we found that semantic information is sufficient to determine the genre; however, text design is helpful in adding more discriminative features for book genres. Second, we investigated the effect of each text design on book genres. As a result, we found that each text design characterizes some book genres. For example, font style is useful to add more discriminative features for genres of “Mystery, Thriller & Suspense” and “Christian books & Bibles”.

DOI： 10.1007/978-3-031-70442-0_10

Web of Science

Scopus

researchmap

Publishing type：Part of collection (book)   Publisher：Springer Nature Switzerland  

Recent diffusion-based generative models show promise in their ability to generate text images, but limitations in specifying the styles of the generated texts render them insufficient in the realm of typographic design. This paper proposes a typographic text generation system to add and modify text on typographic designs while specifying font styles, colors, and text effects. The proposed system is a novel combination of two off-the-shelf methods for diffusion models, ControlNet and Blended Latent Diffusion. The former functions to generate text images under the guidance of edge conditions specifying stroke contours. The latter blends latent noise in Latent Diffusion Models (LDM) to add typographic text naturally onto an existing background. We first show that given appropriate text edges, ControlNet can generate texts in specified fonts while incorporating effects described by prompts. We further introduce text edge manipulation as an intuitive and customizable way to produce texts with complex effects such as “shadows” and “reflections”. Finally, with the proposed system, we successfully add and modify texts on a predefined background while preserving its overall coherence.

DOI： 10.1007/978-3-031-70536-6_4

Web of Science

Scopus

researchmap

Publishing type：Part of collection (book)   Publisher：Springer Nature Switzerland  

Kerning is the task of setting appropriate horizontal spaces for all possible letter pairs of a certain font. One of the difficulties of kerning is that the appropriate space differs for each letter pair. Therefore, for a total of 52 capital and small letters, we need to adjust 52×52=2704 different spaces. Another difficulty is that there is neither a general procedure nor criterion for automatic kerning; therefore, kerning is still done manually or with heuristics. In this paper, we tackle kerning by proposing two machine-learning models, called pairwise and set-wise models. The former is a simple deep neural network that estimates the letter space for two given letter images. In contrast, the latter is a transformer-based model that estimates the letter spaces for three or more given letter images. For example, the set-wise model simultaneously estimates 2704 spaces for 52 letter images for a certain font. Among the two models, the set-wise model is not only more efficient but also more accurate because its internal self-attention mechanism allows for more consistent kerning for all letters. Experimental results on about 2500 Google fonts and their quantitative and qualitative analyses show that the set-wise model has an average estimation error of only about 5.3 pixels when the average letter space of all fonts and letter pairs is about 115 pixels.

DOI： 10.1007/978-3-031-70536-6_2

Web of Science

Scopus

researchmap

Publishing type：Part of collection (book)   Publisher：Springer Nature Switzerland  

Fonts convey different impressions to readers. These impressions often come from the font shapes. However, the correlation between fonts and their impression is weak and unstable because impressions are subjective. To capture such weak and unstable cross-modal correlation between font shapes and their impressions, we propose Impression-CLIP, which is a novel machine-learning model based on CLIP (Contrastive Language-Image Pre-training). By using the CLIP-based model, font image features and their impression features are pulled closer, and font image features and unrelated impression features are pushed apart. This procedure realizes co-embedding between font image and their impressions. In our experiment, we perform cross-modal retrieval between fonts and impressions through co-embedding. The results indicate that Impression-CLIP achieves better retrieval accuracy than the state-of-the-art method. Additionally, our model shows the robustness to noise and missing tags.

DOI： 10.1007/978-3-031-70536-6_5

Web of Science

Scopus

researchmap

Publishing type：Part of collection (book)   Publisher：Springer Nature Switzerland  

Fonts have huge variations in their styles and give readers different impressions. Therefore, generating new fonts is worthy of giving new impressions to readers. In this paper, we employ diffusion models to generate new font styles by interpolating a pair of reference fonts with different styles. More specifically, we propose three different interpolation approaches, image-blending, condition-blending, and noise-blending, with the diffusion models. We perform qualitative and quantitative experimental analyses to understand the style generation ability of the three approaches. According to experimental results, three proposed approaches can generate not only expected font styles but also somewhat serendipitous font styles. We also compare the approaches with a state-of-the-art style-conditional Latin-font generative network model to confirm the validity of using the diffusion models for the style interpolation task.

DOI： 10.1007/978-3-031-70536-6_6

Web of Science

Scopus

researchmap

Publishing type：Part of collection (book)   Publisher：Springer Nature Switzerland  

This paper addresses the challenging task of estimating font impressions from real font images. We use a font dataset with annotation about font impressions and a convolutional neural network (CNN) framework for this task. However, impressions attached to individual fonts are often missing and noisy because of the subjective characteristic of font impression annotation. To realize stable impression estimation even with such a dataset, we propose an exemplar-based impression estimation approach, which relies on a strategy of ensembling impressions of exemplar fonts that are similar to the input image. In addition, we train CNN with synthetic font images that mimic scanned word images so that CNN estimates impressions of font images in the wild. We evaluate the basic performance of the proposed estimation method quantitatively and qualitatively. Then, we conduct a correlation analysis between book genres and font impressions on real book cover images; it is important to note that this analysis is only possible with our impression estimation method. The analysis reveals various trends in the correlation between them—this fact supports a hypothesis that book cover designers carefully choose a font for a book cover considering the impression given by the font.

DOI： 10.1007/978-3-031-70536-6_3

Web of Science

Scopus

researchmap

Publishing type：Part of collection (book)   Publisher：Springer Nature Switzerland  

The purpose of this paper is to enable the conversion between machine-printed character images (i.e., font images) and handwritten character images through machine learning. For this purpose, we propose a novel unpaired image-to-image domain conversion method, CycleDM, which incorporates the concept of CycleGAN into the diffusion model. Specifically, CycleDM has two internal conversion models that bridge the denoising processes of two image domains. These conversion models are efficiently trained without explicit correspondence between the domains. By applying machine-printed and handwritten character images to the two modalities, CycleDM realizes the conversion between them. Our experiments for evaluating the converted images quantitatively and qualitatively found that ours performs better than other comparable approaches.

DOI： 10.1007/978-3-031-70546-5_23

Web of Science

Scopus

researchmap

Publishing type：Research paper (international conference proceedings)   Publisher：Proceedings - International Symposium on Biomedical Imaging  

Diffusion models have recently been used for medical image generation because of their high image quality. In this study, we focus on generating medical images with ordinal classes, which have ordinal relationships, such as severity levels. We propose an Ordinal Diffusion Model (ODM) that controls the ordinal relationships of the estimated noise images among the classes. Our model was evaluated experimentally by generating retinal and endoscopic images of multiple severity classes. ODM achieved higher performance than conventional generative models by generating realistic images, especially in high-severity classes with fewer training samples.

DOI： 10.1109/ISBI56570.2024.10635504

Web of Science

Scopus

researchmap

Other Link： https://dblp.uni-trier.de/db/conf/isbi/isbi2024.html#TakezakiU24

DOI： 10.1007/978-3-031-72120-5_35

Web of Science

Language：Others   Publishing type：Research paper (scientific journal)   Publisher：Elsevier BV  

Signature verification requires high reliability. Especially in the writer-independent scenario with the skilled-forgery-only condition, achieving high reliability is challenging but very important. In this paper, we propose to apply two machine learning frameworks, learning with rejection and top-rank learning, to this task because they can suppress ambiguous results and thus give only reliable verification results. Since those frameworks accept a single input, we transform a pair of genuine and query signatures into a single feature vector, called Paired Contrastive Feature (PCF). PCF internally represents similarity (or discrepancy) between the two paired signatures; thus, reliable machine learning frameworks can make reliable decisions using PCF. Through experiments on three public signature datasets in the offline skilled-forgery-only writer-independent scenario, we evaluate and validate the effectiveness and reliability of the proposed models by comparing their performance with a state-of-the-art model.

DOI： 10.1016/j.patcog.2023.109816

Web of Science

Scopus

researchmap

Language：Others   Publishing type：Research paper (other academic)   Publisher：Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)  

A major approach for defending against adversarial attacks aims at controlling only image classifiers to be more resilient, and it does not care about visual objects, such as pandas and cars, in images. This means that visual objects themselves cannot take any defensive actions, and they are still vulnerable to adversarial attacks. In contrast, letters are artificial symbols, and we can freely control their appearance unless losing their readability. In other words, we can make the letters more defensive to the attacks. This paper poses three research questions related to the adversarial vulnerability of letter images: (1) How defensive are the letters against adversarial attacks? (2) Can we estimate how defensive a given letter image is before attacks? (3) Can we control the letter images to be more defensive against adversarial attacks? For answering the first and second questions, we measure the defensibility of letters by employing Iterative Fast Gradient Sign Method (I-FGSM) and then build a deep regression model for estimating the defensibility of each letter image. We also propose a two-step method based on a generative adversarial network (GAN) for generating character images with higher defensibility, which solves the third research question.

DOI： 10.1007/978-3-031-47634-1_9

Scopus

researchmap

Other Link： https://dblp.uni-trier.de/db/conf/acpr/acpr2023-1.html#KataokaKU23

Language：Others   Publishing type：Research paper (other academic)  

Publishing type：Research paper (international conference proceedings)  

researchmap

Other Link： https://dblp.uni-trier.de/rec/conf/bmvc/2023

Publishing type：Research paper (international conference proceedings)  

researchmap

Language：Others   Publishing type：Research paper (other academic)  

Language：English   Publishing type：Research paper (scientific journal)   Publisher：Digestive Endoscopy  

OBJECTIVES: Existing endoscopic scores for ulcerative colitis (UC) objectively categorize disease severity based on the presence or absence of endoscopic findings; therefore, it may not reflect the range of clinical severity within each category. However, inflammatory bowel disease (IBD) expert endoscopists categorize the severity and diagnose the overall impression of the degree of inflammation. This study aimed to develop an artificial intelligence (AI) system that can accurately represent the assessment of the endoscopic severity of UC by IBD expert endoscopists. METHODS: A ranking-convolutional neural network (ranking-CNN) was trained using comparative information on the UC severity of 13,826 pairs of endoscopic images created by IBD expert endoscopists. Using the trained ranking-CNN, the UC Endoscopic Gradation Scale (UCEGS) was used to express severity. Correlation coefficients were calculated to ensure that there were no inconsistencies in assessments of severity made using UCEGS diagnosed by the AI and the Mayo Endoscopic Subscore, and the correlation coefficients of the mean for test images assessed using UCEGS by four IBD expert endoscopists and the AI. RESULTS: Spearman's correlation coefficient between the UCEGS diagnosed by AI and Mayo Endoscopic Subscore was approximately 0.89. The correlation coefficients between IBD expert endoscopists and the AI of the evaluation results were all higher than 0.95 (P < 0.01). CONCLUSIONS: The AI developed here can diagnose UC severity endoscopically similar to IBD expert endoscopists.

DOI： 10.1111/den.14677

Web of Science

Scopus

PubMed

researchmap

Publisher：Pattern Recognition  

The authors regret to inform that: The FETNet results in Table 3 and Full model results in Table 2 on the SCUT-EnsText dataset should be 34.53(PSNR), 97.01(MSSIM), 0.0013(MSE), 1.7539(AGE), 0.0137(pEPs), 0.0080(pCEPs). The authors would like to apologize for any inconvenience caused.

DOI： 10.1016/j.patcog.2023.109581

Web of Science

Scopus

Language：Others   Publishing type：Research paper (scientific journal)  

DOI： 10.1016/j.patcog.2023.109531

Web of Science

researchmap

Language：Others   Publishing type：Research paper (other academic)   Publisher：Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)  

When we compare fonts, we often pay attention to styles of local parts, such as serifs and curvatures. This paper proposes an attention mechanism to find important local parts. The local parts with larger attention are then considered important. The proposed mechanism can be trained in a quasi-self-supervised manner that requires no manual annotation other than knowing that a set of character images are from the same font, such as Helvetica. After confirming that the trained attention mechanism can find style-relevant local parts, we utilize the resulting attention for local style-aware font generation. Specifically, we design a new reconstruction loss function to put more weight on the local parts with larger attention for generating character images with more accurate style realization. This loss function has the merit of applicability to various font generation models. Our experimental results show that the proposed loss function improves the quality of generated character images by several few-shot font generation models.

DOI： 10.1007/978-3-031-41501-2_17

Web of Science

Scopus

researchmap

Other Link： https://dblp.uni-trier.de/db/conf/icdar/icdar2023w-2.html#HaraguchiU23

Language：Others   Publishing type：Research paper (other academic)   Publisher：Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)  

In documents and graphics, contours are a popular format to describe specific shapes. For example, in the True Type Font (TTF) file format, contours describe vector outlines of typeface shapes. Each contour is often defined as a sequence of points. In this paper, we tackle the contour completion task. In this task, the input is a contour sequence with missing points, and the output is a generated completed contour. This task is more difficult than image completion because, for images, the missing pixels are indicated. Since there is no such indication in the contour completion task, we must solve the problem of missing part detection and completion simultaneously. We propose a Transformer-based method to solve this problem and show the results of the typeface contour completion.

DOI： 10.1007/978-3-031-41734-4_30

Web of Science

Scopus

researchmap

Other Link： https://dblp.uni-trier.de/db/conf/icdar/icdar2023-5.html#NagataIU23

Language：Others   Publishing type：Research paper (other academic)   Publisher：Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)  

There are various font styles in the world. Different styles give different impressions and readability. This paper analyzes the relationship between font styles and contextual factors that might affect font style selection with large-scale datasets. For example, we will analyze the relationship between font style and its surrounding object (such as “bus”) by using about 800,000 words in the Open Images dataset. We also use a book cover dataset to analyze the relationship between font styles with book genres. Moreover, the meaning of the word is assumed as another contextual factor. For these numeric analyses, we utilize our own font-style feature extraction model and word2vec. As a result of co-occurrence-based relationship analysis, we found several instances of specific font styles being used for specific contextual factors.

DOI： 10.1007/978-3-031-41682-8_21

Web of Science

Scopus

researchmap

Other Link： https://dblp.uni-trier.de/db/conf/icdar/icdar2023-3.html#YasukochiHHU23

Language：Others   Publishing type：Research paper (other academic)  

DOI： 10.1007/978-3-031-41682-8_20

Language：Others   Publishing type：Research paper (scientific journal)   Publisher：Elsevier BV  

Fruit quality is determined by multiple complex traits, which are difficult to diagnose by simple criteria and often require expert skills with a long experience. Nevertheless, current fruit sorting systems need a non-destructive, costless, and more rapid evaluation of fruit qualities. For peach, although many techniques have been developed for diagnosing fruit (internal) traits that determine commercial values, those techniques often require special facilities with high costs or take a long time for an assessment. Our study aimed to apply deep learning technology to evaluate multiple peach fruit traits using only simple RGB images for practical applications. We targeted seven fruit traits fundamentally involving commercial fruit quality; skin color, flesh firmness, sugar content, and four internal disorders, including colorless early softening, split-pit, watercore, and damage from peach fruit moth. We performed binary classification and regression analysis for these traits by convolutional neural networks (CNNs). Binary classification is performed to judge whether a fruit trait exceeds a threshold or not for a given image. Regression analysis is performed to estimate the degree of a trait quantitatively. Their results suggested that CNNs can successfully diagnose multiple fruit traits and predict quantitative values from RGB images. We also applied an explainable AI (X-AI) technique to spot the hypothetical symptoms for each trait on a fruit image, giving novel interpretations for physiological reactions associated with each fruit trait.

DOI： 10.1016/j.postharvbio.2023.112348

Web of Science

Scopus

researchmap

Language：Others   Publishing type：Research paper (other academic)   Publisher：IEEE  

This paper proposes a novel and efficient method for Learning from Label Proportions (LLP), whose goal is to train a classifier only by using the class label proportions of instance sets, called bags. We propose a novel LLP method based on an online pseudo-labeling method with regret minimization. As opposed to the previous LLP methods, the proposed method effectively works even if the bag sizes are large. We demonstrate the effectiveness of the proposed method using some benchmark datasets.

DOI： 10.1109/icassp49357.2023.10097069

Scopus

researchmap

Language：English   Publishing type：Research paper (scientific journal)   Publisher：Plant and Cell Physiology  

Deep neural network (DNN) techniques, as an advanced machine learning framework, have allowed various image diagnoses in plants, which often achieve better prediction performance than human experts in each specific field. Notwithstanding, in plant biology, the application of deep neural networks is still mostly limited to rapid and effective phenotyping. Recent development of explainable CNN frameworks has allowed visualization of the features in the prediction by convolutional neural network (CNN), which potentially contributes to the understanding of physiological mechanisms in objective phenotypes. In this study, we propose an integration of explainable CNN and transcriptomic approach to make a physiological interpretation of a fruit internal disorder in persimmon, rapid over-softening. We constructed CNN models to accurately predict the fate to be rapid softening in persimmon cv. Soshu, only with photo images. The explainable CNNs, such as Grad-CAM and Guided Grad-CAM, visualized specific featured regions relevant to the prediction of rapid-softening, which would correspond to the premonitory symptoms in a fruit. Transcriptomic analyses to compare the featured regions of predicted rapid-softening and control fruits suggested that rapid softening is triggered by precocious ethylene signal-dependent cell wall modification, despite exhibiting no direct phenotypic changes. Further transcriptomic comparison between the featured and non-featured regions in predicted rapid-softening fruit suggested that premonitory symptoms reflected hypoxia and the related stress signals finally to induce ethylene signals. These results would provide a good example for the collaboration of image analysis and omics approaches in plant physiology, which uncovered a novel aspect of fruit premonitory reactions in the rapid softening fate.

DOI： 10.1093/pcp/pcad050

Web of Science

Scopus

PubMed

researchmap

Language：Others   Publishing type：Research paper (other academic)   Publisher：Proceedings - International Symposium on Biomedical Imaging  

Disease Severity Regression with Continuous Data Augmentation

DOI： 10.1109/ISBI53787.2023.10230453

Web of Science

Scopus

researchmap

Other Link： https://dblp.uni-trier.de/db/conf/isbi/isbi2023.html#TakezakiTUK23

Language：Others   Publishing type：Research paper (scientific journal)   Publisher：Pattern Recognition  

Similarity measures for time series are important problems for time series classification. To handle the nonlinear time distortions, Dynamic Time Warping (DTW) has been widely used. However, DTW is not learnable and suffers from a trade-off between robustness against time distortion and discriminative power. In this paper, we propose a neural network model for task-adaptive time warping. Specifically, we use the attention model, called the bipartite attention model, to develop an explicit time warping mechanism with greater distortion invariance. Unlike other learnable models using DTW for warping, our model predicts all local correspondences between two time series and is trained based on metric learning, which enables it to learn the optimal data-dependent warping for the target task. We also propose to induce pre-training of our model by DTW to improve the discriminative power. Extensive experiments demonstrate the superior effectiveness of our model over DTW and its state-of-the-art performance in online signature verification.

DOI： 10.1016/j.patcog.2022.109201

Web of Science

Scopus

researchmap

Language：Others   Publishing type：Research paper (other academic)   Publisher：Proceedings - International Symposium on Biomedical Imaging  

Cluster-Guided Semi-Supervised Domain Adaptation for Imbalanced Medical Image Classification

DOI： 10.1109/ISBI53787.2023.10230451

Web of Science

Scopus

researchmap

Other Link： https://dblp.uni-trier.de/db/conf/isbi/isbi2023.html#HaradaBAYTKNAUU23

Language：Others   Publishing type：Part of collection (book)   Publisher：Springer Nature Switzerland  

This work presents a novel self-supervised representation learning method to learn efficient representations without labels on images from a 3DPM sensor (3-Dimensional Particle Measurement; estimates the particle size distribution of material) utilizing RGB images and depth maps of mining material on the conveyor belt. Human annotations for material categories on sensor-generated data are scarce and cost-intensive. Currently, representation learning without human annotations remains unexplored for mining materials and does not leverage on utilization of sensor-generated data. The proposed method, Depth Contrast, enables self-supervised learning of representations without labels on the 3DPM dataset by exploiting depth maps and inductive transfer. The proposed method outperforms material classification over ImageNet transfer learning performance in fully supervised learning settings and achieves an F1 score of 0.73. Further, The proposed method yields an F1 score of 0.65 with an 11% improvement over ImageNet transfer learning performance in a semi-supervised setting when only 20% of labels are used in fine-tuning. Finally, the Proposed method showcases improved performance generalization on linear evaluation. The implementation of proposed method is available on GitHub (https://github.com/prakashchhipa/Depth-Contrast-Self-Supervised-Method ).

DOI： 10.1007/978-3-031-25082-8_14

Scopus

researchmap

Language：English   Publishing type：Research paper (scientific journal)   Publisher：Lab on a Chip  

Here, we described a unique simultaneous electrorotation (ROT) device for monitoring the rotation rate of Jurkat cells via chemical stimulation without fluorescent labeling and an algorithm for estimating cell rotation rates. The device comprised two pairs of interdigitated array electrodes that were stacked orthogonally through a 20 μm-thick insulating layer with rectangular microwells. Four microelectrodes (two were patterned on the bottom of the microwells and the other two on the insulating layer) were arranged on each side of the rectangular microwells. The cells, which were trapped in the microwells, underwent ROT when AC voltages were applied to the four microelectrodes to generate a rotating electric field. These microwells maintained the cells even in fluid flows. Thereafter, the ROT rates of the trapped cells were estimated and monitored during the stimulation. We demonstrated the feasibility of estimating the chemical efficiency of cells by monitoring the ROT rates of the cells. After introducing a Jurkat cell suspension into the device, the cells were subjected to ROT by applying an AC signal. Further, the rotating cells were chemically stimulated by adding an ionomycin (a calcium ionophore)-containing aliquot. The ROT rate of the ionomycin-stimulated cells decreased gradually to 90% of the initial rate after 30 s. The ROT rate was reduced by an increase in membrane capacitance. Thus, our device enabled the simultaneous chemical stimulation-induced monitoring of the alterations in the membrane capacitances of many cells without fluorescent labeling.

DOI： 10.1039/d2lc00627h

Web of Science

Scopus

PubMed

researchmap

Language：English   Publishing type：Research paper (scientific journal)   Publisher：Scientific Reports  

Genome editing is a technology that can remarkably accelerate crop and animal breeding via artificial induction of desired traits with high accuracy. This study aimed to develop a chub mackerel variety with reduced aggression using an experimental system that enables efficient egg collection and genome editing. Sexual maturation and control of spawning season and time were technologically facilitated by controlling the photoperiod and water temperature of the rearing tank. In addition, appropriate low-temperature treatment conditions for delaying cleavage, shape of the glass capillary, and injection site were examined in detail in order to develop an efficient and robust microinjection system for the study. An arginine vasotocin receptor V1a2 (V1a2) knockout (KO) strain of chub mackerel was developed in order to reduce the frequency of cannibalistic behavior at the fry stage. Video data analysis using bioimage informatics quantified the frequency of aggressive behavior, indicating a significant 46% reduction (P = 0.0229) in the frequency of cannibalistic behavior than in wild type. Furthermore, in the V1a2 KO strain, the frequency of collisions with the wall and oxygen consumption also decreased. Overall, the manageable and calm phenotype reported here can potentially contribute to the development of a stable and sustainable marine product.

DOI： 10.1038/s41598-023-30259-x

Web of Science

Scopus

PubMed

researchmap

Web of Science

Language：Others   Publishing type：Research paper (other academic)   Publisher：IEEE  

This work presents a novel self-supervised pre-training method to learn efficient representations without labels on histopathology medical images utilizing magnification factors. Other state-of-the-art works mainly focus on fully supervised learning approaches that rely heavily on human annotations. However, the scarcity of labeled and unlabeled data is a long-standing challenge in histopathology. Currently, representation learning without labels remains unexplored in the histopathology domain. The proposed method, Magnification Prior Contrastive Similarity (MPCS), enables self-supervised learning of representations without labels on small-scale breast cancer dataset BreakHis by exploiting magnification factor, inductive transfer, and reducing human prior. The proposed method matches fully supervised learning state-of-the-art performance in malignancy classification when only 20% of labels are used in fine-tuning and outperform previous works in fully supervised learning settings for three public breast cancer datasets, including BreakHis. Further, It provides initial support for a hypothesis that reducing human-prior leads to efficient representation learning in self-supervision, which will need further investigation. The implementation of this work is available online on GitHub1.

DOI： 10.1109/wacv56688.2023.00274

Web of Science

Scopus

researchmap

Language：English  

DOI： 10.1016/j.heliyon.2023.e13059

Web of Science

PubMed

Publisher：Proceedings - International Conference on Image Processing, ICIP  

This work investigates the unexplored usability of self-supervised representation learning in the direction of functional knowledge transfer. In this work, functional knowledge transfer is achieved by joint optimization of self-supervised learning pseudo task and supervised learning task, improving supervised learning task performance. Recent progress in self-supervised learning uses a large volume of data, which becomes a constraint for its applications on small-scale datasets. This work shares a simple yet effective joint training framework that reinforces human-supervised task learning by learning self-supervised representations just-in-time and vice versa. Experiments on three public datasets from different visual domains, Intel Image, CIFAR, and APTOS, reveal a consistent track of performance improvements on classification tasks during joint optimization. Qualitative analysis also supports the robustness of learnt representations. Source code and trained models are available on GitHub1

DOI： 10.1109/ICIP49359.2023.10222142

Web of Science

Scopus

Web of Science

Publishing type：Research paper (scientific journal)  

DOI： 10.48550/arXiv.2310.14890

researchmap

Language：Others   Publishing type：Research paper (scientific journal)   Publisher：Arthritis Research and Therapy  

Background: X-ray images are commonly used to assess the bone destruction of rheumatoid arthritis. The purpose of this study is to propose an automatic-bone-destruction-evaluation system fully utilizing deep neural networks (DNN). This system detects all target joints of the modified Sharp/van der Heijde score (SHS) from a hand X-ray image. It then classifies every target joint as intact (SHS = 0) or non-intact (SHS ≥ 1). Methods: We used 226 hand X-ray images of 40 rheumatoid arthritis patients. As for detection, we used a DNN model called DeepLabCut. As for classification, we built four classification models that classify the detected joint as intact or non-intact. The first model classifies each joint independently, whereas the second model does it while comparing the same contralateral joint. The third model compares the same joint group (e.g., the proximal interphalangeal joints) of one hand and the fourth model compares the same joint group of both hands. We evaluated DeepLabCut’s detection performance and classification models’ performances. The classification models’ performances were compared to three orthopedic surgeons. Results: Detection rates for all the target joints were 98.0% and 97.3% for erosion and joint space narrowing (JSN). Among the four classification models, the model that compares the same contralateral joint showed the best F-measure (0.70, 0.81) and area under the curve of the precision-recall curve (PR-AUC) (0.73, 0.85) regarding erosion and JSN. As for erosion, the F-measure and PR-AUC of this model were better than the best of the orthopedic surgeons. Conclusions: The proposed system was useful. All the target joints were detected with high accuracy. The classification model that compared the same contralateral joint showed better performance than the orthopedic surgeons regarding erosion.

DOI： 10.1186/s13075-022-02914-7

Web of Science

Scopus

PubMed

researchmap

Language：Others   Publishing type：Research paper (other academic)   Publisher：IEEE  

Our goal is to generate fonts with specific impressions, by training a generative adversarial network with a font dataset with impression labels. The main difficulty is that font impression is ambiguous and the absence of an impression label does not always mean that the font does not have the impression. This paper proposes a font generation model that is robust against missing impression labels. The key ideas of the proposed method are (1) a co-occurrence-based missing label estimator and (2) an impression label space compressor. The first is to interpolate missing impression labels based on the co-occurrence of labels in the dataset and use them for training the model as completed label conditions. The second is an encoder-decoder module to compress the high-dimensional impression space into low-dimensional. We proved that the proposed model generates high-quality font images using multi-label data with missing labels through qualitative and quantitative evaluations. Our code is available at https://github.com/SeiyaMatsuda/Font-Generation-with-Missing-Impression-Labels.

DOI： 10.1109/icpr56361.2022.9956147

Web of Science

Scopus

researchmap

Language：Others   Publishing type：Research paper (other academic)   Publisher：IEEE  

A multi-layer image is more valuable than a single-layer image from a graphic designer's perspective. However, most of the proposed image generation methods so far focus on single-layer images. In this paper, we propose MontageGAN, which is a Generative Adversarial Networks (GAN) framework for generating multi-layer images. Our method utilized a two-step approach consisting of local GANs and global GAN. Each local GAN learns to generate a specific image layer, and the global GAN learns the placement of each generated image layer. Through our experiments, we show the ability of our method to generate multi-layer images and estimate the placement of the generated image layers.

DOI： 10.1109/icpr56361.2022.9956028

Web of Science

Scopus

researchmap

Language：Others   Publishing type：Research paper (other academic)   Publisher：Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)  

Automatic image-based disease severity estimation generally uses discrete (i.e., quantized) severity labels. Annotating discrete labels is often difficult due to the images with ambiguous severity. An easier alternative is to use relative annotation, which compares the severity level between image pairs. By using a learning-to-rank framework with relative annotation, we can train a neural network that estimates rank scores that are relative to severity levels. However, the relative annotation for all possible pairs is prohibitive, and therefore, appropriate sample pair selection is mandatory. This paper proposes a deep Bayesian active-learning-to-rank, which trains a Bayesian convolutional neural network while automatically selecting appropriate pairs for relative annotation. We confirmed the efficiency of the proposed method through experiments on endoscopic images of ulcerative colitis. In addition, we confirmed that our method is useful even with the severe class imbalance because of its ability to select samples from minor classes automatically.

DOI： 10.1007/978-3-031-12053-4_45

Web of Science

Scopus

researchmap

Other Link： https://dblp.uni-trier.de/db/conf/miua/miua2022.html#KadotaHBTU22

Language：Others   Publishing type：Research paper (scientific journal)   Publisher：Institute of Electrical and Electronics Engineers (IEEE)  

Lyric videos, or kinetic typography videos, are music videos showing lyric text in synchronization with the music. The purpose of this paper is to quantitatively and qualitatively analyze lyric videos to understand their design trends via three modalities: word motion, font style, and music style. These trends will not only be helpful as hints for designing new lyric videos but also be meaningful to quantitatively reveal the thought processes of the video design professionals. To achieve this, we needed to develop or utilize several technologies. First, we developed a lyric word tracking method to capture the motion of individual lyric words. The proposed method uses the lyric text as the guiding information for word tracking to overcome the difficulties arising from the various word appearances and motions. Second, we developed a font style estimator to quantify the appearance of each word as a feature vector. Finally, we employed a music style estimator to quantify the mood of the music, e.g., 'techno' and 'fast.' We then analyzed feature vectors of these three style modalities collected at 3,494 time points in 100 lyric videos. After revealing the trend of each modality via k-means, we conducted a co-occurrence analysis to understand the correlation between each modality pair. Our experimental results indicate that such a cluster-wise co-occurrence analysis can capture interesting trends hidden in lyric video designs.

DOI： 10.1109/access.2022.3184028

Web of Science

Scopus

researchmap

Language：Others   Publishing type：Part of collection (book)   Publisher：Springer International Publishing  

Styles of typefaces or fonts are often associated with specific impressions, such as heavy, contemporary, or elegant. This indicates that there are certain correlations between font shapes and their impressions. To understand the correlations, this paper constructs a shared latent space where a font and its impressions are embedded nearby. The difficulty is that the impression words attached to a font are often very noisy. This is because impression words are very subjective and diverse. More importantly, some impression words have no direct relevance to the font shapes and will disturb the construction of the shared latent space. We, therefore, use DeepSets for enhancing shape-relevant words and suppressing shape irrelevant words automatically while training the shared latent space. Quantitative and qualitative experimental results with a large-scale font-impression dataset demonstrate that the shared latent space by the proposed method describes the correlation appropriately, especially for the shape-relevant impression words.

DOI： 10.1007/978-3-030-98355-0_13

Web of Science

Scopus

researchmap

Language：English   Publishing type：Research paper (scientific journal)   Publisher：Plant Cell  

In the evolutionary history of plants, variation in cis-regulatory elements (CREs) resulting in diversification of gene expression has played a central role in driving the evolution of lineage-specific traits. However, it is difficult to predict expression behaviors from CRE patterns to properly harness them, mainly because the biological processes are complex. In this study, we used cistrome datasets and explainable convolutional neural network (CNN) frameworks to predict genome-wide expression patterns in tomato (Solanum lycopersicum) fruit from the DNA sequences in gene regulatory regions. By fixing the effects of trans-acting factors using single cell-type spatiotemporal transcriptome data for the response variables, we developed a prediction model for crucial expression patterns in the initiation of tomato fruit ripening. Feature visualization of the CNNs identified nucleotide residues critical to the objective expression pattern in each gene, and their effects were validated experimentally in ripening tomato fruit. This cis-decoding framework will not only contribute to the understanding of the regulatory networks derived from CREs and transcription factor interactions, but also provides a flexible means of designing alleles for optimized expression.

DOI： 10.1093/plcell/koac079

Web of Science

Scopus

PubMed

researchmap

Language：Others   Publishing type：Research paper (scientific journal)   Publisher：The Japanese Society for Horticultural Science  

<p>In contrast to the progress in the research on physiological disorders relating to shelf life in fruit crops, it has been difficult to non-destructively predict their occurrence. Recent high-tech instruments have gradually enabled non-destructive predictions for various disorders in some crops, while there are still issues in terms of efficiency and costs. Here, we propose application of a deep neural network (or simply deep learning) to simple RGB images to predict a severe fruit disorder in persimmon, rapid over-softening. With 1,080 RGB images of ‘Soshu’ persimmon fruits, three convolutional neural networks (CNN) were examined to predict rapid over-softened fruits with a binary classification and the date to fruit softening. All of the examined CNN models worked successfully for binary classification of the rapid over-softened fruits and the controls with > 80% accuracy using multiple criteria. Furthermore, the prediction values (or confidence) in the binary classification were correlated to the date to fruit softening. Although the features for classification by deep learning have been thought to be in a black box by conventional standards, recent feature visualization methods (or “explainable” deep learning) has allowed identification of the relevant regions in the original images. We applied Grad-CAM, Guided backpropagation, and layer-wise relevance propagation (LRP), to find early symptoms for CNNs classification of rapid over-softened fruits. The focus on the relevant regions tended to be on color unevenness on the surface of the fruit, especially in the peripheral regions. These results suggest that deep learning frameworks could potentially provide new insights into early physiological symptoms of which researchers are unaware.</p>

DOI： 10.2503/hortj.utd-323

Web of Science

Scopus

CiNii Research

researchmap

Language：Others   Publishing type：Part of collection (book)   Publisher：Springer International Publishing  

We propose TrueType Transformer (T 3 ), which can perform character and font style recognition in an outline format. The outline format, such as TrueType, represents each character as a sequence of control points of stroke contours and is frequently used in born-digital documents. T 3 is organized by a deep neural network, so-called Transformer. Transformer is originally proposed for sequential data, such as text, and therefore appropriate for handling the outline data. In other words, T 3 directly accepts the outline data without converting it into a bitmap image. Consequently, T 3 realizes a resolution-independent classification. Moreover, since the locations of the control points represent the fine and local structures of the font style, T 3 is suitable for font style classification, where such structures are very important. In this paper, we experimentally show the applicability of T 3 in character and font style recognition tasks, while observing how the individual control points contribute to classification results.

DOI： 10.1007/978-3-031-06555-2_2

Web of Science

Scopus

researchmap

Language：Others   Publishing type：Part of collection (book)   Publisher：Springer International Publishing  

Signature verification, as a crucial practical documentation analysis task, has been continuously studied by researchers in machine learning and pattern recognition fields. In specific scenarios like confirming financial documents and legal instruments, ensuring the absolute reliability of signatures is of top priority. In this work, we proposed a new method to learn “top-rank pairs” for writer-independent offline signature verification tasks. By this scheme, it is possible to maximize the number of absolutely reliable signatures. More precisely, our method to learn top-rank pairs aims at pushing positive samples beyond negative samples, after pairing each of them with a genuine reference signature. In the experiment, BHSig-B and BHSig-H datasets are used for evaluation, on which the proposed model achieves overwhelming better pos@top (the ratio of absolute top positive samples to all of the positive samples) while showing encouraging performance on both Area Under the Curve (AUC) and accuracy.

DOI： 10.1007/978-3-031-06555-2_22

Web of Science

Scopus

researchmap

Language：Others   Publishing type：Part of collection (book)   Publisher：Springer International Publishing  

Different fonts have different impressions, such as elegant, scary, and cool. This paper tackles part-based shape-impression analysis based on the Transformer architecture, which is able to handle the correlation among local parts by its self-attention mechanism. This ability will reveal how combinations of local parts realize a specific impression of a font. The versatility of Transformer allows us to realize two very different approaches for the analysis, i.e., multi-label classification and translation. A quantitative evaluation shows that our Transformer-based approaches estimate the font impressions from a set of local parts more accurately than other approaches. A qualitative evaluation then indicates the important local parts for a specific impression.

DOI： 10.1007/978-3-031-06555-2_1

Web of Science

Scopus

researchmap

Language：Japanese  

Language：English   Publishing type：Research paper (scientific journal)   Publisher：Journal of Cell Biology  

The epithelial cell sheet functions as a barrier to prevent invasion of pathogens. It is necessary to eliminate intercellular gaps not only at bicellular junctions, but also at tricellular contacts, where three cells meet, to maintain epithelial barrier function. To that end, tight junctions between adjacent cells must associate as closely as possible, particularly at tricellular contacts. Tricellulin is an integral component of tricellular tight junctions (tTJs), but the molecular mechanism of its contribution to the epithelial barrier function remains unclear. In this study, we revealed that tricellulin contributes to barrier formation by regulating actomyosin organization at tricellular junctions. Furthermore, we identified α-catenin, which is thought to function only at adherens junctions, as a novel binding partner of tricellulin. α-catenin bridges tricellulin attachment to the bicellular actin cables that are anchored end-on at tricellular junctions. Thus, tricellulin mobilizes actomyosin contractility to close the lateral gap between the TJ strands of the three proximate cells that converge on tricellular junctions.

DOI： 10.1083/jcb.202009037

Web of Science

Scopus

PubMed

researchmap

Language：Others   Publishing type：Research paper (scientific journal)   Publisher：IEEE Access  

For automatic disease-severity-level estimation, a large-scale medical image dataset with level annotations is generally necessary. However, attaching absolute-level annotations (such as levels 0, 1, and 3) is very costly and even inaccurate due to the level ambiguity. In this study, we proved experimentally that using a ranking function for level estimation can relax this difficulty. We propose a multi-task learning method for automatically estimating disease-severity levels that combine learning to rank with regression. The ranking function of the proposed method is trainable by relative-level and a small number of absolute-level annotations. For relative-level annotation, an annotator only needs to specify that one image has a higher disease level than another - this is much easier than absolute-level annotation. The proposed method enables disease-severity classification by calibrating the ranking function based on relative-level annotation through regression. The effectiveness of the method was proved through a large-scale experiment of ulcerative colitis-severity estimation with colonoscopy images.

DOI： 10.1109/ACCESS.2022.3155769

Web of Science

Scopus

researchmap

CiNii Research

Language：Others  

DOI： 10.1007/978-3-030-92238-2_50

Language：Others   Publishing type：Research paper (scientific journal)  

Top-rank learning identifies a real-valued ranking function that will provide more absolute top samples. These are highly reliable positive samples that are ranked higher than the highest-ranked negative samples. Therefore, top-rank learning is useful for tasks that require reliable decisions. Additionally, it inherits the merits of the ranking functions, such as robustness to the unbalanced condition. However, conventional top-rank learning tasks are formulated as linear or kernel-based problems and are thus limited in coping with complicated tasks. In this study, we propose a Top-rank convolutional neural network (TopRank CNN) to realize top-rank learning with representation learning for complicated tasks. Given that the original objective function of top-rank learning suffers from overfitting, we employ the p-norm relaxation of the original loss function in the proposed method. We prove the usefulness of TopRank CNN experimentally with medical diagnosis tasks that require reliable decisions and robustness to the unbalanced condition.

DOI： 10.1016/j.patcog.2021.108138

Language：English   Publishing type：Research paper (scientific journal)  

Epithelial barriers that prevent dehydration and pathogen invasion are established by tight junctions (TJs), and their disruption leads to various inflammatory diseases and tissue destruction. However, a therapeutic strategy to overcome TJ disruption in diseases has not been established because of the lack of clinically applicable TJ-inducing molecules. Here, we found TJ-inducing peptides (JIPs) in mice and humans that corresponded to 35 to 42 residue peptides of the C terminus of alpha 1-antitrypsin (A1AT), an acute-phase anti-inflammatory protein. JIPs were inserted into the plasma membrane of epithelial cells, which promoted TJ formation by directly activating the heterotrimeric G protein G13. In a mouse intestinal epithelial injury model established by dextran sodium sulfate, mouse or human JIP administration restored TJ integrity and strongly prevented colitis. Our study has revealed TJ-inducing anti-inflammatory physiological peptides that play a critical role in tissue repair and proposes a previously unidentified therapeutic strategy for TJ-disrupted diseases.

DOI： 10.1126/sciadv.abj6895

Language：Others   Publishing type：Research paper (other academic)  

DOI： 10.1109/iccv48922.2021.00111

Language：Others   Publishing type：Research paper (scientific journal)  

DOI： 10.1016/j.patcog.2021.108008

Language：Others   Publishing type：Research paper (other academic)  

Various fonts give different impressions, such as legible, rough, and comic-text. This paper aims to analyze the correlation between the local shapes, or parts, and the impression of fonts. By focusing on local shapes instead of the whole letter shape, we can realize more general analysis independent from letter shapes. The analysis is performed by newly combining SIFT and DeepSets, to extract an arbitrary number of essential parts from a particular font and aggregate them to infer the font impressions by nonlinear regression. Our qualitative and quantitative analyses prove that (1) fonts with similar parts have similar impressions, (2) many impressions, such as legible and rough, largely depend on specific parts, and (3) several impressions are very irrelevant to parts.

DOI： 10.1007/978-3-030-86334-0_47

Language：Others   Publishing type：Research paper (other academic)  

Fonts have had trends throughout their history, not only in when they were invented but also in their usage and popularity. In this paper, we attempt to specifically find the trends in font usage using robust regression on a large collection of text images. We utilize movie posters as the source of fonts for this task because movie posters can represent time periods by using their release date. In addition, movie posters are documents that are carefully designed and represent a wide range of fonts. To understand the relationship between the fonts of movie posters and time, we use a regression Convolutional Neural Network (CNN) to estimate the release year of a movie using an isolated title text image. Due to the difficulty of the task, we propose to use of a hybrid training regimen that uses a combination of Mean Squared Error (MSE) and Tukey’s biweight loss. Furthermore, we perform a thorough analysis on the trends of fonts through time.

DOI： 10.1007/978-3-030-86159-9_9

Language：Others   Publishing type：Research paper (other academic)  

Book covers are intentionally designed and provide an introduction to a book. However, they typically require professional skills to design and produce the cover images. Thus, we propose a generative neural network that can produce book covers based on an easy-to-use layout graph. The layout graph contains objects such as text, natural scene objects, and solid color spaces. This layout graph is embedded using a graph convolutional neural network and then used with a mask proposal generator and a bounding-box generator and filled using an object proposal generator. Next, the objects are compiled into a single image and the entire network is trained using a combination of adversarial training, perceptual training, and reconstruction. Finally, a Style Retention Network (SRNet) is used to transfer the learned font style onto the desired text. Using the proposed method allows for easily controlled and unique book covers.

DOI： 10.1007/978-3-030-86334-0_42

Language：Others   Publishing type：Research paper (other academic)  

Ulcerative colitis (UC) classification, which is an important task for endoscopic diagnosis, involves two main difficulties. First, endoscopic images with the annotation about UC (positive or negative) are usually limited. Second, they show a large variability in their appearance due to the location in the colon. Especially, the second difficulty prevents us from using existing semi-supervised learning techniques, which are the common remedy for the first difficulty. In this paper, we propose a practical semi-supervised learning method for UC classification by newly exploiting two additional features, the location in a colon (e.g., left colon) and image capturing order, both of which are often attached to individual images in endoscopic image sequences. The proposed method can extract the essential information of UC classification efficiently by a disentanglement process with those features. Experimental results demonstrate that the proposed method outperforms several existing semi-supervised learning methods in the classification task, even with a small number of annotated images.

DOI： 10.1007/978-3-030-87196-3_44

Language：Others   Publishing type：Research paper (other academic)  

In convolutional neural network-based character recognition, pooling layers play an important role in dimensionality reduction and deformation compensation. However, their kernel shapes and pooling operations are empirically predetermined; typically, a fixed-size square kernel shape and max pooling operation are used. In this paper, we propose a meta-learning framework for pooling layers. As part of our framework, a parameterized pooling layer is proposed in which the kernel shape and pooling operation are trainable using two parameters, thereby allowing flexible pooling of the input data. We also propose a meta-learning algorithm for the parameterized pooling layer, which allows us to acquire a suitable pooling layer across multiple tasks. In the experiment, we applied the proposed meta-learning framework to character recognition tasks. The results demonstrate that a pooling layer that is suitable across character recognition tasks was obtained via meta-learning, and the obtained pooling layer improved the performance of the model in both few-shot character recognition and noisy image recognition tasks.

DOI： 10.1007/978-3-030-86334-0_13

Language：Others   Publishing type：Research paper (other academic)  

Various fonts give us various impressions, which are often represented by words. This paper proposes Impressions2Font (Imp2Font) that generates font images with specific impressions. Imp2Font is an extended version of conditional generative adversarial networks (GANs). More precisely, Imp2Font accepts an arbitrary number of impression words as the condition to generate the font images. These impression words are converted into a soft-constraint vector by an impression embedding module built on a word embedding technique. Qualitative and quantitative evaluations prove that Imp2Font generates font images with higher quality than comparative methods by providing multiple impression words or even unlearned words.

DOI： 10.1007/978-3-030-86334-0_48

Language：Others   Publishing type：Research paper (other academic)  

When fonts are used on documents, they are intentionally selected by designers. For example, when designing a book cover, the typography of the text is an important factor in the overall feel of the book. In addition, it needs to be an appropriate font for the rest of the book cover. Thus, we propose a method of generating a book title image based on its context within a book cover. We propose an end-to-end neural network that inputs the book cover, a target location mask, and a desired book title and outputs stylized text suitable for the cover. The proposed network uses a combination of a multi-input encoder-decoder, a text skeleton prediction network, a perception network, and an adversarial discriminator. We demonstrate that the proposed method can effectively produce desirable and appropriate book cover text through quantitative and qualitative results. The code can be found at https://github.com/Taylister/FontFits.

DOI： 10.1007/978-3-030-86334-0_37

Language：Others   Publishing type：Research paper (other academic)  

This paper analyzes a large number of logo images from the LLD-logo dataset, by recent deep learning-based techniques, to understand not only design trends of logo images and but also the correlation to their owner company. Especially, we focus on three correlations between logo images and their text areas, between the text areas and the number of followers on Twitter, and between the logo images and the number of followers. Various findings include the weak positive correlation between the text area ratio and the number of followers of the company. In addition, deep regression and deep ranking methods can catch correlations between the logo images and the number of followers.

DOI： 10.1007/978-3-030-86198-8_8

Language：Others   Publishing type：Research paper (other academic)  

Deep time series metric learning is challenging due to the difficult trade-off between temporal invariance to nonlinear distortion and discriminative power in identifying non-matching sequences. This paper proposes a novel neural network-based approach for robust yet discriminative time series classification and verification. This approach adapts a parameterized attention model to time warping for greater and more adaptive temporal invariance. It is robust against not only local but also large global distortions, so that even matching pairs that do not satisfy the monotonicity, continuity, and boundary conditions can still be successfully identified. Learning of this model is further guided by dynamic time warping to impose temporal constraints for stabilized training and higher discriminative power. It can learn to augment the inter-class variation through warping, so that similar but different classes can be effectively distinguished. We experimentally demonstrate the superiority of the proposed approach over previous non-parametric and deep models by combining it with a deep online signature verification framework, after confirming its promising behavior in single-letter handwriting classification on the Unipen dataset.

DOI： 10.1007/978-3-030-86334-0_23

Language：Others   Publishing type：Research paper (scientific journal)  

DOI： 10.1016/j.media.2021.102097

Language：English   Publishing type：Research paper (scientific journal)  

In recent times, deep artificial neural networks have achieved many successes in pattern recognition. Part of this success can be attributed to the reliance on big data to increase generalization. However, in the field of time series recognition, many datasets are often very small. One method of addressing this problem is through the use of data augmentation. In this paper, we survey data augmentation techniques for time series and their application to time series classification with neural networks. We propose a taxonomy and outline the four families in time series data augmentation, including transformation-based methods, pattern mixing, generative models, and decomposition methods. Furthermore, we empirically evaluate 12 time series data augmentation methods on 128 time series classification datasets with six different types of neural networks. Through the results, we are able to analyze the characteristics, advantages and disadvantages, and recommendations of each data augmentation method. This survey aims to help in the selection of time series data augmentation for neural network applications.

DOI： 10.1371/journal.pone.0254841

Language：Others   Publishing type：Research paper (scientific journal)  

DOI： 10.1109/ACCESS.2021.3096530

Language：Others   Publishing type：Research paper (other academic)  

Person re-identification is a challenging task aiming to identify the same person across different cameras. However, most of existing image-based person re-identification methods neglect the spatial and temporal constraint, the information in neighbor frames of each person image is rarely exploited by previous studies. In this paper, we propose a novel neighbor frames mining framework (NFM) to exploit the spatial-temporal information. For each gallery image, we use a dynamic programming-based global optimal tracking method to search images of the same person in its neighbor frames. From those images, the image features extracted by the shared convolutional neural network (CNN) in the constructed neighbor sequence are merged via an attention weighted averaging technology. To this end, a novel supervised attention mechanism is designed for dealing with tracking errors. The final feature with multi-view and robust information is used for matching. Experimental results show the superiority and efficiency of the proposed method on two benchmark datasets including DukeMTMC-reID and PRW.

DOI： 10.1007/978-3-030-78811-7_37

Language：Others   Publishing type：Research paper (other academic)  

The interpretability of neural networks (NNs) is a challenging but essential topic for transparency in the decision-making process using machine learning. One of the reasons for the lack of interpretability is random weight initialization, where the input is randomly embedded into a different feature space in each layer. In this paper, we propose an interpretation method for a deep multilayer perceptron, which is the most general architecture of NNs, based on identity initialization (namely, initialization using identity matrices). The proposed method allows us to analyze the contribution of each neuron to classification and class likelihood in each hidden layer. As a property of the identity-initialized perceptron, the weight matrices remain near the identity matrices even after learning. This property enables us to treat the change of features from the input to each hidden layer as the contribution to classification. Furthermore, we can separate the output of each hidden layer into a contribution map that depicts the contribution to classification and class likelihood, by adding extra dimensions to each layer according to the number of classes, thereby allowing the calculation of the recognition accuracy in each layer and thus revealing the roles of independent layers, such as feature extraction and classification.

DOI： 10.1109/icassp39728.2021.9414873

Language：Others   Publishing type：Research paper (other academic)  

Oftentimes, patterns can be represented through different modalities. For example, leaf data can be in the form of images or contours. Handwritten characters can also be either online or offline. To exploit this fact, we propose the use of self-augmentation and combine it with multi-modal feature embedding. In order to take advantage of the complementary information from the different modalities, the self-augmented multi-modal feature embedding employs a shared feature space. Through experimental results on classification with online handwriting and leaf images, we demonstrate that the proposed method can create effective embeddings.

DOI： 10.1109/icassp39728.2021.9413974

Language：Others   Publishing type：Research paper (other academic)  

Language：Japanese  

AIとは何か? : 入門編: トクシュウ サア 、 AI オ ハジメヨウ : ドボク コウガク エ ノ AI ドウニュウ ノ ススメ

Language：Others   Publishing type：Research paper (scientific journal)  

Noninvasive diagnosis of internal traits in fruit crops is a high unmet need; however it generally requires time, costs, and special methods or facilities. Recent progress in deep neural network (or deep learning) techniques would allow easy, but highly accurate diagnosis with single RGB images, and the latest applications enable visualization of “the reasons for each diagnosis” by backpropagation of neural networks. Here, we propose an application of deep learning for image diagnosis on the classification of internal fruit traits, in this case seedlessness, in persimmon fruit (Diospyros kaki). We examined the classification of seedlessness in persimmon fruit by using four convolutional neural networks (CNN) models with various layer structures. With only 599 pictures of ‘Fuyu’ persimmon fruit from the fruit apex side, the neural networks successfully made a binary classification of seedless and seeded fruits with up to 85% accuracy. Among the four CNN models, the VGG16 model with the simplest layer structure showed the highest classification accuracy of 89%. Prediction values for the binary classification of seeded fruits were significantly increased in proportion to seed numbers in all four CNN models. Furthermore, explainable AI methods, such as Gradient-weighted Class Activation Mapping (Grad-CAM) and Guided Grad-CAM, allowed visualization of the parts and patterns contributing to the diagnosis. The results indicated that finer positions surrounding the apex, which correspond to hypothetical bulges derived from seeds, are an index for seeded fruits. These results suggest the novel potential of deep learning for noninvasive diagnosis of fruit internal traits using simple RGB images and also provide novel insights into previously unrecognized features of seeded/seedless fruits.

DOI： 10.2503/hortj.UTD-248

Language：Others   Publishing type：Research paper (scientific journal)  

DOI： 10.1016/j.patcog.2020.107577

Language：Others   Publishing type：Research paper (other academic)  

DOI： 10.1109/icpr48806.2021.9412545

Language：Others   Publishing type：Research paper (other academic)  

DOI： 10.1109/icpr48806.2021.9412812

Language：English  

DOI： 10.1038/s41467-020-20826-5

Language：Others   Publishing type：Research paper (other academic)  

DOI： 10.1109/icpr48806.2021.9412077

Language：Others   Publishing type：Research paper (scientific journal)  

Recent rapid progress in deep neural network techniques has allowed recognition and classification of various objects, often exceeding the performance of the human eye. In plant biology and crop sciences, some deep neural network frameworks have been applied mainly for effective and rapid phenotyping. In this study, beyond simple optimizations of phenotyping, we propose an application of deep neural networks to make an image-based internal disorder diagnosis that is hard even for experts, and to visualize the reasons behind each diagnosis to provide biological interpretations. Here, we exemplified classification of calyx-end cracking in persimmon fruit by using five convolutional neural network models with various layer structures and examined potential analytical options involved in the diagnostic qualities. With 3,173 visible RGB images from the fruit apex side, the neural networks successfully made the binary classification of each degree of disorder, with up to 90% accuracy. Furthermore, feature visualizations, such as Grad-CAM and LRP, visualize the regions of the image that contribute to the diagnosis. They suggest that specific patterns of color unevenness, such as in the fruit peripheral area, can be indexes of calyx-end cracking. These results not only provided novel insights into indexes of fruit internal disorders but also proposed the potential applicability of deep neural networks in plant biology.

DOI： 10.1093/pcp/pcaa111

Language：Others   Publishing type：Research paper (other academic)  

Temporal prediction is a still difficult task due to the chaotic behavior, non-Markovian characteristics, and nonstationary noise of temporal signals. Handwriting prediction is also challenging because of uncertainty arising from inter-class bifurcation structures, in addition to the above problems. For example, the classes '0' and '6' are very similar in terms of their beginning parts; therefore it is nearly impossible to predict their subsequent parts from the beginning part. In other words, '0' and '6' have a bifurcation structure due to ambiguity between classes, and we cannot make a long-term prediction in this context. In this paper, we propose a temporal prediction model that can deal with this bifurcation structure. Specifically, the proposed model learns the bifurcation structure explicitly as a Gaussian mixture model (GMM) for each class as well as the posterior probability of the classes. The final result of prediction is represented as the weighted sum of GMMs using the class probabilities as weights. When multiple classes have large weights, the model can handle a bifurcation and thus avoid an inaccurate prediction. The proposed model is formulated as a neural network including long short-term memories and is thus trained in an end-to-end manner. The proposed model was evaluated on the UNIPEN online handwritten character dataset, and the results show that the model can catch and deal with the bifurcation structures.

DOI： 10.1109/ICFHR2020.2020.00029

Language：Others   Publishing type：Research paper (other academic)  

In convolutional neural networks (CNNs), pooling operations play important roles such as dimensionality reduction and deformation compensation. In general, max pooling, which is the most widely used operation for local pooling, is performed independently for each kernel. However, the deformation may be spatially smooth over the neighboring kernels. This means that max pooling is too flexible to compensate for actual deformations. In other words, its excessive flexibility risks canceling the essential spatial differences between classes. In this paper, we propose regularized pooling, which enables the value selection direction in the pooling operation to be spatially smooth across adjacent kernels so as to compensate only for actual deformations. The results of experiments on handwritten character images and texture images showed that regularized pooling not only improves recognition accuracy but also accelerates the convergence of learning compared with conventional pooling operations.

DOI： 10.1007/978-3-030-61616-8_20

Language：Others   Publishing type：Research paper (other academic)  

Analyzing the handwriting generation process is an important issue and has been tackled by various generation models, such as kinematics based models and stochastic models. In this study, we use a reinforcement learning (RL) framework to realize handwriting generation with the careful future planning ability. In fact, the handwriting process of human beings is also supported by their future planning ability; for example, the ability is necessary to generate a closed trajectory like '0' because any shortsighted model, such as a Markovian model, cannot generate it. For the algorithm, we employ generative adversarial imitation learning (GAIL). Typical RL algorithms require the manual definition of the reward function, which is very crucial to control the generation process. In contrast, GAIL trains the reward function along with the other modules of the framework. In other words, through GAIL, we can understand the reward of the handwriting generation process from handwriting examples. Our experimental results qualitatively and quantitatively show that the learned reward catches the trends in handwriting generation and thus GAIL is well suited for the acquisition of handwriting behavior.

DOI： 10.1109/ICFHR2020.2020.00030

Language：Others   Publishing type：Research paper (other academic)  

In document analysis research, image-to-image conversion models such as a U-Net have been shown significant performance. Recently, cascaded U-Nets research is suggested for solving complex document analysis studies. However, improving performance by adding U-Net modules requires using too many parameters in cascaded U-Nets. Therefore, in this paper, we propose a method for enhancing the performance of cascaded U-Nets. We suggest a novel document image binarization method by utilizing Cascading Modular U-Nets (CMU-Nets) and Squeeze and Excitation blocks (SE-blocks). Through verification experiments, we point out the problems caused by the use of SE-blocks in existing CMU-Nets and suggest how to use SE-blocks in CMU-Nets. We use the Document Image Binarization (DIBCO) 2017 dataset to evaluate the proposed model.

DOI： 10.1007/978-3-030-57058-3_9

Language：Others   Publishing type：Research paper (other academic)  

We attempt to recognize and track lyric words in lyric videos. Lyric video is a music video showing the lyric words of a song. The main characteristic of lyric videos is that the lyric words are shown at frames synchronously with the music. The difficulty of recognizing and tracking the lyric words is that (1) the words are often decorated and geometrically distorted and (2) the words move arbitrarily and drastically in the video frame. The purpose of this paper is to analyze the motion of the lyric words in lyric videos, as the first step of automatic lyric video generation. In order to analyze the motion of lyric words, we first apply a state-of-the-art scene text detector and recognizer to each video frame. Then, lyric-frame matching is performed to establish the optimal correspondence between lyric words and the frames. After fixing the motion trajectories of individual lyric words from correspondence, we analyze the trajectories of the lyric words by k-medoids clustering and dynamic time warping (DTW).

DOI： 10.1007/978-3-030-57058-3_30

Language：Others   Publishing type：Research paper (other academic)  

In natural scenes and documents, we can find a correlation between text and its color. For instance, the word, “hot,” is often printed in red, while “cold” is often in blue. This correlation can be thought of as a feature that represents the semantic difference between the words. Based on this observation, we propose the idea of using text color for word embeddings. While text-only word embeddings (e.g. word2vec) have been extremely successful, they often represent antonyms as similar since they are often interchangeable in sentences. In this paper, we try two tasks to verify the usefulness of text color in understanding the meanings of words, especially in identifying synonyms and antonyms. First, we quantify the color distribution of words from the book cover images and analyze the correlation between the color and meaning of the word. Second, we try to retrain word embeddings with the color distribution of words as a constraint. By observing the changes in the word embeddings of synonyms and antonyms before and after re-training, we aim to understand the kind of words that have positive or negative effects in their word embeddings when incorporating text color information.

DOI： 10.1007/978-3-030-57058-3_24

Language：Others   Publishing type：Research paper (scientific journal)  

There are a countless number of fonts with various shapes and styles. In addition, there are many fonts that only have subtle differences in features. Due to this, font identification is a difficult task. In this paper, we propose a method of determining if any two characters are from the same font or not. This is difficult due to the difference between fonts typically being smaller than the difference between alphabet classes. Additionally, the proposed method can be used with fonts regardless of whether they exist in the training or not. In order to accomplish this, we use a Convolutional Neural Network (CNN) trained with various font image pairs. In the experiment, the network is trained on image pairs of various fonts. We then evaluate the model on a different set of fonts that are unseen by the network. The evaluation is performed with an accuracy of 92.27%. Moreover, we analyzed the relationship between character classes and font identification accuracy.

DOI： 10.1007/978-3-030-57058-3_35

Language：Others   Publishing type：Research paper (scientific journal)  

Designing logos, typefaces, and other decorated shapes can require professional skills. In this paper, we aim to produce new and unique decorated shapes by stylizing ordinary shapes with machine learning. Specifically, we combined parametric and non-parametric neural style transfer algorithms to transfer both local and global features. Furthermore, we introduced a distance-based guiding to the neural style transfer process, so that only the foreground shape will be decorated. Lastly, qualitative evaluation and ablation studies are provided to demonstrate the usefulness of the proposed method.

DOI： 10.1371/journal.pone.0233489

Language：Others   Publishing type：Research paper (scientific journal)  

Designing fonts requires a great deal of time and effort. It requires professional skills, such as sketching, vectorizing, and image editing. Additionally, each letter has to be designed individually. In this paper, we introduce a method to create fonts automatically. In our proposed method, the difference of font styles between two different fonts is transferred to another font using neural style transfer. Neural style transfer is a method of stylizing the contents of an image with the styles of another image. We proposed a novel neural style difference and content difference loss for the neural style transfer. With these losses, new fonts can be generated by adding or removing font styles from a font. We provided experimental results with various combinations of input fonts and discussed limitations and future development for the proposed method.

DOI： 10.1007/978-3-030-57058-3_38

Language：Others   Publishing type：Research paper (other academic)  

In this paper, we tackle a challenging domain conversion task between photo and icon images. Although icons often originate from real object images (i.e., photographs), severe abstractions and simplifications are applied to generate icon images by professional graphic designers. Moreover, there is no one-to-one correspondence between the two domains, for this reason we cannot use it as the ground-truth for learning a direct conversion function. Since generative adversarial networks (GAN) can undertake the problem of domain conversion without any correspondence, we test CycleGAN and UNIT to generate icons from objects segmented from photo images. Our experiments with several image datasets prove that CycleGAN learns sufficient abstraction and simplification ability to generate icon-like images.

DOI： 10.1145/3379173.3393708

Language：Others   Publishing type：Research paper (scientific journal)  

DOI： 10.1109/TIP.2020.2995062

Language：Others   Publishing type：Research paper (scientific journal)  

DOI： 10.1007/s42979-020-00132-z

Language：English   Publishing type：Research paper (international conference proceedings)  

We propose an online visual-object tracking method that is robust even in an adversarial environment, where various disturbances may occur on the target appearance, etc. The proposed method is based on a delayed-Hedge algorithm for aggregating multiple arbitrary online trackers with adaptive weights. The robustness in the tracking performance is guaranteed theoretically in term of "regret" by the property of the delayed-Hedge algorithm. Roughly speaking, the proposed method can achieve a similar tracking performance as the best one among all the trackers to be aggregated in an adversarial environment. The experimental study on various tracking tasks shows that the proposed method could achieve state-of-the-art performance by aggregating various online trackers.

DOI： 10.1109/WACV45572.2020.9093613

Language：English   Publishing type：Research paper (scientific journal)  

Apoptotic cells form membrane blebs, but little is known about how the formation and dynamics of membrane blebs are regulated. The size of blebs gradually increases during the progression of apoptosis, eventually forming large extracellular vesicles called apoptotic bodies that have immune-modulating activities. In this study, we investigated the molecular mechanism involved in the differentiation of blebs into apoptotic blebs by comparing the dynamics of the bleb formed during cell migration and the bleb formed during apoptosis. We revealed that the enhanced activity of ROCK1 is required for the formation of small blebs in the early phase of apoptosis, which leads to the physical disruption of nuclear membrane and the degradation of Lamin A. In the late phase of apoptosis, the loss of asymmetry in phospholipids distribution caused the enlargement of blebs, which enabled translocation of damage-associated molecular patterns to the bleb cytoplasm and maturation of functional apoptotic blebs. Thus, changes in cell membrane dynamics are closely linked to cytoplasmic changes during apoptotic bleb formation.

DOI： 10.1091/MBC.E19-12-0691

Language：Others   Publishing type：Research paper (scientific journal)  

This paper describes a novel model for time series recognition called a Dynamic Time Warping Neural Network (DTW-NN). DTW-NN is a feedforward neural network that exploits the elastic matching ability of DTW to dynamically align the inputs of a layer to the weights. This weight alignment replaces the standard dot product within a neuron with DTW. In this way, the DTW-NN is able to tackle difficulties with time series recognition such as temporal distortions and variable pattern length within a feedforward architecture. We demonstrate the effectiveness of DTW-NNs on four distinct datasets: online handwritten characters, accelerometer-based active daily life activities, spoken Arabic numeral Mel-Frequency Cepstrum Coefficients (MFCC), and one-dimensional centroid-radii sequences from leaf shapes. We show that the proposed method is an effective general approach to temporal pattern learning by achieving state-of-the-art results on these datasets.

DOI： 10.1016/j.knosys.2019.104971

Language：Others   Publishing type：Research paper (scientific journal)  

DOI： 10.1109/mcg.2019.2931431

Language：Others   Publishing type：Research paper (scientific journal)  

We propose the use of a novel feature, called local distance features, for time series classification. The local distance features are extracted using Dynamic Time Warping (DTW) and classified using Convolutional Neural Networks (CNN). DTW is classically as a robust distance measure for distance-based time series recognition methods. However, by using DTW strictly as a global distance measure, information about the matching is discarded. We show that this information can further be used as supplementary input information in temporal CNNs. This is done by using both the raw data and the features extracted from DTW in multi-modal fusion CNNs. Furthermore, we explore the effects of different prototype selection methods, prototype numbers, and data fusion schemes induce on the accuracy. We perform experiments on a wide range of time series datasets including three Unipen handwriting datasets, four UCI Machine Learning Repository datasets, and 85 UCR Time Series Classification Archive datasets.

DOI： 10.1016/j.patcog.2019.107024

Language：Others   Publishing type：Research paper (scientific journal)  

DOI： 10.1016/j.knosys.2019.104927

Language：English   Publishing type：Research paper (other academic)  

Optimal Rejection Function Meets Character Recognition Tasks

Language：English  

Convolutional Neural Networks (CNN) have become state-of-the-art in the field of image classification. However, not everything is understood about their inner representations. This paper tackles the interpretability and explainability of the predictions of CNNs for multi-class classification problems. Specifically, we propose a novel visualization method of pixel-wise input attribution called Softmax-Gradient Layer-wise Relevance Propagation (SGLRP). The proposed model is a class discriminate extension to Deep Taylor Decomposition (DTD) using the gradient of softmax to back propagate the relevance of the output probability to the input image. Through qualitative and quantitative analysis, we demonstrate that SGLRP can successfully localize and attribute the regions on input images which contribute to a target object's classification. We show that the proposed method excels at discriminating the target objects class from the other possible objects in the images. We confirm that SGLRP performs better than existing Layer-wise Relevance Propagation (LRP) based methods and can help in the understanding of the decision process of CNNs.

DOI： 10.1109/ICCVW.2019.00513

Language：Others   Publishing type：Research paper (scientific journal)  

DOI： 10.1109/ACCESS.2019.2945825

Language：English  

In this paper, we propose a novel Convolutional Neural Network (CNN) based method that extracts the location information (displacement features) of the maximums in the max-pooling operation and fuses it with the pooling features to capture the micro deformations between the genuine signatures and skilled forgeries as a feature extraction procedure. After the feature extraction procedure, we apply support vector machines (SVMs) as writer-dependent classifiers for each user to build the signature verification system. The extensive experimental results on GPDS-150, GPDS-300, GPDS-1000, GPDS-2000, and GPDS-5000 datasets demonstrate that the proposed method can discriminate the genuine signatures and their corresponding skilled forgeries well and achieve state-of-the-art results on these datasets.

DOI： 10.1109/ICDAR.2019.00180

Language：Others   Publishing type：Research paper (other academic)  

DOI： 10.1109/ICDAR.2019.00023

Language：Others   Publishing type：Research paper (other academic)  

DOI： 10.1109/ICDAR.2019.00071

Language：Others   Publishing type：Research paper (other academic)  

DOI： 10.1109/ICDAR.2019.00137

Language：Others   Publishing type：Research paper (other academic)  

DOI： 10.1109/ICDAR.2019.00153

Language：Others   Publishing type：Research paper (other academic)  

DOI： 10.1109/ICDAR.2019.00078

Language：Others   Publishing type：Research paper (other academic)  

DOI： 10.1109/ICDAR.2019.00072

Language：Others   Publishing type：Research paper (scientific journal)  

DOI： 10.1016/j.patcog.2019.05.014

Language：Others   Publishing type：Research paper (other academic)  

DOI： 10.1109/ICDAR.2019.00179

Language：Others   Publishing type：Research paper (other academic)  

DOI： 10.1109/ICDAR.2019.00113

Language：English  

In this paper, we conduct a large-scale study of font statistics in book covers and online advertisements. Through the statistical study, we try to understand how graphic designers relate fonts and content genres and identify the relationship between font styles, colors, and genres. We propose an automatic approach to extract font information from graphic designs by applying a sequence of character detection, style classification, and clustering techniques to the graphic designs. The extracted font information is accumulated together with genre information, such as romance or business, for further trend analysis. Through our unique empirical study, we show that the collected font statistics reveal interesting trends in terms of how typographic design represents the impression and the atmosphere of the content genres.

DOI： 10.1109/ICDAR.2019.00170

Language：English  

In document analysis, page segmentation is a fundamental task that divides a document image into semantic regions. In addition to local features, such as pixel-wise information, co-occurrence features are also useful for extracting texture-like periodic information for accurate segmentation. However, existing convolutional neural network (CNN)-based methods do not have any mechanisms that explicitly extract co-occurrence features. In this paper, we propose a method for page segmentation using a CNN with trainable multiplication layers (TMLs). The TML is specialized for extracting co-occurrences from feature maps, thereby supporting the detection of objects with similar textures and periodicities. This property is also considered to be effective for document image analysis because of regularity in text line structures, tables, etc. In the experiment, we achieved promising performance on a pixel-wise page segmentation task by combining TMLs with U-Net. The results demonstrate that TMLs can improve performance compared to the original U-Net. The results also demonstrate that TMLs are helpful for detecting regions with periodically repeating features, such as tables and main text.

DOI： 10.1109/ICDAR.2019.00167

Language：English  

In this paper, we analyze logo designs by using machine learning, as a promising trial of graphic design analysis. Specifically, we will focus on favicon images, which are tiny logos used as company icons on web browsers, and analyze them to understand their trends in individual industry classes. For example, if we can catch the subtle trends in favicons of financial companies, they will suggest to us how professional designers express the atmosphere of financial companies graphically. For the purpose, we will use top-rank learning, which is one of the recent machine learning methods for ranking and very suitable for revealing the subtle trends in graphic designs.

DOI： 10.1109/ICDAR.2019.00238

Language：Others   Publishing type：Research paper (scientific journal)  

DOI： 10.1109/ACCESS.2019.2934928

Language：Others   Publishing type：Research paper (other academic)  

DOI： 10.1109/EMBC.2019.8857011

Language：Others   Publishing type：Research paper (other academic)  

DOI： 10.1109/EMBC.2019.8856465

Language：Others  

Activation functions play an important role in the training of artificial
neural networks and the Rectified Linear Unit (ReLU) has been the mainstream in
recent years. Most of the activation functions currently used are deterministic
in nature, whose input-output relationship is fixed. In this work, we propose a
probabilistic activation function, called ProbAct. The output value of ProbAct
is sampled from a normal distribution, with the mean value same as the output
of ReLU and with a fixed or trainable variance for each element. In the
trainable ProbAct, the variance of the activation distribution is trained
through back-propagation. We also show that the stochastic perturbation through
ProbAct is a viable generalization technique that can prevent overfitting. In
our experiments, we demonstrate that when using ProbAct, it is possible to
boost the image classification performance on CIFAR-10, CIFAR-100, and STL-10
datasets.

Language：Others   Publishing type：Research paper (other academic)  

DOI： 10.1109/ICASSP.2019.8683036

Language：Others   Publishing type：Research paper (other academic)  

DOI： 10.1109/ICASSP.2019.8682908

Language：Others   Publishing type：Research paper (scientific journal)  

Scene word recognition from pieces to whole.

DOI： 10.1007/s11704-017-6420-2

Language：English  

We propose a soft-constrained clustering method for group-based labeling of medical images. Since the idea of group-based labeling is to attach the label to a group of samples at once, we need to have groups (i.e., clusters) with high purity. The proposed method is formulated to achieve high purity even for difficult clustering tasks such as medical image clustering, where image samples of the same class are often very distant in their feature space. In fact, those images degrade the performance of conventional constrained clustering methods. Experiments with an endoscopy image dataset demonstrated that our method outperformed various state-of-the-art methods.

DOI： 10.1007/978-3-030-32254-0_47

Language：Others  

Comic Text Detection Using Neural Network Approach.

DOI： 10.1007/978-3-030-05716-9_60

Language：English  

We propose a deep learning approach for identifying tropical cyclones (TCs) and their precursors. Twenty year simulated outgoing longwave radiation (OLR) calculated using a cloud-resolving global atmospheric simulation is used for training two-dimensional deep convolutional neural networks (CNNs). The CNNs are trained with 50,000 TCs and their precursors and 500,000 non-TC data for binary classification. Ensemble CNN classifiers are applied to 10 year independent global OLR data for detecting precursors and TCs. The performance of the CNNs is investigated for various basins, seasons, and lead times. The CNN model successfully detects TCs and their precursors in the western North Pacific in the period from July to November with a probability of detection (POD) of 79.9–89.1% and a false alarm ratio (FAR) of 32.8–53.4%. Detection results include 91.2%, 77.8%, and 74.8% of precursors 2, 5, and 7 days before their formation, respectively, in the western North Pacific. Furthermore, although the detection performance is correlated with the amount of training data and TC lifetimes, it is possible to achieve high detectability with a POD exceeding 70% and a FAR below 50% during TC season for several ocean basins, such as the North Atlantic, with a limited sample size and short lifetime. [Figure not available: see fulltext.].

DOI： 10.1186/s40645-018-0245-y

Language：Others   Publishing type：Research paper (scientific journal)  

DOI： 10.1186/s40645-018-0245-y

Language：English  

We propose a deep learning approach for identifying tropical cyclones (TCs) and their precursors. Twenty year simulated outgoing longwave radiation (OLR) calculated using a cloud-resolving global atmospheric simulation is used for training two-dimensional deep convolutional neural networks (CNNs). The CNNs are trained with 50,000 TCs and their precursors and 500,000 non-TC data for binary classification. Ensemble CNN classifiers are applied to 10 year independent global OLR data for detecting precursors and TCs. The performance of the CNNs is investigated for various basins, seasons, and lead times. The CNN model successfully detects TCs and their precursors in the western North Pacific in the period from July to November with a probability of detection (POD) of 79.9–89.1% and a false alarm ratio (FAR) of 32.8–53.4%. Detection results include 91.2%, 77.8%, and 74.8% of precursors 2, 5, and 7 days before their formation, respectively, in the western North Pacific. Furthermore, although the detection performance is correlated with the amount of training data and TC lifetimes, it is possible to achieve high detectability with a POD exceeding 70% and a FAR below 50% during TC season for several ocean basins, such as the North Atlantic, with a limited sample size and short lifetime. [Figure not available: see fulltext.].

DOI： 10.1186/s40645-018-0245-y

Language：English  

We propose a deep learning approach for identifying tropical cyclones (TCs) and their precursors. Twenty year simulated outgoing longwave radiation (OLR) calculated using a cloud-resolving global atmospheric simulation is used for training two-dimensional deep convolutional neural networks (CNNs). The CNNs are trained with 50,000 TCs and their precursors and 500,000 non-TC data for binary classification. Ensemble CNN classifiers are applied to 10 year independent global OLR data for detecting precursors and TCs. The performance of the CNNs is investigated for various basins, seasons, and lead times. The CNN model successfully detects TCs and their precursors in the western North Pacific in the period from July to November with a probability of detection (POD) of 79.9–89.1% and a false alarm ratio (FAR) of 32.8–53.4%. Detection results include 91.2%, 77.8%, and 74.8% of precursors 2, 5, and 7 days before their formation, respectively, in the western North Pacific. Furthermore, although the detection performance is correlated with the amount of training data and TC lifetimes, it is possible to achieve high detectability with a POD exceeding 70% and a FAR below 50% during TC season for several ocean basins, such as the North Atlantic, with a limited sample size and short lifetime. [Figure not available: see fulltext.].

DOI： 10.1186/s40645-018-0245-y

Language：Others  

A Trainable Multiplication Layer for Auto-correlation and Co-occurrence Extraction.

DOI： 10.1007/978-3-030-20890-5_27

Language：English  

We propose a deep learning approach for identifying tropical cyclones (TCs) and their precursors. Twenty year simulated outgoing longwave radiation (OLR) calculated using a cloud-resolving global atmospheric simulation is used for training two-dimensional deep convolutional neural networks (CNNs). The CNNs are trained with 50,000 TCs and their precursors and 500,000 non-TC data for binary classification. Ensemble CNN classifiers are applied to 10 year independent global OLR data for detecting precursors and TCs. The performance of the CNNs is investigated for various basins, seasons, and lead times. The CNN model successfully detects TCs and their precursors in the western North Pacific in the period from July to November with a probability of detection (POD) of 79.9–89.1% and a false alarm ratio (FAR) of 32.8–53.4%. Detection results include 91.2%, 77.8%, and 74.8% of precursors 2, 5, and 7 days before their formation, respectively, in the western North Pacific. Furthermore, although the detection performance is correlated with the amount of training data and TC lifetimes, it is possible to achieve high detectability with a POD exceeding 70% and a FAR below 50% during TC season for several ocean basins, such as the North Atlantic, with a limited sample size and short lifetime. [Figure not available: see fulltext.].

DOI： 10.1186/s40645-018-0245-y

Language：English  

We propose a deep learning approach for identifying tropical cyclones (TCs) and their precursors. Twenty year simulated outgoing longwave radiation (OLR) calculated using a cloud-resolving global atmospheric simulation is used for training two-dimensional deep convolutional neural networks (CNNs). The CNNs are trained with 50,000 TCs and their precursors and 500,000 non-TC data for binary classification. Ensemble CNN classifiers are applied to 10 year independent global OLR data for detecting precursors and TCs. The performance of the CNNs is investigated for various basins, seasons, and lead times. The CNN model successfully detects TCs and their precursors in the western North Pacific in the period from July to November with a probability of detection (POD) of 79.9–89.1% and a false alarm ratio (FAR) of 32.8–53.4%. Detection results include 91.2%, 77.8%, and 74.8% of precursors 2, 5, and 7 days before their formation, respectively, in the western North Pacific. Furthermore, although the detection performance is correlated with the amount of training data and TC lifetimes, it is possible to achieve high detectability with a POD exceeding 70% and a FAR below 50% during TC season for several ocean basins, such as the North Atlantic, with a limited sample size and short lifetime. [Figure not available: see fulltext.].

DOI： 10.1186/s40645-018-0245-y

Language：English  

We propose a deep learning approach for identifying tropical cyclones (TCs) and their precursors. Twenty year simulated outgoing longwave radiation (OLR) calculated using a cloud-resolving global atmospheric simulation is used for training two-dimensional deep convolutional neural networks (CNNs). The CNNs are trained with 50,000 TCs and their precursors and 500,000 non-TC data for binary classification. Ensemble CNN classifiers are applied to 10 year independent global OLR data for detecting precursors and TCs. The performance of the CNNs is investigated for various basins, seasons, and lead times. The CNN model successfully detects TCs and their precursors in the western North Pacific in the period from July to November with a probability of detection (POD) of 79.9–89.1% and a false alarm ratio (FAR) of 32.8–53.4%. Detection results include 91.2%, 77.8%, and 74.8% of precursors 2, 5, and 7 days before their formation, respectively, in the western North Pacific. Furthermore, although the detection performance is correlated with the amount of training data and TC lifetimes, it is possible to achieve high detectability with a POD exceeding 70% and a FAR below 50% during TC season for several ocean basins, such as the North Atlantic, with a limited sample size and short lifetime. [Figure not available: see fulltext.].

DOI： 10.1186/s40645-018-0245-y

Language：English  

Anatomical location classification of gastroscopic images using DenseNet trained from Cyclical Learning Rate

Language：Others  

On Fast Sample Preselection for Speeding up Convolutional Neural Network Training.

DOI： 10.1007/978-3-319-97785-0_7

Language：Others  

Introducing Local Distance-Based Features to Temporal Convolutional Neural Networks.

DOI： 10.1109/ICFHR-2018.2018.00025

Language：Others  

How do Convolutional Neural Networks Learn Design?

DOI： 10.1109/ICPR.2018.8545624

Language：Others  

Discovering Class-Wise Trends of Max-Pooling in Subspace.

DOI： 10.1109/ICFHR-2018.2018.00026

Language：Others  

An Image-Based Representation for Graph Classification.

DOI： 10.1007/978-3-319-97785-0_14

Language：English  

The cytoplasmic region of the amyloid β-protein precursor (APP) is necessary and sufficient for the enhanced fast velocity of APP transport by kinesin-1.
Amyloid β-protein precursor (APP) is transported mainly by kinesin-1 and at a higher velocity than other kinesin-1 cargos, such as Alcadein α (Alcα); this is denoted by the enhanced fast velocity (EFV). Interaction of the APP cytoplasmic region with kinesin-1, which is essential for EFV transport, is mediated by JNK-interacting protein 1 (JIP1). To determine the roles of interactions between the APP luminal region and cargo components, we monitored transport of chimeric cargo receptors, Alcα (luminal)-APP (cytoplasmic) and APP (luminal)-Alcα (cytoplasmic). Alcα-APP is transported at the EFV, whereas APP-Alcα is transported at the same velocity as wild-type Alcα. Thus, the cytoplasmic region of APP is necessary and sufficient for the EFV of APP transport by kinesin-1.

DOI： 10.1002/1873-3468.13204

Language：Others  

Biosignal Data Augmentation Based on Generative Adversarial Networks.

DOI： 10.1109/EMBC.2018.8512396

Language：Others  

CNN Training with Graph-Based Sample Preselection: Application to Handwritten Character Recognition.

DOI： 10.1109/DAS.2018.10

Language：Others  

Text Line Extraction Based on Integrated K-Shortest Paths Optimization.

DOI： 10.1109/DAS.2018.68

Language：Others  

Contained Neural Style Transfer for Decorated Logo Generation.

DOI： 10.1109/DAS.2018.78

Language：English   Publishing type：Research paper (scientific journal)  

Text in images contains exact semantic information and the text knowledge can be utilized in many image cognition and understanding applications. The human reading habits can provide the clues of text line structure for text line extraction. In this paper, we propose a novel human reading knowledge inspired text line extraction method based on k-shortest paths global optimization. Firstly, the candidate character extraction is reformulated as Maximal Stable Extremal Region (MSER) algorithm on gray, red, blue, and green channels of the target images, and the extracted MSERs are fed into Convolutional Neural Network (CNN) to remove the noise components. Then, the directed graph is built upon the character component nodes with edges inspired by human reading sense. The directed graph can automatically construct the relationship to eliminate the disorder of candidate text components. The text line paths optimization is inspired by the human reading ability in planning of a text line path sequentially. Therefore, the text line extraction problem can be solved using the k-shortest paths optimization algorithm by taking advantage of the human reading sense structure of the directed graph. It can extract the text lines iteratively to avoid the exhaustive searching and obtain global optimized text line number. The proposed method achieves the f-measure of 0.820 and 0.812 on public ICDAR2011 and ICDAR2013 dataset, respectively. The experimental results demonstrate the effectiveness of the proposed human reading knowledge inspired text line extraction method in comparison with state-of-the-art methods This paper presents one human reading knowledge inspired text line extraction method, which approves that the human reading knowledge can benefit the text line extraction and image text discovery.

DOI： 10.1007/s12559-017-9490-4

Language：English   Publishing type：Research paper (other academic)  

In this work, we investigate the ability of Convolutional Neural Networks (CNN) to infer the presence of components that comprise an image. In recent years, CNNs have achieved powerful results in classification, detection, and segmentation. However, these models learn from instance-level supervision of the detected object. In this paper, we determine if CNNs can detect objects using image-level weakly supervised labels without localization. To demonstrate that a CNN can infer awareness of objects, we evaluate a CNN's classification ability with a database constructed of Chinese characters with only character-level labeled components. We show that the CNN is able to achieve a high accuracy in identifying the presence of these components without specific knowledge of the component. Furthermore, we verify that the CNN is deducing the knowledge of the target component by comparing the results to an experiment with the component removed. This research is important for applications with large amounts of data without robust annotation such as Chinese character recognition.

DOI： 10.1109/ICDAR.2017.72

Language：English   Publishing type：Research paper (other academic)  

Applying object proposal technique for scene text detection becomes popular for its significant improvement in speed and accuracy for object detection. However, some of the text regions after the proposal classification are overlapped and hard to remove or merge. In this paper, we present a scene text relocation system that refines the detection from text proposals to text. An object proposal-based deep neural network is employed to get the text proposals. To tackle the detection overlapping problem, a refinement deep neural network relocates the overlapped regions by estimating the text probability inside, and locating the accurate text regions by thresholding. Since the spacebetweenwordsindifferenttextlinesarevarious, aguidance mechanism is proposed in text relocation to guide where to extract the text regions in word level. This refinement procedure helps boost the precision after removing multiple overlapped text regions or joint cracked text regions. The experimental results on standard benchmark ICDAR 2013 demonstrate the effectiveness of the proposed approach.

DOI： 10.1109/ICDAR.2017.212

Language：English   Publishing type：Research paper (other academic)  

The character information in natural scene images contains various personal information, such as telephone numbers, home addresses, etc. It is a high risk of leakage the information if they are published. In this paper, we proposed a scene text erasing method to properly hide the information via an inpainting convolutional neural network (CNN) model. The input is a scene text image, and the output is expected to be text erased image with all the character regions filled up the colors of the surrounding background pixels. This work is accomplished byaCNNmodelthroughconvolutiontodeconvolutionwithinterconnection process. The training samples and the corresponding inpainting images are considered as teaching signals for training. To evaluate the text erasing performance, the output images are detected by a novel scene text detection method. Subsequently, the same measurement on text detection is utilized for testing the images in benchmark dataset ICDAR2013. Compared with direct text detection way, the scene text erasing process demonstrates a drastically decrease on the precision, recall and f-score. That proves the effectiveness of proposed method for erasing the text in natural scene images.

DOI： 10.1109/ICDAR.2017.141

Language：English   Publishing type：Research paper (other academic)  

In this paper, we chose an approach to generate fonts by using neural style transfer. Neural style transfer uses Convolution Neural Networks(CNN) to transfer the style of one image to another. By modifying neural style transfer, we can achieve neural font style transfer. We also demonstrate the effects of using different weighted factors, character placements, and orientations. In addition, we show the results of using non-Latin alphabets, non-text patterns, and non-text images as style images. Finally, we provide insight into the characteristics of style transfer with fonts.

DOI： 10.1109/ICDAR.2017.328

Language：English   Publishing type：Research paper (other academic)  

In past OCR research, different OCR engines are used for different printing types, i.e., machine-printed characters, handwritten characters, and decorated fonts. A recent research, however, reveals that convolutional neural networks (CNN) can realize a universal OCR, which can deal with any printing types without pre-classification into individual types. In this paper, we analyze how CNN for universal OCR manage the different printing types. More specifically, we try to find where a handwritten character of a class and a machine-printed character of the same class are 'fused' in CNN. For analysis, we use two different approaches. The first approach is statistical analysis for detecting the CNN units which are sensitive (or insensitive) to type difference. The second approach is network-based visualization of pattern distribution in each layer. Both analyses suggest the same trend that types are not fully fused in convolutional layers but the distributions of the same class from different types become closer in upper layers.

DOI： 10.1109/ICDAR.2017.167

Language：Japanese  

Language：English   Publishing type：Research paper (other academic)  

Object tracking is one of the most important but still difficult tasks in computer vision and pattern recognition. The main difficulties in the tracking task are appearance variation of target objects and occlusion. To deal with those difficulties, we propose a object tracking method combining Single Shot Multibox Detector (SSD), Fully Convolutional Network (FCN) and Dynamic Programming (DP). SSD and FCN provide a probability value of the target object which allows for appearance variation within each category. DP provides a globally optimal tracking path even with severe occlusions. Through several experiments, we confirmed that their combination realized a robust object tracking method. Also, in contrast to traditional trackers, initial position and a template of the target do not need to be specified. We show that the proposed method has a higher performance than the traditional trackers in tracking various single objects through video frames.

DOI： 10.1007/978-3-319-75786-5_10

Language：Others  

Font Creation Using Class Discriminative Deep Convolutional Generative Adversarial Networks.

DOI： 10.1109/ACPR.2017.99

Language：English  

Language：English   Publishing type：Research paper (scientific journal)  

Dissimilarity space embedding (DSE) presents a method of representing data as vectors of dissimilarities. This representation is interesting for its ability to use a dissimilarity measure to embed various patterns (e.g. graph patterns with different topology and temporal patterns with different lengths) into a vector space. The method proposed in this paper uses a dynamic tithe warping (DTW) based DSE for the purpose of the classification of massive sets of temporal patterns. However, using large data sets introduces the problem of requiring a high computational cost. To address this, we consider a prototype selection approach. A vector space created by DSE offers us the ability to treat its independent dimensions as features allowing for the use of feature selection. The proposed method exploits this and reduces the number of prototypes required for accurate classification. To validate the proposed method we use two-class classification on a data set of handwritten on-line numerical digits. We show that by using DSE with ensemble classification, high accuracy classification is possible with very few prototypes.

DOI： 10.1016/j.patcog.2016.11.013

Language：English  

We studied social approach behaviour in medaka fish using three-dimensional computer graphic (3DCG) animations based on the morphological features and motion characteristics obtained from real fish. This is the first study which used 3DCG animations and examined the relative effects of morphological and motion cues on social approach behaviour in medaka. Various visual stimuli, e.g., lack of motion, lack of colour, alternation in shape, lack of locomotion, lack of body motion, and normal virtual fish in which all four features (colour, shape, locomotion, and body motion) were reconstructed, were created and presented to fish using a computer display. Medaka fish presented with normal virtual fish spent a long time in proximity to the display, whereas time spent near the display was decreased in other groups when compared with normal virtual medaka group. The results suggested that the naturalness of visual cues contributes to the induction of social approach behaviour. Differential effects between body motion and locomotion were also detected. 3DCG animations can be a useful tool to study the mechanisms of visual processing and social behaviour in medaka.

DOI： 10.1371/journal.pone.0175059

Language：English  

We studied social approach behaviour in medaka fish using three-dimensional computer graphic (3DCG) animations based on the morphological features and motion characteristics obtained from real fish. This is the first study which used 3DCG animations and examined the relative effects of morphological and motion cues on social approach behaviour in medaka. Various visual stimuli, e.g., lack of motion, lack of colour, alternation in shape, lack of locomotion, lack of body motion, and normal virtual fish in which all four features (colour, shape, locomotion, and body motion) were reconstructed, were created and presented to fish using a computer display. Medaka fish presented with normal virtual fish spent a long time in proximity to the display, whereas time spent near the display was decreased in other groups when compared with normal virtual medaka group. The results suggested that the naturalness of visual cues contributes to the induction of social approach behaviour. Differential effects between body motion and locomotion were also detected. 3DCG animations can be a useful tool to study the mechanisms of visual processing and social behaviour in medaka.

DOI： 10.1371/journal.pone.0175059

Language：English  

We studied social approach behaviour in medaka fish using three-dimensional computer graphic (3DCG) animations based on the morphological features and motion characteristics obtained from real fish. This is the first study which used 3DCG animations and examined the relative effects of morphological and motion cues on social approach behaviour in medaka. Various visual stimuli, e.g., lack of motion, lack of colour, alternation in shape, lack of locomotion, lack of body motion, and normal virtual fish in which all four features (colour, shape, locomotion, and body motion) were reconstructed, were created and presented to fish using a computer display. Medaka fish presented with normal virtual fish spent a long time in proximity to the display, whereas time spent near the display was decreased in other groups when compared with normal virtual medaka group. The results suggested that the naturalness of visual cues contributes to the induction of social approach behaviour. Differential effects between body motion and locomotion were also detected. 3DCG animations can be a useful tool to study the mechanisms of visual processing and social behaviour in medaka.

DOI： 10.1371/journal.pone.0175059

Language：English  

We studied social approach behaviour in medaka fish using three-dimensional computer graphic (3DCG) animations based on the morphological features and motion characteristics obtained from real fish. This is the first study which used 3DCG animations and examined the relative effects of morphological and motion cues on social approach behaviour in medaka. Various visual stimuli, e.g., lack of motion, lack of colour, alternation in shape, lack of locomotion, lack of body motion, and normal virtual fish in which all four features (colour, shape, locomotion, and body motion) were reconstructed, were created and presented to fish using a computer display. Medaka fish presented with normal virtual fish spent a long time in proximity to the display, whereas time spent near the display was decreased in other groups when compared with normal virtual medaka group. The results suggested that the naturalness of visual cues contributes to the induction of social approach behaviour. Differential effects between body motion and locomotion were also detected. 3DCG animations can be a useful tool to study the mechanisms of visual processing and social behaviour in medaka.

DOI： 10.1371/journal.pone.0175059

Language：English  

We studied social approach behaviour in medaka fish using three-dimensional computer graphic (3DCG) animations based on the morphological features and motion characteristics obtained from real fish. This is the first study which used 3DCG animations and examined the relative effects of morphological and motion cues on social approach behaviour in medaka. Various visual stimuli, e.g., lack of motion, lack of colour, alternation in shape, lack of locomotion, lack of body motion, and normal virtual fish in which all four features (colour, shape, locomotion, and body motion) were reconstructed, were created and presented to fish using a computer display. Medaka fish presented with normal virtual fish spent a long time in proximity to the display, whereas time spent near the display was decreased in other groups when compared with normal virtual medaka group. The results suggested that the naturalness of visual cues contributes to the induction of social approach behaviour. Differential effects between body motion and locomotion were also detected. 3DCG animations can be a useful tool to study the mechanisms of visual processing and social behaviour in medaka.

DOI： 10.1371/journal.pone.0175059

Language：English  

We studied social approach behaviour in medaka fish using three-dimensional computer graphic (3DCG) animations based on the morphological features and motion characteristics obtained from real fish. This is the first study which used 3DCG animations and examined the relative effects of morphological and motion cues on social approach behaviour in medaka. Various visual stimuli, e.g., lack of motion, lack of colour, alternation in shape, lack of locomotion, lack of body motion, and normal virtual fish in which all four features (colour, shape, locomotion, and body motion) were reconstructed, were created and presented to fish using a computer display. Medaka fish presented with normal virtual fish spent a long time in proximity to the display, whereas time spent near the display was decreased in other groups when compared with normal virtual medaka group. The results suggested that the naturalness of visual cues contributes to the induction of social approach behaviour. Differential effects between body motion and locomotion were also detected. 3DCG animations can be a useful tool to study the mechanisms of visual processing and social behaviour in medaka.

DOI： 10.1371/journal.pone.0175059

Language：English  

We studied social approach behaviour in medaka fish using three-dimensional computer graphic (3DCG) animations based on the morphological features and motion characteristics obtained from real fish. This is the first study which used 3DCG animations and examined the relative effects of morphological and motion cues on social approach behaviour in medaka. Various visual stimuli, e.g., lack of motion, lack of colour, alternation in shape, lack of locomotion, lack of body motion, and normal virtual fish in which all four features (colour, shape, locomotion, and body motion) were reconstructed, were created and presented to fish using a computer display. Medaka fish presented with normal virtual fish spent a long time in proximity to the display, whereas time spent near the display was decreased in other groups when compared with normal virtual medaka group. The results suggested that the naturalness of visual cues contributes to the induction of social approach behaviour. Differential effects between body motion and locomotion were also detected. 3DCG animations can be a useful tool to study the mechanisms of visual processing and social behaviour in medaka.

DOI： 10.1371/journal.pone.0175059

Language：English  

Language：English  

"You are what you read." As this sentence implies, reading is important for building our minds. We are investing a huge amount of time for reading to input information. However the activity of "reading" is done only by each individual in an analog way and nothing is digitally recorded and reused. In order to solve this problem, we record reading activities as digital data and analyze them for various goals. We call this research "reading-life log." In this chapter, we describe our achievements of the reading-life log. A target of the reading-life log is to analyze reading activities quantitatively and qualitatively: when, how much, what you read, and how you read in terms of your interests and understanding. Body-worn sensors including intelligent eyewear are employed for this purpose. Another target is to analyze the contents of documents based on the users' reading activities: for example, which are the parts most people feel difficult/interesting. Materials to be read are not limited to books and documents. Scene texts are also important materials which guide human activities.

DOI： 10.1007/978-4-431-56535-2_7

Language：English   Publishing type：Research paper (scientific journal)  

Cytoplasmic streaming refers to a collective movement of cytoplasm observed in many cell types(1-7). The mechanism of meiotic cytoplasmic streaming (MeiCS) in Caenorhabditis elegans zygotes is puzzling as the direction of the flow is not predefined by cell polarity and occasionally reverses(6). Here, we demonstrate that the endoplasmic reticulum (ER) network structure is required for the collective flow. Using a combination of RNAi, microscopy and image processing of C. elegans zygotes, we devise a theoretical model, which reproduces and predicts the emergence and reversal of the flow. We propose a positive-feedback mechanism, where a local flow generated along a microtubule is transmitted to neighbouring regions through the ER. This, in turn, aligns microtubules over a broader area to self-organize the collective flow. The proposed model could be applicable to various cytoplasmic streaming phenomena in the absence of predefined polarity. The increased mobility of cortical granules by MeiCS correlates with the efficient exocytosis of the granules to protect the zygotes from osmotic and mechanical stresses.

DOI： 10.1038/ncb3490

Language：English  

We propose a pre-selection method for training support vector machines (SVM) with a largescale dataset. Specifically, the proposed method selects patterns around the class boundary and the selected data is fed to train an SVM. For the selection, that is, searching for boundary patterns, we utilize a compressed representation of relative neighborhood graph (Clustered-RNG). A Clustered-RNG is a network of neighboring patterns which have a different class label and thus, we can find boundary patterns between different classes. Through large-scale handwritten digit pattern recognition experiments, we show that the proposed pre-selection method accelerates SVM training process 10 times faster without degrading recognition accuracy.

Language：English   Publishing type：Research paper (other academic)  

This paper proposes prediction methods for people flows and anomalies in people flows on a university campus. The proposed methods are based on deep learning frameworks. By predicting the statistics of people flow conditions on a university campus, it becomes possible to create applications that predict future crowded places and the time when congestion will disappear. Our prediction methods will be useful for developing applications for solving problems in cities.

DOI： 10.5220/0006248806760683

Language：English   Publishing type：Research paper (scientific journal)  

Cells may exchange information with other cells and tissues by exerting forces on the extracellular matrix (ECM). Fibronectin (FN) is an important ECM component that forms fibrils through cell contacts and creates directionally biased geometry. Here, we demonstrate that FN is deposited as pillars between widely separated germ layers, namely the somitic mesoderm and the endoderm, in quail embryos. Alongside the FN pillars, long filopodia protrude from the basal surfaces of somite epithelial cells. Loss-of-function of Ena/VASP, α5β1-integrins or talin in the somitic cells abolished the FN pillars, indicating that FN pillar formation is dependent on the basal filopodia through these molecules. The basal filopodia and FN pillars are also necessary for proper somite morphogenesis. We identified a new mechanism contributing to FN pillar formation by focusing on cyclic expansion of adjacent dorsal aorta. Maintenance of the directional alignment of the FN pillars depends on pulsatile blood flow through the dorsal aortae. These results suggest that the FN pillars are specifically established through filopodia-mediated and pulsating force-related mechanisms.

DOI： 10.1242/dev.141259

Language：English   Publishing type：Research paper (other academic)  

Scene text is one of the most important information sources for our daily life because it has particular functions such as disambiguation and navigation. In contrast, ordinary document text has no such function. Consequently, it is natural to have a hypothesis that scene text and document text have different characteristics. This paper tries to prove this hypothesis by semantic analysis of texts by word2vec, which is a neural network model to give a vector representation of each word. By the vector representation, we can have the semantic distributions of scene text and document text in Euclidean space and then determine their semantic categories by simple clustering. Experimental study reveals several differences between scene text and document text. For example, it is found that scene text is a semantic subset of document text and several semantic categories are very specific to scene text.

DOI： 10.1109/ICPR.2016.7900267

Language：English   Publishing type：Research paper (scientific journal)  

Scene text detection and scene segmentation are meaningful tasks in the computer vision field. Could the semantic scene segmentation assist scene text detection in any degree? For example, can we expect the probability of a region being text is low if its surrounding segment, i.e., its context, is labeled as sky? In this paper, we have a positive answer by constructing a scene context-based text detection model. In this model, we use texton features and a fully-connected conditional random field (CRF) to estimate pixel-level scene class's probability to be considered as image's context feature. Meanwhile, maximally stable extremal regions (MSERs) are extracted, integrated and extended as image patches of character candidates. Then, each image patch is fed to a simple two-layer convolutional neural network (CNN) to automatically extract its character feature. The averaged context feature of the corresponding patch is considered as the patch's context feature. The character feature and context feature are fused as the input into a support vector machine for text/non-text determination. Finally, as a post-processing, neighboring text regions are grouped hierarchically. The performance evaluation on ICDAR2013 and SVT databases, as well as a preliminary evaluation on a patch-level database, proves that the scene context can improve the performance of scene text detection. Moreover, the comparative study with state-of-the-art methods shows the top-level performance of our method. (C) 2016 Elsevier Ltd. All rights reserved.

DOI： 10.1016/j.patcog.2016.04.011

Language：English   Publishing type：Research paper (other academic)  

Temporal pattern recognition is challenging because temporal patterns require extra considerations over other data types, such as order, structure, and temporal distortions. Recently, there has been a trend in using large data and deep learning, however, many of the tools cannot be directly used with temporal patterns. Convolutional Neural Networks (CNN) for instance are traditionally used for visual and image pattern recognition. This paper proposes a method using a neural network to classify isolated temporal patterns directly. The proposed method uses dynamic time warping (DTW) as a kernel-like function to learn dissimilarity-based feature maps as the basis of the network. We show that using the proposed DTW-NN, efficient classification of on-line handwritten digits is possible with accuracies comparable to state-of-the-art methods.

DOI： 10.1109/ICFHR.2016.53

Language：English   Publishing type：Research paper (other academic)  

Convolutional Neural Networks (CNN) are on the forefront of accurate character recognition. This paper explores CNNs at their maximum capacity by implementing the use of large datasets. We show a near-perfect performance by using a dataset of about 820,000 real samples of isolated handwritten digits, much larger than the conventional MNIST database. In addition, we report a near-perfect performance on the recognition of machine-printed digits and multi-font digital born digits. Also, in order to progress toward a universal OCR, we propose methods of combining the datasets into one classifier. This paper reveals the effects of combining the datasets prior to training and the effects of transfer learning during training. The results of the proposed methods also show an almost perfect accuracy suggesting the ability of the network to generalize all forms of text.

DOI： 10.1109/ICFHR.2016.77

Language：English   Publishing type：Research paper (other academic)  

One of the important problems of cytological image analysis is the cell segmentation. Today the most perspective direction of cytological image analysis is living cells investigation. Such images lead to many troubles for cell analysis. In this paper, we propose a solution of one such problems: pattern extraction of living cells from its aggregation and measurement of their 3D characteristics.

DOI： 10.1109/DT.2016.7557173

Language：English   Publishing type：Research paper (scientific journal)  

Despite theoretical and physical studies implying that cell-extracellular matrix adhesion geometry governs the orientation of the cell division axis, the molecular mechanisms that translate interphase adhesion geometry to the mitotic spindle orientation remain elusive. Here, we show that the cellular edge retraction during mitotic cell rounding correlates with the spindle axis. At the onset of mitotic cell rounding, caveolin-1 is targeted to the retracting cortical region at the proximal end of retraction fibres, where ganglioside GM1-enriched membrane domains with clusters of caveola-like structures are formed in an integrin and RhoA-dependent manner. Furthermore, G alpha i1-LGN-NuMA, a well-known regulatory complex of spindle orientation, is targeted to the caveolin-1-enriched cortical region to guide the spindle axis towards the cellular edge retraction. We propose that retraction-induced cortical heterogeneity of caveolin-1 during mitotic cell rounding sets the spindle orientation in the context of adhesion geometry.

DOI： 10.1038/ncomms11858

Language：English   Publishing type：Research paper (scientific journal)  

Anomaly detection is the process of identifying unexpected items or events in datasets, which differ from the norm. In contrast to standard classification tasks, anomaly detection is often applied on unlabeled data, taking only the internal structure of the dataset into account. This challenge is known as unsupervised anomaly detection and is addressed in many practical applications, for example in network intrusion detection, fraud detection as well as in the life science and medical domain. Dozens of algorithms have been proposed in this area, but unfortunately the research community still lacks a comparative universal evaluation as well as common publicly available datasets. These shortcomings are addressed in this study, where 19 different unsupervised anomaly detection algorithms are evaluated on 10 different datasets from multiple application domains. By publishing the source code and the datasets, this paper aims to be a new well-funded basis for unsupervised anomaly detection research. Additionally, this evaluation reveals the strengths and weaknesses of the different approaches for the first time. Besides the anomaly detection performance, computational effort, the impact of parameter settings as well as the global/local anomaly detection behavior is outlined. As a conclusion, we give an advise on algorithm selection for typical real-world tasks.

DOI： 10.1371/journal.pone.0152173

Language：English   Publishing type：Research paper (other academic)  

The task of text line extraction in images is a crucial prerequisite for content-based image understanding applications. In this paper, we propose a novel text line extraction method based on k-shortest paths global optimization in images. Firstly, the candidate connected components are extracted by reformulating it as Maximal Stable Extremal Region (MSER) results in images. Then, the directed graph is built upon the connected component nodes with edges comprising of unary and pairwise cost function. Finally, the text line extraction problem is solved using the k-shortest paths optimization algorithm by taking advantage of the particular structure of the directed graph. Experimental results on public dataset demonstrate the effectiveness of proposed method in comparison with state-of-the-art methods.

DOI： 10.1109/DAS.2016.12

Language：English   Publishing type：Research paper (scientific journal)  

The actin cytoskeleton usually lies beneath the plasma membrane. When the membrane-associated actin cytoskeleton is transiently disrupted or the intracellular pressure is increased, the plasma membrane detaches from the cortex and protrudes. Such protruded membrane regions are called blebs. However, the molecular mechanisms underlying membrane blebbing are poorly understood. This study revealed that epidermal growth factor receptor kinase substrate 8 (Eps8) and ezrin are important regulators of rapid actin reassembly for the initiation and retraction of protruded blebs. Live-cell imaging of membrane blebbing revealed that local reassembly of actin filaments occurred at Eps8- and activated ezrin-positive foci of membrane blebs. Furthermore, we found that a RhoA-ROCK-Rnd3 feedback loop determined the local reassembly sites of the actin cortex during membrane blebbing.

DOI： 10.1073/pnas.1600968113

Language：English  

Outlier removal from training data is a classical problem in pattern recognition. Nowadays, this problem becomes more important for large-scale datasets by the following two reasons: First, we will have a higher risk of "unexpected" outliers, such as mislabeled training data. Second, a large-scale dataset makes it more difficult to grasp the distribution of outliers. On the other hand, many unsupervised anomaly detection methods have been proposed, which can be also used for outlier removal. In this paper, we present a comparative study of nine different anomaly detection methods in the scenario of outlier removal from a large-scale dataset. For accurate performance observation, we need to use a simple and describable recognition procedure and thus utilize a nearest neighbor-based classifier. As an adequate large-scale dataset, we prepared a handwritten digit dataset comprising of more than 800,000 manually labeled samples. With a data dimensionality of 16×16=256, it is ensured that each digit class has at least 100 times more instances than data dimensionality. The experimental results show that the common understanding that outlier removal improves classification performance on small datasets is not true for high-dimensional large-scale datasets. Additionally, it was found that local anomaly detection algorithms perform better on this data than their global equivalents.

Language：English  

Outlier removal from training data is a classical problem in pattern recognition. Nowadays, this problem becomes more important for large-scale datasets by the following two reasons: First, we will have a higher risk of "unexpected" outliers, such as mislabeled training data. Second, a large-scale dataset makes it more difficult to grasp the distribution of outliers. On the other hand, many unsupervised anomaly detection methods have been proposed, which can be also used for outlier removal. In this paper, we present a comparative study of nine different anomaly detection methods in the scenario of outlier removal from a large-scale dataset. For accurate performance observation, we need to use a simple and describable recognition procedure and thus utilize a nearest neighbor-based classifier. As an adequate large-scale dataset, we prepared a handwritten digit dataset comprising of more than 800,000 manually labeled samples. With a data dimensionality of 16×16=256, it is ensured that each digit class has at least 100 times more instances than data dimensionality. The experimental results show that the common understanding that outlier removal improves classification performance on small datasets is not true for high-dimensional large-scale datasets. Additionally, it was found that local anomaly detection algorithms perform better on this data than their global equivalents.

Language：English  

Outlier removal from training data is a classical problem in pattern recognition. Nowadays, this problem becomes more important for large-scale datasets by the following two reasons: First, we will have a higher risk of "unexpected" outliers, such as mislabeled training data. Second, a large-scale dataset makes it more difficult to grasp the distribution of outliers. On the other hand, many unsupervised anomaly detection methods have been proposed, which can be also used for outlier removal. In this paper, we present a comparative study of nine different anomaly detection methods in the scenario of outlier removal from a large-scale dataset. For accurate performance observation, we need to use a simple and describable recognition procedure and thus utilize a nearest neighbor-based classifier. As an adequate large-scale dataset, we prepared a handwritten digit dataset comprising of more than 800,000 manually labeled samples. With a data dimensionality of 16×16=256, it is ensured that each digit class has at least 100 times more instances than data dimensionality. The experimental results show that the common understanding that outlier removal improves classification performance on small datasets is not true for high-dimensional large-scale datasets. Additionally, it was found that local anomaly detection algorithms perform better on this data than their global equivalents.

Language：English  

Outlier removal from training data is a classical problem in pattern recognition. Nowadays, this problem becomes more important for large-scale datasets by the following two reasons: First, we will have a higher risk of "unexpected" outliers, such as mislabeled training data. Second, a large-scale dataset makes it more difficult to grasp the distribution of outliers. On the other hand, many unsupervised anomaly detection methods have been proposed, which can be also used for outlier removal. In this paper, we present a comparative study of nine different anomaly detection methods in the scenario of outlier removal from a large-scale dataset. For accurate performance observation, we need to use a simple and describable recognition procedure and thus utilize a nearest neighbor-based classifier. As an adequate large-scale dataset, we prepared a handwritten digit dataset comprising of more than 800,000 manually labeled samples. With a data dimensionality of 16×16=256, it is ensured that each digit class has at least 100 times more instances than data dimensionality. The experimental results show that the common understanding that outlier removal improves classification performance on small datasets is not true for high-dimensional large-scale datasets. Additionally, it was found that local anomaly detection algorithms perform better on this data than their global equivalents.

Language：English  

Outlier removal from training data is a classical problem in pattern recognition. Nowadays, this problem becomes more important for large-scale datasets by the following two reasons: First, we will have a higher risk of "unexpected" outliers, such as mislabeled training data. Second, a large-scale dataset makes it more difficult to grasp the distribution of outliers. On the other hand, many unsupervised anomaly detection methods have been proposed, which can be also used for outlier removal. In this paper, we present a comparative study of nine different anomaly detection methods in the scenario of outlier removal from a large-scale dataset. For accurate performance observation, we need to use a simple and describable recognition procedure and thus utilize a nearest neighbor-based classifier. As an adequate large-scale dataset, we prepared a handwritten digit dataset comprising of more than 800,000 manually labeled samples. With a data dimensionality of 16×16=256, it is ensured that each digit class has at least 100 times more instances than data dimensionality. The experimental results show that the common understanding that outlier removal improves classification performance on small datasets is not true for high-dimensional large-scale datasets. Additionally, it was found that local anomaly detection algorithms perform better on this data than their global equivalents.

Language：English  

Outlier removal from training data is a classical problem in pattern recognition. Nowadays, this problem becomes more important for large-scale datasets by the following two reasons: First, we will have a higher risk of "unexpected" outliers, such as mislabeled training data. Second, a large-scale dataset makes it more difficult to grasp the distribution of outliers. On the other hand, many unsupervised anomaly detection methods have been proposed, which can be also used for outlier removal. In this paper, we present a comparative study of nine different anomaly detection methods in the scenario of outlier removal from a large-scale dataset. For accurate performance observation, we need to use a simple and describable recognition procedure and thus utilize a nearest neighbor-based classifier. As an adequate large-scale dataset, we prepared a handwritten digit dataset comprising of more than 800,000 manually labeled samples. With a data dimensionality of 16×16=256, it is ensured that each digit class has at least 100 times more instances than data dimensionality. The experimental results show that the common understanding that outlier removal improves classification performance on small datasets is not true for high-dimensional large-scale datasets. Additionally, it was found that local anomaly detection algorithms perform better on this data than their global equivalents.

Language：English   Publishing type：Research paper (scientific journal)  

Text detection and recognition in video is challenging due to the presence of different types of texts, namely, graphics (video caption), scene (natural text), 2D, 3D, static and dynamic texts. Developing a universal method that works well for all the types is hard. In this paper, we propose a novel method for classifying graphics-scene and 2D-3D texts in video to enhance text detection and recognition accuracies. We first propose an iterative method to classify static and dynamic clusters based on the fact that static texts have zero velocity while dynamic texts have non-zero velocity. This results in text candidates for both static and dynamic texts regardless of 2D and 3D types. We then propose symmetry for text candidates using stroke width distances and medial axis values, which results in potential text candidates. We group potential text candidates using their geometrical properties to form text regions. Next, for each text region, we study the distribution of the dominant medial axis values given by ring radius transform in a new way to classify graphics and scene texts. Similarly, we study the proximity among the pixels that satisfy the gradient directions symmetry to classify 2D and 3D texts. We evaluate each step of the proposed method in terms of classification and recognition rates through classification with the existing methods to show that video text classification is effective and necessary for enhancing the capability of current text detection and recognition systems. (C) 2015 Elsevier Ltd. All rights reserved.

DOI： 10.1016/j.patcog.2015.07.002

Language：English   Publishing type：Research paper (scientific journal)  

Anchor graph hashing (AGH) is a promising hashing method for nearest neighbor (NN) search. AGH realizes efficient search by generating and utilizing a small number of points that are called anchors. In this paper, we propose a method for improving AGH, which considers data distribution in a similarity space and selects suitable anchors by performing principal component analysis (PCA) in the similarity space.

DOI： 10.1587/transinf.2015EDL8060

Language：English   Publishing type：Research paper (other academic)  

Recently, two different trends in neural network-based machine learning could be observed. The first one are the introduction of Bidirectional Long Short-Term Memory (BLSTM) neural networks (NN) which made sequences with long-distant dependencies amenable for neural network-based processing. The second one are deep learning techniques, which greatly increased the performance of neural networks, by making use of many hidden layers. In this paper, we propose to combine these two ideas for the task of unconstrained handwriting recognition. Extensive experimental evaluation on the IAM database demonstrate an increase of the recognition performance when using deep learning approaches over commonly used BLSTM neural networks, as well as insight into how different types of hidden layers affect the recognition accuracy.

DOI： 10.1109/ICDAR.2015.7333894

Language：English   Publishing type：Research paper (other academic)  

Color feature, as one of the low level features, plays important role in image processing, object recognition and other fields. For example, in the task of scene text detection and recognition, lots of methodologies employ features that utilize color contrast of text and the corresponding background for connected component extraction. However, the true distributions of text and its background, in terms of color, is still not examined because it requires an enough number of scene text database with pixel-level labelled text/non-text ground truth. To clarify the relationship between text and its background, in this paper, we aim at investigating the color non-parametric distribution of text and its background using a large database that contains 3018 scene images and 98,600 characters. The results of our experiments show that text and its background can be discriminated by means of color, therefore color feature can be used for scene text detection.

DOI： 10.1109/ICDAR.2015.7333813

Language：English   Publishing type：Research paper (other academic)  

This paper introduces a novel method of reducing the number of prototype patterns necessary for accurate recognition of temporal patterns. The nearest neighbor (NN) method is an effective tool in pattern recognition, but the downside is it can be computationally costly when using large quantities of data. To solve this problem, we propose a method of representing the temporal patterns by embedding dynamic time warping (DTW) distance based dissimilarities in vector space. Adaptive boosting (AdaBoost) is then applied for classifier training and feature selection to reduce the number of prototype patterns required for accurate recognition. With a data set of handwritten digits provided by the International Unipen Foundation (iUF), we successfully show that a large quantity of temporal data can be efficiently classified produce similar results to the established NN method while performing at a much smaller cost.

DOI： 10.1109/ICDAR.2015.7333875

Language：English   Publishing type：Research paper (other academic)  

This paper presents an experimental analysis on the use of semi-supervised learning in the handwritten digit recognition field. More specifically, two new feedback-based techniques for retraining individual classifiers in a multi-expert scenario are discussed. These new methods analyze the final decision provided by the multi-expert system so that sample classified with a confidence greater than a specific threshold is used to update the system itself. Experimental results carried out on the CEDAR (handwritten digits) database are presented. In particular, error rate, similarity index and a new correlation score among them are considered in order to evaluate the best retraining rule. For the experimental evaluation, an SVM classifier and five different combination techniques at abstract and measurement level have been used. Finally, the results show that iterating the feedback process, on different multi-expert systems built with the five combination techniques, one retraining rule is winning over the other respect to the best correlation score.

DOI： 10.1109/ICDAR.2015.7333734

Language：English   Publishing type：Research paper (other academic)  

We propose a pre-selection method for training support vector machines (SVM) with a large-scale dataset. Specifically, the proposed method selects patterns around the class boundary and the selected data is fed to train an SVM. For the selection, that is, searching for boundary patterns, we utilize a relative neighborhood graph (RNG). An RNG has an edge for each pair of neighboring patterns and thus, we can find boundary patterns by looking for edges connecting patterns from different classes. Through large-scale handwritten digit pattern recognition experiments, we show that the proposed pre-selection method accelerates SVM training process 5-15 times faster without degrading recognition accuracy.

DOI： 10.1109/ICDAR.2015.7333773

Language：English   Publishing type：Research paper (other academic)  

Recently, the horizon of dynamic time warping (DTW) for matching two sequential patterns has been extended to deal with non-Markovian constraints. The non-Markovian constraints regulate the matching in a wider scale, whereas Markovian constraints regulate the matching only locally. The global optimization of the non-Markovian DTW is proved to be solvable in polynomial time by a graph cut algorithm. The main contribution of this paper is to reveal what is the best constraint for handwriting recognition by using the non-Markovian DTW. The result showed that the best constraint is not a Markovian but a totally non-Markovian constraint that regulates the matching between very distant points; that is, it was proved that the conventional Markovian DTW has a clear limitation and the non-Markovian DTW should be more focused in future research.

DOI： 10.1109/ICDAR.2015.7333801

Language：English   Publishing type：Research paper (other academic)  

Results of the ICDAR 2015 Robust Reading Competition are presented. A new Challenge 4 on Incidental Scene Text has been added to the Challenges on Born-Digital Images, Focused Scene Images and Video Text. Challenge 4 is run on a newly acquired dataset of 1,670 images evaluating Text Localisation, Word Recognition and End-to-End pipelines. In addition, the dataset for Challenge 3 on Video Text has been substantially updated with more video sequences and more accurate ground truth data. Finally, tasks assessing End-to-End system performance have been introduced to all Challenges. The competition took place in the first quarter of 2015, and received a total of 44 submissions. Only the tasks newly introduced in 2015 are reported on. The datasets, the ground truth specification and the evaluation protocols are presented together with the results and a brief summary of the participating methods.

DOI： 10.1109/ICDAR.2015.7333942

Language：English   Publishing type：Research paper (other academic)  

This paper has two contributions toward understanding the principles in font design. The first contribution of this paper is to discover the most standard font shape of each letter class by analyzing thousands of different fonts. For this analysis, two different methods are used. The first method is congealing for aligning multiple images based on a nonlinear geometric transformation model. The average of the aligned image is considered as a standard font shape. The second method is network analysis for representing font variations as a large-scale relative neighborhood graph (RNG) and then finding its center. The font corresponding to the center is considered as the standard font shape. Both of the standard font shapes given by the two methods are plain without decoration, serif, or slant, and thus give an objective reason why we consider the plain font as the typical font shape. The second contribution is to utilize the RNG and the pairwise congealing technique for discovering unexplored font designs and then generating totally new fonts automatically.

DOI： 10.1109/ICDAR.2015.7333800

Language：Others  

付加情報の一般的な割り当て
特徴量のみでは本質的に避けることができない誤認識を回避するために， 付加情報を用いるパターン認識という枠組みが提案されている． この方式では，パターン認識を行う際に， 付加情報と呼ばれるクラスの決定を補助する少量の情報を特徴量と 同時に用いて認識性能の改善を目指す． 付加情報は自由に設定でき，通常は誤認識率が最小になるように設定する． ここで問題となるのは，誤認識率が最小になる付加情報の設定方法である． 常に正しい付加情報が得られるいう理想的な条件においては 既に問題が定式化され，付加情報の割り当て方法が導かれている． しかし，実環境での使用を考えると， 付加情報に生じる観測誤差を考慮した割り当て方法が求められる． そこで本論文では 付加情報の観測誤差を考慮に入れて，問題を新たに定式化する． これは付加情報が誤らない場合にも有効な一般的なものである． 本論文で導いた割り当て方法が有効に機能することを マハラノビス距離を用いた実験で例示する．

Language：English   Publishing type：Research paper (other academic)  

In order to cope with the exponential time complexity of graph edit distance, several polynomial-time approximation algorithms have been proposed in recent years. The Hausdorff edit distance is a quadratic-time matching procedure for labeled graphs which reduces the edit distance to a correspondence problem between local substructures. In its original formulation, nodes and their adjacent edges have been considered as local substructures. In this paper, we integrate a more general structural node context into the matching procedure based on hierarchical subgraphs. In an experimental evaluation on diverse graph data sets, we demonstrate that the proposed generalization of Hausdorff edit distance can significantly improve the accuracy of graph classification while maintaining low computational complexity.

DOI： 10.1007/978-3-319-18224-7_15

Language：English   Publishing type：Research paper (scientific journal)  

This paper reviews several trials of re-designing conventional communication medium, i.e., characters, for enriching their functions by using data-embedding techniques. For example, characters are redesigned to have better machine-readability even under various geometric distortions by embedding a geometric invariant into each character image to represent class label of the character. Another example is to embed various information into handwriting trajectory by using a new pen device, called a data-embedding pen. An experimental result showed that we can embed 32-bit information into a handwritten line of 5 cm length by using the pen device. In addition to those applications, we also discuss the relationship between data-embedding and pattern recognition in a theoretical point of view. Several theories tell that if we have appropriate supplementary information by data-embedding, we can enhance pattern recognition performance up to 100%.

DOI： 10.1587/transinf.2014MUI0002

Language：English   Publishing type：Research paper (scientific journal)  

This paper evaluates the degree of saliency of texts in natural scenes using visual saliency models. A large scale scene image database with pixel level ground truth is created for this purpose. Using this scene image database and five state-of-the-art models, visual saliency maps that represent the degree of saliency of the objects are calculated. The receiver operating characteristic curve is employed in order to evaluate the saliency of scene texts, which is calculated by visual saliency models. A visualization of the distribution of scene texts and non-texts in the space constructed by three kinds of saliency maps, which are calculated using Itti's visual saliency model with intensity, color and orientation features, is given. This visualization of distribution indicates that text characters are more salient than their non-text neighbors, and can be captured from the background. Therefore, scene texts can be extracted from the scene images. With this in mind, a new visual saliency architecture, named hierarchical visual saliency model, is proposed. Hierarchical visual saliency model is based on Itti's model and consists of two stages. In the first stage, Itti's model is used to calculate the saliency map, and Otsu's global thresholding algorithm is applied to extract the salient region that we are interested in. In the second stage, Itti's model is applied to the salient region to calculate the final saliency map. An experimental evaluation demonstrates that the proposed model outperforms Itti's model in terms of captured scene texts.

DOI： 10.1371/journal.pone.0114539

Language：English   Publishing type：Research paper (scientific journal)  

Alzheimer's β-amyloid precursor protein (APP) associates with kinesin-1 via JNK-interacting protein 1 (JIP1); however, the role of JIP1 in APP transport by kinesin-1 in neurons remains unclear. We performed a quantitative analysis to understand the role of JIP1 in APP axonal transport. In JIP1-deficient neurons, we find that both the fast velocity (∼2.7 μm/s) and high frequency (66%) of anterograde transport of APP cargo are impaired to a reduced velocity (∼1.83 μm/s) and a lower frequency (45%). We identified two novel elements linked to JIP1 function, located in the central region of JIP1b, that interact with the coiled-coil domain of kinesin light chain 1 (KLC1), in addition to the conventional interaction of the JIP1b 11-amino acid C-terminal (C11) region with the tetratricopeptide repeat of KLC1. High frequency of APP anterograde transport is dependent on one of the novel elements in JIP1b. Fast velocity of APP cargo transport requires the C11 domain, which is regulated by the second novel region of JIP1b. Furthermore, efficient APP axonal transport is not influenced by phosphorylation of APP at Thr-668, a site known to be phosphorylated by JNK. Our quantitative analysis indicates that enhanced fast-velocity and efficient high-frequency APP anterograde transport observed in neurons are mediated by novel roles of JIP1b.

DOI： 10.1091/mbc.E14-06-1111

Language：English   Publishing type：Research paper (scientific journal)  

For stroke-order free online multi-stroke character recognition, stroke-to-stroke correspondence search between an input pattern and a reference pattern plays an important role to deal with the stroke-order variation. Although various methods of stroke correspondence have been proposed, no comparative study for clarifying the relative superiority of those methods has been done before. In this paper, we firstly review the approaches for solving the stroke-order variation problem. Then, five representative methods of stroke correspondence proposed by different groups, including cube search (CS), bipartite weighted matching (BWM), individual correspondence decision (ICD), stable marriage (SM), and deviation-expansion model (DE), are experimentally compared, mainly in regard of recognition accuracy and efficiency. The experimental results on an online Kanji character dataset, showed that 99.17%, 99.17%, 96.37%, 98.54%, and 96.59% were attained by CS, BWM, ICD, SM, and DE, respectively as their recognition rates. Extensive discussions are made on their relative superiorities and practicalities.

DOI： 10.1007/s11704-014-3207-6

Language：English   Publishing type：Research paper (other academic)  

A new decoding for hidden Markov models is presented. As opposed to the commonly used Viterbi algorithm, it is based on the Min-CuUMax-Flow algorithm instead of dynamic programming. Therefore non-Markovian long-term dependencies can easily be added to influence the decoding path while still finding the optimal decoding in polynomial time. We demonstrate through an experimental evaluation how these constraints can be used to improve an HMM-based handwritten word recognition system that model words via linear character-HMM by restricting the length of each character.

DOI： 10.1109/ICFHR.2014.29

Language：English   Publishing type：Research paper (other academic)  

This paper proposes a constrained AdaBoost algorithm for utilizing global features in a dynamic time warping (DTW) framework. Global features are defined as a spatial relationship between temporally-distant points of a temporal pattern and are useful to represent global structure of the pattern. An example is the spatial relationship between the first and the last points of a handwritten pattern of the digit "0". Those temporally-distant points should be spatially close enough to form a closed circle, whereas those points of "6" should be distant enough. For a temporal pattern of an N-point sequence, it is possible to have N(N - 1)/2 global features. One problem of using the global features is that they are not ordered as a one-dimensional sequence any more. Consequently, it is impossible to use them in a left-to-right Markovian model, such as DTW and HMM. The proposed constrained AdaBoost algorithm can select a totally-ordered subset from the set of N (N - 1)/2 global features. Since the totally-ordered features can be arranged as a one-dimensional sequence, they can be incorporated into a DTW framework for compensating nonlinear temporal fluctuation. Since the selection is governed by the AdaBoost framework, the selected features can retain discriminative power.

DOI： 10.1109/ICFHR.2014.72

Language：English   Publishing type：Research paper (other academic)  

The purpose of writing this paper is two-fold. First, it presents a novel signature stability analysis based on signature's local / part-based features. The Speeded Up Local features (SURF) are used for local analysis which give various clues about the potential areas from whom the features should be exclusively considered while performing signature verification. Second, based on the results of the local stability analysis we present a novel signature verification system and evaluate this system on the publicly available dataset of forensic signature verification competition, 4NSigComp2010, which contains genuine, forged, and disguised signatures. The proposed system achieved an equal error rate of 15%, which is considerably very low when compared against all the participants of the said competition. Furthermore, we also compare the proposed system with some of the earlier reported systems on the said data. The proposed system also outperforms these systems.

DOI： 10.1109/ICFHR.2014.109

Language：English   Publishing type：Research paper (other academic)  

While bi-directional long short-term (BLSTM) neural network have been demonstrated to perform very well for English or Arabic, the huge number of different output classes (characters) encountered in many Asian fonts, poses a severe challenge. In this work we investigate different encoding schemes of Bangla compound characters and compare the recognition accuracies. We propose to model complex characters not as unique symbols, which are represented by individual nodes in the output layer. Instead, we exploit the property of long-distance-dependent classification in BLSTM neural networks. We classify only basic strokes and use special nodes which react to semantic changes in the writing, i.e., distinguishing inter-character spaces from intra-character spaces. We show that our approach outperforms the common approaches to BLSTM neural network-based handwriting recognition considerably.

DOI： 10.1007/978-3-662-44415-3_41

Language：English   Publishing type：Research paper (other academic)  

A method of concealing characters is proposed for degrading legibility of privacy sensitive textual information in natural scene, such as car license plate numbers and name tags. An important property of the proposed method is that it realizes selective concealing of characters; that is, the proposed method degrades legibility of character regions without degrading the quality of non-character regions. This selective concealment is realized because characters have special concealment characteristics. Specifically, character legibility can be degraded by damaging stroke structure by using exemplar-based image inpainting, which does not affect non-character regions. Experimental results of qualitative and quantitative evaluations have proven that the selective concealment is practically possible. Furthermore, the quantitative evaluation through a subjective experiment revealed appropriate setups of image inpainting for maximizing selective concealment performance.

DOI： 10.1109/ICPR.2014.66

Language：English   Publishing type：Research paper (other academic)  

In this paper a method for Early Recognition (ER) of Motion Templates (MTs) is presented. We define ER as an algorithm to provide recognition results before a motion sequence is completed. In our experiments we apply Long Short-Term Memory (LSTM) and optimize the training for the task of recognizing the motion template as early as possible. The evaluation has shown that the recognition accuracy for a frame-by-frame classification the LSTM achieves a recognition accuracy of 88% if no training data of the person him/herself is included, and 92% if the training data also contains motion sequences of the person. Furthermore, the average earliness - the number of time frames it takes before the LSTM correctly classifies a motion pattern - is around 24.77 frames, which is less than a second with the used tracking technology, i.e., the Microsoft Kinect.

DOI： 10.1109/ICPR.2014.611

Language：English   Publishing type：Research paper (other academic)  

At the current rate of technological advancement and social acceptance thereof, it will not be long before wearable devices will be common that constantly record the field of view of the user. We introduce a new database of image sequences, taken with a first person view camera, of realistic, everyday scenes. As a distinguishing feature, we manually transcribed the scene text of each image. This way, sophisticated OCR algorithms can be simulated that can help in the recognition of the location and the activity. To test this hypothesis, we performed a set of experiments using visual features, textual features, and a combination of both. We demonstrate that, although not very powerful when considered alone, the textual information improves the overall recognition rates.

DOI： 10.1109/ICPR.2014.512

Language：English   Publishing type：Research paper (scientific journal)  

This paper focuses on the importance of global features for online character recognition. Global features represent the relationship between two temporally distant points in a handwriting pattern. For example, it can be defined as the relative vector of two xy-coordinate features of two temporally separated points. Most existing online character recognition methods do not utilize global features, since their non-Markovian property prevents the use of the traditional recognition methodologies, such as dynamic time warping and hidden Markov models. However, we can understand the importance of, for example, the relationship between the starting and the ending points by attempting to discriminate "0" and "6". This relationship cannot be represented by local features defined at individual points but by global features. Since O(N-2) global features can be extracted from a handwriting pattern with N points, selecting those that are truly discriminative is very important. In this paper, AdaBoost is employed for feature selection. Experiments prove that many global features are discriminative and the combined use of local and global features can improve the recognition accuracy. (C) 2013 Elsevier B.V. All rights reserved.

DOI： 10.1016/j.patrec.2013.03.036

Language：English  

This chapter reviews techniques on text localization and recognition in scene images captured by camera. Since properties of scene texts are very different from scanned documents in various aspects, specific techniques are necessary to localize and recognize them. In fact, localization of scene text is a difficult and important task because there is no prior information on the location, layout, direction, size, typeface, and color of texts in a scene image in general and there are many textures and patterns similar to characters. In addition, recognition of scene text is also a difficult task because there are many characters distorted by blurring, perspective, nonuniform lighting, and low resolution. Decoration of characters makes the recognition task far more difficult. As reviewed in this chapter, those difficult tasks have been tackled with not only modified versions of conventional OCR techniques but also state-of-the-art computer vision and pattern recognition methodologies.

DOI： 10.1007/978-0-85729-859-1_28

Language：English   Publishing type：Research paper (scientific journal)  

In tracking analysis, the movement of cargos by motor proteins in axons is often represented by a time-space plot termed a kymograph'. Manual creation of kymographs is time-consuming and complicated for cell biologists. Therefore, we developed KYMOMAKER, a simple system that automatically creates a kymograph from a movie without generating multiple time-dissected movie stacks. In addition, KYMOMAKER can automatically extract faint vesicle traces, and can thereby effectively analyze cargos expressed at low levels in axons. A filter can be applied to remove traces of non-physiological movements and to extract meaningful traces of anterograde or retrograde cargo transport. For example, only cargos that move at a speed of >0.4 mu m/second for a distance of >1 mu m can be included. Another function of KYMOMAKER is to create a color kymograph in which the color of the trace varies according to the position of the fluorescent particle in the axis perpendicular to the long axis of the axon. Such positional information is completely lost in conventional kymographs. KYMOMAKER is an open access program that can be easily used to analyze vesicle transport in axons by cell biologists who do not have specific knowledge of bioimage informatics.

DOI： 10.1111/tra.12127

Language：English   Publishing type：Research paper (scientific journal)  

We propose a camera-based method for digital recovery of handwritings on ordinary paper. Our method is characterized by the following two points: (1) it requires no special device such as special paper other than a camera-pen to recover handwritings, (2) if the handwriting is on a printed document, the method is capable of localizing it onto an electronic equivalent of the printed document. The above points are enabled by the following processing. The handwriting is recovered by the LK tracking to trace the move of the pen-tip. The recovered shape is localized onto the corresponding part of the electronic document with the help of document image retrieval called LLAH (locally likely arrangement hashing). A new framework for stably estimating the homography from a camera-captured image to the corresponding electronic document allows us to localize the recovered handwritings accurately. We experimentally evaluate the accuracy, processing time and memory usage of the proposed method using 30 handwritings. From the comparison to other methods that implement alternative ways for realizing the same functionality, we have confirmed that the proposed method is superior to those other methods. (C) 2012 Elsevier B.V. All rights reserved.

DOI： 10.1016/j.patrec.2012.10.003

Language：English   Publishing type：Research paper (scientific journal)  

In this paper we present a novel digital pen device, called data-embedding pen, for enhancing the value of handwriting on physical paper. This pen produces an additional ink-dot sequence along a written stroke during writing. This ink-dot sequence represents arbitrary information, such as writer's name and writing date. Since the information is placed on the paper as an ink-dot sequence, it can be retrieved just by scanning or photographing the paper. In addition to the hardware of the data-embedding pen, this paper also proposes a coding scheme for reliable data-embedding and retrieval. In fact, the physical data-embedding on a paper will undergo various severe errors and therefore a robust coding scheme is necessary. Through experiments on data written by two writers, we show that we can embed 32 bits on short and simple or even on more complex patterns and finally retrieve them with a high reliability. (C) 2012 Elsevier B.V. All rights reserved.

DOI： 10.1016/j.patrec.2012.09.001

Language：English   Publishing type：Research paper (scientific journal)  

Recently, compressive sensing-based encryption methods which combine sampling, compression and encryption together have been proposed. However, since the quantized measurement data obtained from linear dimension reduction projection directly serve as the encrypted image, the existing compressive sensing-based encryption methods fail to resist against the chosen-plaintext attack. To enhance the security, a block cipher structure consisting of scrambling, mixing, S-box and chaotic lattice XOR is designed to further encrypt the quantized measurement data. In particular, the proposed method works efficiently in the parallel computing environment. Moreover, a communication unit exchanges data among the multiple processors without collision. This collision-free property is equivalent to optimal diffusion. The experimental results demonstrate that the proposed encryption method not only achieves the remarkable confusion, diffusion and sensitivity but also outperforms the existing parallel image encryption methods with respect to the compressibility and the encryption speed. © 2012 Springer Science+Business Media New York.

DOI： 10.1007/s11042-012-1337-0

Language：English   Publishing type：Research paper (other academic)  

Development of automatic multiple intracellular-objects tracking methods is one of the significant challenges in Bioimage-Informatics. The challenge becomes more difficult in case the tracking targets have the same shape and appearance. In order to obtain stable results under that condition, we propose a tracking method based on global optimization. Particularly, we first detect tracking targets by our proposed detection method. Then, we formulate the multiple object tracking problem as a combinatorial optimization problem over a pair of consecutive frames. Finally, we solve the problem by the stable marriage algorithm. In this paper, we describe our proposed detection and tracking methods.

DOI： 10.1109/CANDAR.2013.53

Language：English   Publishing type：Research paper (other academic)  

This paper provides a bioimage informatics system of detecting and tracking protein molecules, called APP-GFPs, in a live-cell video captured by a fluorescent microscope. Since both processes encounter many difficulties such as many targets, less appearance information, and heavy background noise, we will try to design the system as robust as possible. Specifically, for the detection, a machine learning-based method is employed. For tracking, a method based on a global optimization strategy is newly developed. Experimental results showed that the speed and direction distributions of molecular motion by the proposed system were very similar to that by manual inspection.

DOI： 10.1109/CANDAR.2013.47

Language：English   Publishing type：Research paper (scientific journal)  

We propose a novel method for recognizing sequential patterns such as motion trajectory of biological objects (i.e., cells, organelle, protein molecules, etc.), human behavior motion, and meteorological data. In the proposed method, a local classifier is prepared for every point (or timing or frame) and then the whole pattern is recognized by majority voting of the recognition results of the local classifiers. The voting strategy has a strong benefit that even if an input pattern has a very large deviation from a prototype locally at several points, they do not severely influence the recognition result; they are treated just as several incorrect votes and thus will be neglected successfully through the majority voting. For regularizing the recognition result, we introduce partial-dependency to local classifiers. An important point is that this dependency is introduced to not only local classifiers at neighboring point pairs but also to those at distant point pairs. Although, the dependency makes the problem non-Markovian (i.e., higher-order Markovian), it can still be solved efficiently by using a graph cut algorithm with polynomial-order computations. The experimental results revealed that the proposed method can achieve better recognition accuracy while utilizing the above characteristics of the proposed method.

DOI： 10.1371/journal.pone.0076980

Language：English   Publishing type：Research paper (scientific journal)  

To handle the variety of scene characters, we propose a cooperative multiple-hypothesis framework which consists of an image operator set module, an Optical Character Recognition (OCR) module and an integration module. Multiple image operators activated by multiple parameters probe suspected character regions. The OCR module is then applied to each suspected region and returns multiple candidates with weight values for future integration. Without the aid of the heuristic rules which impose constraints on segmentation area, aspect ratio, color consistency, text line orientations, etc., the integration module automatically prunes the redundant detection/recognition and pads the missing detection/recognition. The proposed framework bridges the gap between scene character detection and recognition, in the sense that a practical OCR engine is effectively leveraged for result refinement. In addition, the proposed method achieves the detection and recognition at the character level, which enables dealing with special scenarios such as single character, text along arbitrary orientations or text along curves. We perform experiments on the benchmark ICDAR 2011 Robust Reading Competition dataset which includes a text localization task and a word recognition task. The quantitative results demonstrate that multiple hypotheses outperform a single hypothesis, and be comparable with state-of-the-art methods in terms of recall, precision, F-measure, character recognition rate, total edit distance and word recognition rate. Moreover, two additional experiments are conducted to confirm the simplicity of parameter setting in this proposal.

DOI： 10.1587/transinf.E96.D.2235

Language：English  

Applying mobile sensing technology to cognitive tasks will enable novel forms of activity recognition.

DOI： 10.1109/MC.2013.339

Language：English  

Language：English   Publishing type：Research paper (other academic)  

Reading life log is a type of techniques to automatically and unconsciously record people's reading intentions, interests and habits. Besides, it can also serve as various assistants in our daily life. In this paper, a reading-life log system is implemented by a head-mounted and unobtrusive video camera with a high resolution and a high shutter speed. We utilize DP matching, and propose a text-based frame mosaicing method to integrate multiple frames in a clip. The developed system is tested in the various environments indoor and outdoor. The experimental results show that our system can provide reliable outputs with respect to the most correct responses. The infrequent misregistration between lines also indicates the feasibility and validity of the text-based frame mosaicing.

DOI： 10.1109/ICDAR.2013.26

Language：English   Publishing type：Research paper (other academic)  

Edge is a type of valuable clues for scene character detection task. Generally, the existing edge-based methods rely on the assumption of straight text line to prune away the non-character candidates. This paper proposes a new edge-based method, called edge-ray filter, to detect the scene character. The main contribution of the proposed method lies in filtering out complex backgrounds by fully utilizing the essential spatial layout of edges instead of the assumption of straight text line. Edges are extracted by a combination of Canny and Edge Preserving Smoothing Filter (EPSF). To effectively boost the filtering strength of the designed edge-ray filter, we employ a new Edge Quasi-Connectivity Analysis (EQCA) to unify complex edges as well as contour of broken character. Label Histogram Analysis (LHA) then filters out non-character edges and redundant rays through setting proper thresholds. Finally, two frequently-used heuristic rules, namely aspect ratio and occupation, are exploited to wipe off distinct false alarms. In addition to have the ability to handle special scenarios, the proposed method can accommodate dark-on-bright and bright-on-dark characters simultaneously, and provides accurate character segmentation masks. We perform experiments on the benchmark ICDAR 2011 Robust Reading Competition dataset as well as scene images with special scenarios. The experimental results demonstrate the validity of our proposal.

DOI： 10.1109/ICDAR.2013.99

Language：English   Publishing type：Research paper (other academic)  

In this paper, the part-based recognition method is introduced and applied to the arbitrary font recognition. The principle of the part-based method is to represent the character image as a set of parts and then recognize the image by finding the most possible parts set from the reference database. Since the part-based method does not rely on the global structure of a character, it is supposed to be robust against the variant appearances of the character. The experiment results indicate that it is possible to apply the part-based method to the font recognition, which is always considered as a difficult task by most of the researchers.

DOI： 10.1109/ICDAR.2013.41

Language：English   Publishing type：Research paper (other academic)  

In this paper, we propose a structure learning-based scene character detector which is inspired by the observation that characters have their own inherent structures compared with the background. Graphs are extracted from the thinned binary image to represent the topological line structures of scene contents. Then, a graph classifier, namely gBoost classifier, is trained with the intent to seek out the inherent structures of character and the counterparts of non-character. The experimental results show that the proposed detector achieves the remarkable classification performance with the accuracy of about 70%, which demonstrates the existence and separability of the inherent structures.

DOI： 10.1109/ICDAR.2013.101

Language：English   Publishing type：Research paper (other academic)  

This report presents the final results of the ICDAR 2013 Robust Reading Competition. The competition is structured in three Challenges addressing text extraction in different application domains, namely born-digital images, real scene images and real-scene videos. The Challenges are organised around specific tasks covering text localisation, text segmentation and word recognition. The competition took place in the first quarter of 2013, and received a total of 42 submissions over the different tasks offered. This report describes the datasets and ground truth specification, details the performance evaluation protocols used and presents the final results along with a brief summary of the participating methods.

DOI： 10.1109/ICDAR.2013.221

Language：English   Publishing type：Research paper (other academic)  

The off-the-shelf Optical Character Recognition (OCR) engines return mediocre performance on the decorative characters which usually appear in natural scenes such as signboards. A reasonable way towards the so-called camera-based OCR is to collect a large-scale font set and analyze the distribution of font samples for realizing some character recognition engine which is tolerant to font shape variations. This paper is concerned with the issue of font distribution analysis by network. Minimum Spanning Tree (MST) is employed to construct font network with respect to Chamfer distance. After clustering, some centrality criterion, namely closeness centrality, eccentricity centrality or betweenness centrality, is introduced for extracting typical font samples. The network structure allows us to observe the font shape transition between any two samples, which is useful to create new fonts and recognize unseen decorative characters. Moreover, unlike the Principal Component Analysis (PCA), the font network fulfills distribution visualization through measuring the dissimilarity between samples rather than the lossy processing of dimensionality reduction. Compared with K-means algorithm, network-based clustering has the ability to preserve small size font clusters which generally consist of samples taking special appearances. Experiments demonstrate that the proposed network-based analysis is an effective way to grasp font distribution, and thus provides helpful information for decorative character recognition.

DOI： 10.1007/978-3-319-05167-3_7

Language：English   Publishing type：Research paper (other academic)  

The goal of this research is to understand the true distribution of character patterns. Advances in computer technology for mass storage and digital processing have paved way to process a massive dataset for various pattern recognition problems. If we can represent and analyze the distribution of a large-scale character pattern set directly and understand its relationships deeply, it should be helpful for improving character recognizer. For this purpose, we propose a network analysis method to represent the distribution of patterns using a relative neighborhood graph and its clustered version. In this paper, the properties and validity of the proposed method are confirmed on 410,564 machine-printed digit patterns and 622,660 handwritten digit patterns which were manually ground-truthed and resized to 16 times 16 pixels. Our network analysis method represents the distribution of the patterns without any assumption, approximation or loss.

DOI： 10.1109/ICDAR.2013.10

Language：English   Publishing type：Research paper (other academic)  

Visual saliency models have been introduced to the field of character recognition for detecting characters in natural scenes. Researchers believe that characters have different visual properties from their non-character neighbors, which make them salient. With this assumption, characters should response well to computational models of visual saliency. However in some situations, characters belonging to scene text mignt not be as salient as one might expect. For instance, a signboard is usually very salient but the characters on the signboard might not necessarily be so salient globally. In order to analyze this hypothesis in more depth, we first give a view of how much these background regions, such as sign boards, affect the task of saliency-based character detection in natural scenes. Then we propose a hierarchical-saliency method for detecting characters in natural scenes. Experiments on a dataset with over 3,000 images containing scene text show that when using saliency alone for scene text detection, our proposed hierarchical method is able to capture a larger percentage of text pixels as compared to the conventional single-pass algorithm.

DOI： 10.1007/978-3-319-05167-3_2

Language：English  

Language：English   Publishing type：Research paper (scientific journal)  

In this paper, we intensively study the behavior of three part-based methods for handwritten digit recognition. The principle of the proposed methods is to represent a handwritten digit image as a set of parts and recognize the image by aggregating the recognition results of individual parts. Since part-based methods do not rely on the global structure of a character, they are expected to be more robust against various deformations which may damage the global structure. The proposed three methods are based on the same principle but different in their details, for example, the way of aggregating the individual results. Thus, those methods have different performances. Experimental results show that even the simplest part-based method can achieve recognition rate as high as 98.42% while the improved one achieved 99.15%, which is comparable or even higher than some state-of-the-art method. This result is important because it reveals that characters can be recognized without their global structure. The results also show that the part-based method has robustness against deformations which usually appear in handwriting.

DOI： 10.1007/s11704-013-2297-x

Language：English  

Language：English  

Language：English   Publishing type：Research paper (scientific journal)  

This paper proposes a new part-based approach for skew estimation of document images. The proposed method first estimates skew angles on rather small areas, which are the local parts of characters, and subsequently determines the global skew angle by aggregating those local estimations. A local skew estimation on a part of a skewed character is performed by finding an identical part from prepared upright character images and calculating the angular difference. Specifically, a keypoint detector (e.g. SURF) is used to determine the local parts of characters, and once the parts are described as feature vectors, a nearest neighbor search is conducted in the instance database to identify the parts. Finally, a local skew estimation is acquired by calculating the difference of the dominant angles of brightness gradient of the parts. After the local skew estimation, the global skew angle is estimated by the majority voting of those local estimations, disregarding some noisy estimations. Our experiments have shown that the proposed method is more robust to short and sparse text lines and non-text backgrounds in document images compared to conventional methods.

DOI： 10.1587/transinf.E96.D.1503

Language：English   Publishing type：Research paper (other academic)  

A Proposal of Writing-Life Log and Its Implementation Using a Retrieval-Based Camera-Pen

Language：English  

This paper reviews image processing and pattern recognition techniques, which will be useful to analyze bioimages. Although this paper does not provide their technical details, it will be possible to grasp their main tasks and typical tools to handle the tasks. Image processing is a large research area to improve the visibility of an input image and acquire some valuable information from it. As the main tasks of image processing, this paper introduces gray-level transformation, binarization, image filtering, image segmentation, visual object tracking, optical flow and image registration. Image pattern recognition is the technique to classify an input image into one of the predefined classes and also has a large research area. This paper overviews its two main modules, that is, feature extraction module and classification module. Throughout the paper, it will be emphasized that bioimage is a very difficult target for even state-of-the-art image processing and pattern recognition techniques due to noises, deformations, etc. This paper is expected to be one tutorial guide to bridge biology and image processing researchers for their further collaboration to tackle such a difficult target.

DOI： 10.1111/dgd.12054

Language：English   Publishing type：Research paper (other academic)  

This paper proposes a new dynamic time warping (DTW) method, called non-Markovian DTW In the conventional DTW, the warping function is optimized generally by dynamic programming (DP) subject to some Markovian constraints which restrict the relationship between neighboring time points. In contrast, the non-Markovian DTW can introduce non-Markovian constraints for dealing with the relationship between points with a large time interval. This new and promising ability of DTW is realized by using graph cut as the optimizer of the warping function instead of DP. Specifically, the conventional DTW problem is first converted as an equivalent minimum cut problem on a graph and then edges representing the non-Markovian constraints are added to the graph. An experiment on online character recognition showed the advantage of using non-Markovian constraints during DTW.

Language：English   Publishing type：Research paper (other academic)  

To handle the diversity of scene characters, we propose a multiple hypotheses framework which consists of an image operator set module, an optical character recognition (OCR) module, and an integration module. Image operators detect multiple suspicious character areas. The OCR engine is then applied to each detected area and returns multiple candidates with weight values for future integration. Without the aid of heuristic constraints on area, aspect ratio or color etc., the integration module prunes the redundant detection and pads the missing detection based on the outputs of OCR. The experimental results demonstrate that the whole multiple hypotheses outperforms each operator's hypotheses and be comparable with existing methods in terms of recall, precision, F-measure and recognition rate.

Language：English   Publishing type：Research paper (other academic)  

This paper reports a trial of handwritten text recognition by a part-based method. The part-based method recognizes individual characters by their parts without considering their whole shape. This realizes great robustness to severe deformations. This robustness is also effective for text recognition. Especially, for handwritten texts whose segmentation into individual characters is very difficult by deep touching and heavy slant, the part-based method still can recognize them because it does not request segmentation results to provide their whole shapes. Experimental results using digit sequences proved this robustness.

Language：English   Publishing type：Research paper (other academic)  

In this paper, we show that the truly two-dimensional elastic image matching problem can be solved analytically using dynamic programming (DP) in polynomial time if the problem is formulated as a maximum a posteriori problem using Gaussian distributions for the likelihood and prior. After giving the derivation of the analytical DP matching algorithm, we evaluate its performance on handwritten character images containing various nonlinear deformations, and compare other elastic image matching methods.

DOI： 10.1007/978-3-642-33863-2_10

Language：English   Publishing type：Research paper (other academic)  

This paper tackles the stroke recovery problem, which is a typical ill-posed reverse problem, by an instance-based method. The basic idea of the instance-based stroke recovery is to refer to the drawing order of a similar instance. The instance-based method has a strong merit that it can deal with multi-stroke characters and other complex characters without any special consideration. However, it requires a sufficient numbers of instances to cover those various characters. As an initial trial of the instance-based stroke recovery method, this paper describes the principle of the method and then provides several experimental results. The experimental results indicate the potential of the proposed method on recovering the drawing order of complex characters, as expected.

DOI： 10.1109/ICFHR.2012.248

Language：English   Publishing type：Research paper (other academic)  

This paper proposes a dynamic programming (DP) matching method with global features for online character recognition. Many online character recognition methods have utilized the ability of DP matching on compensating temporal fluctuation. On the other hand, DP requires the Markovian property on its matching process. Consequently, most traditional DP matching methods have utilized local information of strokes such as xy-coordinates or local directions as features, because it is easy to satisfy the Markovian property with those features. Unfortunately, these local features cannot represent global structure of character shapes. Although global features that extract global structures of characters have high potential to represent various key characteristics of character shapes, conventional DP matching methods cannot handle global features. This is because the incorporation of global features is not straightforward due to the Markovian property of DP. In this paper we propose a new scheme for DP matching using global features. Our method first selects global features which not only satisfy the Markovian property but also have sufficient discrimination ability. By embedding the selected global features into DP matching process, we can compensate temporal fluctuation while considering the global structure of the pattern. Experimental results show that our methods can enhance the recognition accuracy for online numeral characters.

DOI： 10.1109/ICFHR.2012.199

Language：English   Publishing type：Research paper (other academic)  

The ambitious goal of this research is to understand the real distribution of character patterns. Ideally, if we can collect all possible character patterns, we can totally understand how they are distributed in the image space. In addition, we also have the perfect character recognizer because we know the correct class for any character image. Of course, it is practically impossible to collect all those patterns - however, if we collect character patterns massively and analyze how the distribution changes according to the increase of patterns, we will be able to estimate the real distribution asymptotically. For this purpose, we use 822,714 manually ground-truthed 32 x 32 handwritten digit patterns in this paper. The distribution of those patterns are observed by nearest neighbor analysis and network analysis, both of which do not make any approximation (such as low-dimensional representation) and thus do not corrupt the details of the distribution.

DOI： 10.1109/ICFHR.2012.190

Language：English   Publishing type：Research paper (scientific journal)  

Recognizing characters in scene images suffering from perspective distortion is a challenge. Although there are some methods to overcome this difficulty, they are time-consuming. In this paper, we propose a set of affine-invariant features and a new recognition scheme called progressive removing that can help reduce the processing time. Progressive removing gradually removes less feasible categories and skew angles by using multiple classifiers. We observed that progressive removing and the use of the affine invariant features reduced the processing time by about 60% in comparison to a trivial algorithm without decreasing the recognition rate. (c) 2012 Wiley Periodicals, Inc. Electr Eng Jpn, 180(2): 5563, 2012; Published online in Wiley Online Library (). DOI 10.1002/eej.22276

DOI： 10.1002/eej.22276

Language：English  

Language：English  

DOI： 10.1109/DAS.2012.55

Language：English   Publishing type：Research paper (other academic)  

In this paper we propose a part-based skew estimation method which is more robust to larger varieties of text images, such as camera-captured scene images. Specifically, the skew angle at each local part of the input image is estimated independently by referring the local part of upright character images stored as a database. Then the global skew angle is estimated by aggregating the estimated local skews. The proposed method does not assume that characters are laid-out in straight lines and thus have more robustness to the varieties of text images than conventional methods. The experimental results show the advantage of the proposed method over the conventional methods under several conditions. © 2012 IEEE.

DOI： 10.1109/DAS.2012.7

Language：English   Publishing type：Research paper (other academic)  

Document image decoding (DID) is a trial to understand the contents of a whole document without any reference information about font, language, etc. Typically, DID approaches assume the correct segmentation of the document and some a priori knowledge about the language or the script. Unfortunately, this assumption will not hold if we deal with various documents, such as documents with various sized fonts, camera-captured documents, free-layout documents, or historical documents. In this paper, we propose a part-based character identification method where no segmentation into characters is necessary and no a priori information about the document is needed. The approach clusters similar key points and groups frequent neighboring key point clusters. Then a second iteration is performed, i.e., the groups are again clustered and optionally pairs frequent group clusters are detected. Our first experimental results on multi font-size documents look already very promising. We could find nearly perfect correspondences between characters and detected group clusters. © 2012 IEEE.

DOI： 10.1109/DAS.2012.90

Language：English   Publishing type：Research paper (other academic)  

Computational models of visual attention use image features to identify salient locations in an image that are likely to attract human attention. Attention models have been quite effectively used for various object detection tasks. However, their use for scene text detection is under-investigated. As a general observation, scene text often conveys important information and is usually prominent or salient in the scene itself. In this paper, we evaluate four state-of-the-art attention models for their response to scene text. Initial results indicate that saliency maps produced by these attention models can be used for aiding scene text detection algorithms by suppressing non-text regions. © 2012 IEEE.

DOI： 10.1109/DAS.2012.42

Language：English   Publishing type：Research paper (other academic)  

This paper studies the importance of the features that represent the global structure of character strokes to character recognition. Most existing character recognition methods based on character stroke features utilize a set or a sequence of local features such as xy-coordinates and local direction of strokes. This is natural from the viewpoint that each stroke is a trajectory and thus can be represented as a sequence of local features. This viewpoint, however, has a clear limitation in that local features cannot deal with global structure directly. For example, the sequence of local features cannot deal with the fact that the two end points of character "0" should be close to each other. In this paper we propose a simple and novel global feature that describes the global structure of the character shape of each class. We prove the importance of the global feature through a feature selection experiment. Specifically, we show that the global features are more often selected than local features to enhance classification accuracy under the AdaBoost-based machine learning framework. Recognition experiments using online numeral data show also that the use of global features improves recognition accuracy. © 2012 IEEE.

DOI： 10.1109/DAS.2012.41

Language：English  

Odor gas detection is important for the detection of explosives, environmental sensing, biometrics, foodstuffs and a comfortable life. Such odor-source localizations is an active research area for robotics. In this study, we tried to detect odor chemicals with an optical method that can be applied for the spatiotemporal detection of odor. We used four types of fluorescence dyes; tryptophan, quinine sulfate, acridine orange, and 1-anilinonaphthalene-8-sulfonate (ANS). As analyses, we measured the following four odor chemicals, 2-furaldehyde, vanillin, acetophenone, and benzaldehyde. The fluorescence-quenching mechanism of PET (Photoinduced Electron Transfer) or FRET (Fluorescence Resonance Electron Transfer), which occur between fluorescence dyes and odor compounds, could prevent unintended detection of various odorants that is caused by their unspecific adsorption onto the detecting materials. The fluorescence changes were then observed. Thus, we could detect the odor substances through fluorescent quenching by using the fluorescence dyes. Odor information could be obtained by response patterns across all the fluorescence dyes. Moreover, we captured odor images with a cooled CCD camera. Shapes of the targets that emitted odor could be roughly recognized by the odor-shape images. From the spatiotemporal images of odors, twodimensional odor expanse could be obtained.

DOI： 10.20965/jrm.2012.p0047

Language：English  

Odor gas detection is important for the detection of explosives, environmental sensing, biometrics, foodstuffs and a comfortable life. Such odor-source localizations is an active research area for robotics. In this study, we tried to detect odor chemicals with an optical method that can be applied for the spatiotemporal detection of odor. We used four types of fluorescence dyes; tryptophan, quinine sulfate, acridine orange, and 1-anilinonaphthalene-8-sulfonate (ANS). As analyses, we measured the following four odor chemicals, 2-furaldehyde, vanillin, acetophenone, and benzaldehyde. The fluorescence-quenching mechanism of PET (Photoinduced Electron Transfer) or FRET (Fluorescence Resonance Electron Transfer), which occur between fluorescence dyes and odor compounds, could prevent unintended detection of various odorants that is caused by their unspecific adsorption onto the detecting materials. The fluorescence changes were then observed. Thus, we could detect the odor substances through fluorescent quenching by using the fluorescence dyes. Odor information could be obtained by response patterns across all the fluorescence dyes. Moreover, we captured odor images with a cooled CCD camera. Shapes of the targets that emitted odor could be roughly recognized by the odor-shape images. From the spatiotemporal images of odors, twodimensional odor expanse could be obtained.

DOI： 10.20965/jrm.2012.p0047

Language：English  

Odor gas detection is important for the detection of explosives, environmental sensing, biometrics, foodstuffs and a comfortable life. Such odor-source localizations is an active research area for robotics. In this study, we tried to detect odor chemicals with an optical method that can be applied for the spatiotemporal detection of odor. We used four types of fluorescence dyes; tryptophan, quinine sulfate, acridine orange, and 1-anilinonaphthalene-8-sulfonate (ANS). As analyses, we measured the following four odor chemicals, 2-furaldehyde, vanillin, acetophenone, and benzaldehyde. The fluorescence-quenching mechanism of PET (Photoinduced Electron Transfer) or FRET (Fluorescence Resonance Electron Transfer), which occur between fluorescence dyes and odor compounds, could prevent unintended detection of various odorants that is caused by their unspecific adsorption onto the detecting materials. The fluorescence changes were then observed. Thus, we could detect the odor substances through fluorescent quenching by using the fluorescence dyes. Odor information could be obtained by response patterns across all the fluorescence dyes. Moreover, we captured odor images with a cooled CCD camera. Shapes of the targets that emitted odor could be roughly recognized by the odor-shape images. From the spatiotemporal images of odors, twodimensional odor expanse could be obtained.

DOI： 10.20965/jrm.2012.p0047

Language：English  

Odor gas detection is important for the detection of explosives, environmental sensing, biometrics, foodstuffs and a comfortable life. Such odor-source localizations is an active research area for robotics. In this study, we tried to detect odor chemicals with an optical method that can be applied for the spatiotemporal detection of odor. We used four types of fluorescence dyes; tryptophan, quinine sulfate, acridine orange, and 1-anilinonaphthalene-8-sulfonate (ANS). As analyses, we measured the following four odor chemicals, 2-furaldehyde, vanillin, acetophenone, and benzaldehyde. The fluorescence-quenching mechanism of PET (Photoinduced Electron Transfer) or FRET (Fluorescence Resonance Electron Transfer), which occur between fluorescence dyes and odor compounds, could prevent unintended detection of various odorants that is caused by their unspecific adsorption onto the detecting materials. The fluorescence changes were then observed. Thus, we could detect the odor substances through fluorescent quenching by using the fluorescence dyes. Odor information could be obtained by response patterns across all the fluorescence dyes. Moreover, we captured odor images with a cooled CCD camera. Shapes of the targets that emitted odor could be roughly recognized by the odor-shape images. From the spatiotemporal images of odors, twodimensional odor expanse could be obtained.

DOI： 10.20965/jrm.2012.p0047

Language：English  

Odor gas detection is important for the detection of explosives, environmental sensing, biometrics, foodstuffs and a comfortable life. Such odor-source localizations is an active research area for robotics. In this study, we tried to detect odor chemicals with an optical method that can be applied for the spatiotemporal detection of odor. We used four types of fluorescence dyes; tryptophan, quinine sulfate, acridine orange, and 1-anilinonaphthalene-8-sulfonate (ANS). As analyses, we measured the following four odor chemicals, 2-furaldehyde, vanillin, acetophenone, and benzaldehyde. The fluorescence-quenching mechanism of PET (Photoinduced Electron Transfer) or FRET (Fluorescence Resonance Electron Transfer), which occur between fluorescence dyes and odor compounds, could prevent unintended detection of various odorants that is caused by their unspecific adsorption onto the detecting materials. The fluorescence changes were then observed. Thus, we could detect the odor substances through fluorescent quenching by using the fluorescence dyes. Odor information could be obtained by response patterns across all the fluorescence dyes. Moreover, we captured odor images with a cooled CCD camera. Shapes of the targets that emitted odor could be roughly recognized by the odor-shape images. From the spatiotemporal images of odors, twodimensional odor expanse could be obtained.

DOI： 10.20965/jrm.2012.p0047

Language：English  

Odor gas detection is important for the detection of explosives, environmental sensing, biometrics, foodstuffs and a comfortable life. Such odor-source localizations is an active research area for robotics. In this study, we tried to detect odor chemicals with an optical method that can be applied for the spatiotemporal detection of odor. We used four types of fluorescence dyes; tryptophan, quinine sulfate, acridine orange, and 1-anilinonaphthalene-8-sulfonate (ANS). As analyses, we measured the following four odor chemicals, 2-furaldehyde, vanillin, acetophenone, and benzaldehyde. The fluorescence-quenching mechanism of PET (Photoinduced Electron Transfer) or FRET (Fluorescence Resonance Electron Transfer), which occur between fluorescence dyes and odor compounds, could prevent unintended detection of various odorants that is caused by their unspecific adsorption onto the detecting materials. The fluorescence changes were then observed. Thus, we could detect the odor substances through fluorescent quenching by using the fluorescence dyes. Odor information could be obtained by response patterns across all the fluorescence dyes. Moreover, we captured odor images with a cooled CCD camera. Shapes of the targets that emitted odor could be roughly recognized by the odor-shape images. From the spatiotemporal images of odors, twodimensional odor expanse could be obtained.

DOI： 10.20965/jrm.2012.p0047

Language：English  

Language：English  

Language：English   Publishing type：Research paper (other academic)  

We consider personal identification using stroke order variations of online handwritten character patterns, which are written on, e.g., electric tablets. To extract the stroke order variation of an input character pattern, it is necessary to establish the accurate stroke correspondence between the input pattern and the reference pattern of the same category. In this paper we compare five stroke correspondence methods: the individual correspondence decision (ICD), the cube search (CS), the bipartite weighted matching (BWM), the stable marriage (SM), and the deviation-expansion model (DE). After their brief review, they are experimentally compared quantitatively by not only their stroke correspondence accuracy but also character recognition accuracy. The experimental results showed the superiority CS and BWM over ICD, SM and DE.

DOI： 10.1007/978-3-642-19376-7_4

Language：English   Publishing type：Research paper (other academic)  

A new generative model of handwriting patterns is proposed for interpreting their deformations. The model is based on feature desynchronization, which is a coupling process of x and y coordinate features of different timings. By changing the timings to be coupled, the model can generate various deformed patterns from a single pattern. The model is further enhanced by incorporating an adaptive rotation at each timing for increasing the variety of deformed patterns. An important fact is that this enhanced desynchronization model can be interpreted intuitively as a deformation process in actual handwriting. Experimental results showed that the model can generate various handwriting patterns close to actual deformed patterns.

DOI： 10.1109/ICDAR.2011.124

Language：English   Publishing type：Research paper (other academic)  

The purpose of this paper is to analyze how image patterns distribute inside their feature space. For this purpose, 832,612 manually ground-truthed handwritten digit patterns are used. Use of character patterns instead of general visual object patterns is very essential for our purpose. First, since there are only 10 classes for digits, it is possible to have an enough number of patterns per class. Second, since the feature space of small binary character images is rather compact, it is easier to observe the precise pattern distribution with a fixed number of patterns. Third, the classes of character patterns can be defined far more clearly than visual objects. Through nearest neighbor analysis on 832,612 patterns, their distribution in the 32 x 32 binary feature space is observed quantitatively and qualitatively. For example, the visual similarity of nearest neighbors and the existence of outliers, which are surrounded by patterns from different classes, are observed.

DOI： 10.1109/MLSP.2011.6064640

Language：English   Publishing type：Research paper (other academic)  

For scenery character detection, we introduce environmental context, which is modeled by scene components, such as sky and building. Environmental context is expected to regulate the probability of character existence at a specific region in a scenery image. For example, if a region looks like a part of a building, the region has a higher probability than another region like a part of the sky. In this paper, environmental context is represented by state-of-the-art texture and color features and utilized in two different ways. Through experimental results, it was clearly shown that the environmental context has an effect of improving detection accuracy.

DOI： 10.1109/ICDAR.2011.212

Language：English   Publishing type：Research paper (other academic)  

In this paper we propose a complete system for online stroke recovery from offline data. The key idea of our approach is to use a novel pen device which is able to embed meta information into the ink during writing the strokes. This pen-device overcomes the need to get access to any memory on the pen when trying to recover the information, which is especially useful in multi-writer or multi-pen scenarios. The actual data-embedding is achieved by an additional ink-dot sequence along a handwritten pattern during writing. We design the ink-dot sequence in such a way that it is possible to retrieve the writing direction from a scanned image. Furthermore, we propose novel processing steps in order to retrieve the original writing direction and finally the embedded data. In our experiments we show that we can reliably recover the writing direction of various patterns. Our system is able to determine the writing direction of straight lines, simple patterns with crossings (e.g., "x" and "ll"), and even more complex patterns like handwritten words and symbols.

DOI： 10.1109/ICDAR.2011.278

Language：English   Publishing type：Research paper (other academic)  

Part-based recognition is expected to be robust in difficult handwritten character recognition tasks. This is because part-based recognition is based on aggregation of independent recognition results at individual local parts without considering their global relations and thus is robust against various deformations, such as partial occlusion, overlap, broken stroke, etc. Since part-based recognition is a new approach, there are still several open problems toward its practical use. For example, compared with entire images, local parts are more ambiguous, i.e., less discriminative. For better recognition accuracy and less computations, we need to know the characteristics of local parts and then, for example, discard less discriminative parts. The purpose of this paper is to conduct some experiments in order to observe and analyze how the local parts of multiple classes are distributed in feature spaces. By handling parts appropriately based on the analysis, we will be able to enhance the usefulness of the part-based method.

DOI： 10.1109/ICDAR.2011.161

Language：English   Publishing type：Research paper (other academic)  

In this paper, we report recent work of the data-embedding pen, which adds an ink-dot sequence along a handwritten pattern during writing. The ink-dot sequence represents some information, such as writer's name, date of writing, and URL. This information drastically increases the value of handwriting on a paper. The embedded information can be extracted from the handwritten pattern by image processing techniques and a stroke recovery technique. Consequently, we can augment the handwritten pattern by the data-embedding pen to carry arbitrary information. © 2011 Springer-Verlag.

DOI： 10.1007/978-3-642-23866-6_22

Language：English   Publishing type：Research paper (other academic)  

The purpose of this paper is to introduce three part-based methods for handwritten character recognition and then compare their performances experimentally. All of those methods decompose handwritten characters into "parts". Then some recognition processes are done in a part-wise manner and, finally, the recognition results at all the parts are combined via voting to have the recognition result of the entire character. Since part-based methods do not rely on the global structure of the character, we can expect their robustness against various deformations. Three voting methods have been investigated for the combination: single voting, multiple voting, and class distance. All of them use different strategies for voting. Experimental results on the MNIST database showed the relative superiority of the class distance method and the robustness of the multiple voting method against the reduction of training set.

DOI： 10.1109/ICDAR.2011.167

Language：English   Publishing type：Research paper (other academic)  

This paper proposes a new approach to a method to estimate a skew angle of a rotated document image. This is realized by using Speeded-Up Robust Features (SURF), and the goal is that it enables the image to be rotated back to the correct orientation. SURF detects a number of keypoints both from the reference image on which a set of standard alphabets (e.g. letter eaf through ezf in a certain font) are written, and the image of the rotated document. Two nearest features each from the reference image and the input image are compared to decide to how many degrees the feature in the input image is rotated. Finally the skew angle of the whole input image( the global skew angle) is decided by the majority of the total votes of angles that have been calculated as mentioned above. © 2011 Springer-Verlag.

DOI： 10.1007/978-3-642-23866-6_21

Language：English   Publishing type：Research paper (other academic)  

This paper proposes a new approach toward scenery character detection. This is a keypoint-based approach where local features and a saliency map are fully utilized. Local features, such as SIFT and SURF, have been commonly used for computer vision and object pattern recognition problems; however, they have been rarely employed in character recognition and detection problems. Local feature, however, is similar to directional features, which have been employed in character recognition applications. In addition, local feature can detect corners and thus it is suitable for detecting characters, which are generally comprised of many corners. For evaluating the performance of the local feature, an experimental result was done and its results showed that SURF, i.e., a simple gradient feature, can detect about 70% of characters in scenery images. Then the saliency map was employed as an additional feature to the local feature. This trial is based on the expectation that scenery characters are generally printed to be salient and thus higher salient area will have a higher probability to be a character area. An experimental result showed that this expectation was reasonable and we can have better discrimination accuracy with the saliency map.

DOI： 10.1109/ICDAR.2011.168

Language：Japanese   Publishing type：Research paper (scientific journal)  

Affine invariant character recognition by progressive removing
Recognizing characters in scene images suffering from perspective distortion is a challenge. Although there are some methods to overcome this difficulty, they are time-consuming. In this paper, we propose a set of affine invariant features and a new recognition scheme called "progressive removing" that can help reduce the processing time. Progressive removing gradually removes less feasible categories and skew angles by using multiple classifiers. We observed that progressive removing and the use of the affine invariant features reduced the processing time by about 60% in comparison to a trivial one without decreasing the recognition rate. © 2011 The Institute of Electrical Engineers of Japan.

DOI： 10.1541/ieejias.131.873

Language：English  

Language：Others   Publishing type：Research paper (scientific journal)  

Object tracking with RFID
This paper reports a new method for visual tracking of humans using active RFID technology. Previous studies were based on the assumption that the radio intensity from an RFID tag will be linearly proportional to the distance between the tag and the antenna or will remain unchanged
however, in reality, the intensity fluctuates significantly and changes drastically with a small change in the environment. The proposed method helps to overcome this problem by using only accurate binary information that reveals whether the target person is close to the antenna. Several experimental results have shown that the information from the RFID tag was useful for reliable tracking of humans. © 2011 The Institute of Electrical Engineers of Japan.

DOI： 10.1541/ieejias.131.441

Language：English   Publishing type：Research paper (other academic)  

In character recognition, multiple prototype classifiers, where multiple patterns are prepared as representative patterns of each class, have often been employed to improve recognition accuracy. Our question is how we can improve the recognition accuracy by increasing prototypes massively in the multiple prototype classifier. In this paper, we will answer this question through several experimental analyses, using a simple 1-nearest neighbor (1-NN) classifier and about 550,000 manually labeled handwritten numeral patterns. The analysis results under the leave-one-out evaluation showed not only a simple fact that more prototypes provide fewer recognition errors, but also a more important fact that the error rate decreases approximately to 40% by increasing the prototypes 10 times. The analysis results also showed other phenomena in massive character recognition, such that the NN prototypes become visually closer to the input pattern by increasing the prototypes. © 2011 ACM.

DOI： 10.1145/1982185.1982241

Language：English   Publishing type：Research paper (other academic)  

Visual tracking is formulated as an optimization problem of the position of a target object on video frames. This paper proposes a new tracking method based on dynamic programming (DP). Conventional DP-based tracking methods have utilized DP as an efficient breadth-first search algorithm. Thus, their computational complexity becomes prohibitive if the search breadth becomes large according to the increase of the number of parameters to be optimized. In contrast, the proposed method can avoid this problem by utilizing DP as an analytical solver rather than the conventional breadth-first search algorithm. In addition to experimental evaluations, it will be revealed that the proposed method has a close relation to the well-known KLT tracker.

DOI： 10.1007/978-3-642-19315-6_23

Language：English   Publishing type：Research paper (other academic)  

This paper presents a method of recovering digital ink for an intelligent camera pen, which is characterized by the functions that (1) it works on ordinary paper and (2) if an electronic document is printed on the paper the recovered digital ink is associated with the document. Two technologies called paper fingerprint and document image retrieval are integrated for realizing the above functions. The key of the integration is the introduction of image mosaicing and fast retrieval of previously seen fingerprints based on hashing of SURF local features. From the experimental results of 50 handwritings, we have confirmed that the proposed method is effective to recover and locate the digital ink from the handwriting on a physical paper. © 2010 IEEE.

DOI： 10.1109/ICFHR.2010.50

Language：English   Publishing type：Research paper (other academic)  

In the part-based recognition method proposed in this paper, a handwritten character image is represented by just a set of local parts. Then, each local part of the input pattern is recognized by a nearest-neighbor classifier. Finally, the category of the input pattern is determined by aggregating the local recognition results. This approach is opposed to conventional character recognition approaches which try to benefit from the global structure information as much as possible. Despite a pessimistic expectation, we have reached recognition rates much higher than 90% for a digit recognition task. In this paper we provide a detailed analysis in order to understand the results and find the merits of the local approach. © 2010 IEEE.

DOI： 10.1109/ICFHR.2010.90

Language：English   Publishing type：Research paper (other academic)  

In this paper a more compact and more reliable coding scheme for the data-embedding pen is proposed. The data-embedding pen produces an additional ink-dot sequence along a handwritten pattern during writing. The ink-dot sequence represents, for example, meta-information (such as the writer's name and the date of writing) and thus drastically increases the value of the handwriting on a physical paper. There is no need to get access to any memory on the pen to recover the information, which is especially useful in multi-writer or multi-pen scenarios. In this paper we focus on the compactness of the encoded information. The aim of this paper is to encode as much information as possible in short stroke sequences. In our experiments we show that we can embed more information in shorter strokes than in previous work. In straight lines as short as 5 cm, 32 bits can successfully be embedded. Furthermore, the new encoding scheme also works reliably on more complex patterns. © 2010 IEEE.

DOI： 10.1109/ICFHR.2010.127

Language：English   Publishing type：Research paper (other academic)  

This paper is concerned with automatic construction algorithm for gesture network. Gesture network is a network model of gestures for gesture recognition, especially early recognition and motion prediction. Manual construction of gesture network is inefficient, and thus its automatic construction method is expected; this is because gesture network has to be constructed, whenever target gestures are changed. This paper proposes an automatic construction algorithm for gesture network by logical DP matching. The experiment was conducted for evaluating the performance of the gesture network constructed automatically. The experimental result indicated that the proposed automatic construction algorithm for gesture network can be alternative of manual construction.

DOI： 10.1109/TENCON.2010.5686549

Language：English   Publishing type：Research paper (other academic)  

This paper investigates a part-based recognition method of handwritten digits. In the proposed method, the global structure of digit patterns is discarded by representing each pattern by just a set of local feature vectors. The method is then comprised of two steps. First, each of J local feature vectors of a target pattern is recognized into one of ten categories ("0"-"9") by the nearest neighbor discrimination with a large database of reference vectors. Second, the category of the target pattern is determined by the majority voting on the J local recognition results. Despite a pessimistic expectation, we have reached recognition rates much higher than 90% for the task of digit recognition. © 2010 IEEE.

DOI： 10.1109/ICPR.2010.479

Language：English   Publishing type：Research paper (other academic)  

This paper addresses the problem of how to extract, describe, and evaluate handwriting deformation from the deterministic viewpoint for improving recognition accuracy. The key ideas are threefold. The first is to extract handwriting deformation vector field (DVF) between a pair of input and target images by 2D warping. The second is to hierarchically decompose the DVF by a parametric deformation model of global/local affine transformation, where local affine transformation is iteratively applied to the DVF by decreasing window sizes. The third is to accept only low-order deformation components as natural, within-class handwriting deformation. Experiments using the handwritten numeral database IPTP CDROM1B show that correlation-based matching absorbing components of global affine transformation and local affine transformation up to the 3rd order achieved a higher recognition rate of 92.1% than that of 87.0% obtained by original 2D warping. © 2010 IEEE.

DOI： 10.1109/ICPR.2010.459

Language：Others   Publishing type：Research paper (other academic)  

DOI： 10.1145/1815330.1815336

Language：Others   Publishing type：Research paper (other academic)  

DOI： 10.1145/1815330.1815370

Language：English   Publishing type：Research paper (scientific journal)  

This paper proposes the use of a formal grammar for the verification of mathematical formulae for a practical mathematical OCR system. Like a C compiler detecting syntax errors in a source file, we want to have a verification mechanism to find errors in the output of mathematical OCR. A linear monadic context-free tree grammar (LM-CFTG) is employed as a formal framework to define "well-formed" mathematical formulae. A cubic time parsing algorithm for LM-CFTGs is presented. For the purpose of practical evaluation, a verification system for mathematical OCR is developed, and the effectiveness of the system is demonstrated by using the ground-truthed mathematical document database InftyCDB-1 and a misrecognition database newly constructed for this study. © 2010 Birkhäuser Verlag Basel/Switzerland.

DOI： 10.1007/s11786-010-0023-8

Language：Japanese  

Language：Japanese  

A General Assignment of Supplementary Information
特徴量のみでは本質的に避けることができない誤認識を回避するために,付加情報を用いるパターン認識という枠組みが提案されている.この方式では,パターン認識を行う際に,付加情報と呼ばれるクラスの決定を補助する少量の情報を特徴量と同時に用いて認議性能の改善を目指す.付加情報は自由に設定でき,通常は誤認識率が最小になるように設定する.ここで問題となるのは,誤認識率が最小になる付加情報の設定方法である.常に正しい付加情報が得られるいう理想的な条件においては既に問題が定式化され,付加情報の割当方法が導かれている.しかし,実環境での使用を考えると,付加情報に生じる観測誤差を考慮した割当方法が求められる.そこで本論文では付加情報の観測誤差を考慮に入れて,問題を新たに定式化する.これは付加情報が誤らない場合にも有効な一般的なものである.本論文で導いた割当方法が有効に機能することをマハラノビス距離を用いた実験で例示する.

Language：Japanese  

Recognition of Sequential Patterns by Combining Mutually Constrained Local Classifiers
本論文では,時系列パターンの認識手法として,各サンプル点(各時刻)で認識すなわちクラスラベルの決定を行い,最終的にクラスラベル数の多数決によってクラスを確定する手法を検討する.その一つの特徴として,必要に応じて複数サンプル点間に相互制約を設け,それらをできるだけ同じクラスにラベリングする点が挙げられる.これにより,クラスラベルの割当方を制御でき,自由度の高い識別が可能となる.クラスラベルの割当の組合せは総サンプル点数に対し指数関数的に増加する.そこで,グラフの最小切断アルゴリズムいわゆるグラフカットを用いることで,総サンプル点数に対して多項式時間での計算を実現する.オンライン文字データを対象とした認識実験を行い,本手法の有効性を検証した.

Language：Japanese  

Real-Time Nonlinear FEM-Based Simulator for Deforming Volume Model of Soft Organ by Neural Network
本論文では,ニューラルネットワークを用いて,軟性臓器モデルの変形をシミュレートする新たな手法を提案する.提案手法は,基本的なモデルの変形(以後,変形モードと呼ぶ)の組合せに基づいて,モデルの変形を推定する.つまり,変形モードをあらかじめ非線形有限要素法で求め,臓器に加わった外力と,それに対応する変形モードの関係をニューラルネットワークで学習する.学習したニューラルネットワークは,非線形有限要素解析によりモデルの振舞いを推定することを模倣する.実験結果より,提案手法は,非線形有限要素解析とほぼ同程度の精度を保ちつつ,計算コストを大幅に削減することができた.

Language：Japanese   Publishing type：Research paper (scientific journal)  

Digital pen
ディジタルペンについて概観する.ディジタルペンは,スムーズな文字・図形入力機能,および迅速なポインティング機能を持った,優れたインタフェースである.種類としては,筆記対象が限定されたもの,筆記対象が任意のものに大別される.本稿では,タブレットやアノトペンなどすでに製品化されている技術について述べ,今後の課題を考察する.

DOI： 10.3169/itej.64.293

Language：Japanese  

Trachea and Esophagus Classification by AdaBoost
気道確保法の一つである気道挿管では,通常まず喉頭鏡を使って喉頭展開を行い,声門の位置を目視により確認する.しかし実際の医療現場では,上気道閉塞など様々な要因で,声門の位置を目視により確認しづらい場合がある.この不完全な確認が原因で食道へ誤挿管した場合,気道が確保されず危険なだけでなく,無理な目視のために頸椎や歯牙損傷などの合併症を引き起こす危険性がある.安全・確実な気道挿管の実現に向けて,我々は,スタイレット先端に小型カメラを搭載した自動気管内挿管システムを開発することを自指している.本論文では,その要素機能として,カメラから取得される画像から,挿管チューブが気道あるいは食道に挿管されているかを自動的に識別する方法を提案する.本手法は,気道画像には気道周囲の輪状軟骨が特徴的に観察されることから,まずこの環状模様の記述に適した特徴量を定義し,それに基づいた気道・食道識別器をAdaBoostによって構築する.実験の結果,97.6%の高い識別率で気道・食道の判別が可能であり,提案手法の有効性が確認できた.

Language：English  

Language：English   Publishing type：Research paper (scientific journal)  

Machine recognition of mathematical expressions on printed documents is not trivial even when all the individual characters and symbols in an expression can be recognized correctly. In this paper, an automatic classification method of spatial relationships between the adjacent symbols in a pair is presented. This classification is important to realize an accurate structure analysis module of math OCR. Experimental results on very large databases showed that this classification worked well with an accuracy of 99.525% by using distribution maps which are defined by two geometric features, relative size and relative position, with careful treatment on document-dependent characteristics.

DOI： 10.1587/transinf.E92.D.2235

Language：Japanese  

Visual Tracking Based on Global Optimization : DP Tracking
映像中の物体のトラッキングは,その物体のフレーム間の移動量の最適推定問題として定式化される.本論文では,その大局的最適解を得るために,動的計画法(DP)を用いたトラッキング手法を提案する.従来,幅優先探索の一種として扱われていたDP最適化では,画像のサイズやパラメータの増加により,探索幅が非常に大きくなり計算量が増加するという問題がある.これに対し本論文ではDPの解析的解法をトラッキング問題に適用する.これは,最適化の評価に用いられる局所的な誤差関数を二次関数近似することで,DPによる最適化過程に微分による最適化を導入した手法である.幅優先探索なしに解析的にかつ高速に最適解を得ることができ,トラッキング問題には特に有効といえる.本論文では,本手法の定式化と実験結果を示す.

Language：English   Publishing type：Research paper (other academic)  

Document-Level Positioning of a Pen Tip by Retrieval of Image Fragments

Language：English   Publishing type：Research paper (other academic)  

This paper proposes a syntactic method for detection and correction of misrecognized mathematical formulae for a practical mathematical OCR system. Linear monadic context-free tree grammar (LM-CFTG) is employed as a formal framework to define syntactically acceptable mathematical formulae. For the purpose of practical evaluation, a verification system is developed, and the effectiveness of the method is demonstrated by using the ground-truthed mathematical document database InftyCDB-1 and a misrecognition database newly constructed for this study. A satisfactory number of misrecognitions are detected and delivered to the correction process. © 2009 IEEE.

DOI： 10.1109/ICDAR.2009.150

Language：English   Publishing type：Research paper (other academic)  

A stochastic model of stroke order variation is proposed and applied to the stroke-order free on-line Kanji character recognition. The proposed model is a hidden Markov model (HMM) with a special topology to represent all stroke order variations. A sequence of state transitions from the initial state to the final state of the model represents one stroke order and provides a probability of the stroke order. The distribution of the stroke order probability can be trained automatically by using an EM algorithm from a training set of on-line character patterns. Experimental results on large-scale test patterns showed that the proposed model could represent actual stroke order variations appropriately and improve recognition accuracy by penalizing incorrect stroke orders. © 2009 IEEE.

DOI： 10.1109/ICDAR.2009.146

Language：English   Publishing type：Research paper (other academic)  

In this paper, a statistical decision method for automatic classification of spatial relationships between each adjacent pair is proposed. Each pair is composed of mathematical symbols and/or alphabetical characters. Special treatment of mathematical symbols with variable size is important. This classification is important to recognize an accurate structure analysis module of math OCR. Experimental results on a very large database showed that the proposed method worked well with an accuracy of 99.57% by two important geometric feature relative size and relative position. © 2009 IEEE.

DOI： 10.1109/ICDAR.2009.90

Language：English   Publishing type：Research paper (other academic)  

This paper addresses the basic problem of how to extract, describe, and evaluate handwriting deformation from not the statistical but the deterministic viewpoint. The key ideas are threefold. The first idea is to apply 2D warping to extraction of handwriting deformation vector field (DVF) between a pair of input and target images. The second idea is to hierarchically decompose the DVF by a parametric deformation model of global/local affine transformation. As a result, the DVF is expressed by a series of deformation components each of which is characterized by a window size of local affine transformation. The third idea is interrupting of the series of deformation components to obtain natural, reasonable handwriting deformation. Experiments using the handwritten numeral database IPTP CDROM1B show that 31.1% of the handwriting DVF is expressed by global affine transformation, and the subsequent few local affine transformations successfully discriminate natural handwriting deformation from unnatural one. © 2009 IEEE.

DOI： 10.1109/ICDAR.2009.33

Language：English   Publishing type：Research paper (other academic)  

Detection of characters in scenery images is often a very difficult problem. Although many researchers have tackled this difficult problem and achieved a good performance, it is still difficult to suppress many false alarms andalthough missings. This paper investigates a conspicuous character pattern, which is a special pattern designed for easier detection. In order to have an example of the conspicuous character pattern, we select a character font with a larger distance from a non-character pattern distribution and, simultaneously, with a smaller distance from a character pattern distribution. Experimental results showed that the character font selected by this method is actually more conspicuous (i.e., detected more easily) than other fonts. © 2009 IEEE.

DOI： 10.1109/ICDAR.2009.196

Language：English   Publishing type：Research paper (other academic)  

This paper presents a new method of capturing digital ink for pen-based computing. Current technologies such as tablets, ultrasonic and the Anoto pens rely on special mechanisms for locating the pen tip, which result in limiting the applicability. Our proposal is to ease this problem - a camera pen that allows us to write on ordinary paper for capturing digital ink. A document image retrieval method called LLAH is tuned to locate the pen tip efficiently and accurately on the coordinates of a document only by capturing its tiny fragment. In this papeic we report some results on captured digital ink as well as to evaluate their quality. ©2009 IEEE.

DOI： 10.1109/ICDAR.2009.192

Language：English   Publishing type：Research paper (other academic)  

Selecting and Evaluating Conspicuous Character Patterns

Language：English   Publishing type：Research paper (other academic)  

On a Possibility of Pen-Tip Camera for the Reconstruction of Handwritings

Language：Japanese  

Visual Tracking of an Object with its Motion Information
Tracking of a moving robot in surveillance video is an important task for coexistence of human beings with robots. An essential technology to manage coexistence environment of human beings and moving robots is separation and tracking of moving robots. For this task, the moving robot should be separated from other moving objects, i.e., human beings. We assume that the robot provides its additional motion information to the surveillance system to ease the task. The robot can be tracked from the other objects as a moving region being consistent with the additional motion information. For this purpose, we modify a tracking algorithm based on particle filter in order to incorporate the additional motion information. The results of an experiment on real surveillance video sequences have indicated that the proposed framework can separate and track a moving robot under the existence of several walking persons.

DOI： 10.1541/ieejeiss.129.977

Language：English   Publishing type：Research paper (other academic)  

For user convenience, processing of document images captured by a digital camera has been attracted much attention. However, most existing processing methods require an upright image such like captured by a scanner. Therefore, we have to cancel perspective distortion of a camera-captured image before processing. Although there are rectification methods of the distortion, most of them work under certain assumptions on the layout
the borders of a document are available, lextlines are in parallel, a stereo camera or a video image is required and so on. In this paper, we propose a layout-free rectification method which requires none of the above assumptions. We confirm the effectiveness of the proposed method by experiments. © 2009 SPIE-IS&amp
T.

DOI： 10.1117/12.806122

Language：English   Publishing type：Research paper (scientific journal)  

In mathematical OCR, it is necessary to analyze two-dimensional structures of the component characters and symbols in mathematical expressions printed in scientific documents. In this paper, we analyze the positional relationships between adjacent characters for the purpose of automatic discrimination between baseline characters, subscripts, and superscripts, which is one of the most important and delicate parts of structure analysis. It has been proven through a large-scale experiment that this discrimination can be carried out almost perfectly (~99.89%) by using the relative size and position of adjacent characters. © 2008 Birkhäuser Verlag Basel/Switzerland.

DOI： 10.1007/s11786-008-0051-9

Language：English   Publishing type：Research paper (other academic)  

This paper proposes an early recognition method, i.e., a method for recognizing sequential patterns at their beginning parts. The method is based on a combination of frame classifiers prepared at individual frames. The training patterns misrecognized by the frame classifier at a certain frame are heavily weighted for the complementary training of the frame classifier at the next frame. The method was applied to an online character recognition task for showing its usefulness.

DOI： 10.1109/ICPR.2008.4761137

Language：English   Publishing type：Research paper (other academic)  

A new hidden Markov model (HMM) is proposed for on-line character recognition using two typical features, pen-direction feature and pen-coordinate feature. These two features are quite different in their stationarity; pen-direction feature is stationary within evey line segment of a stroke whereas pen-coordinate feature is not. In the proposed HMM, these contrasting features are used in a separative and selective wary Specifically speaking, pen-direction feature is outputted repeatedly at intra-stale transition whereas pen-coordinate feature is outputted once at inter-state transition. The superiority of the proposed HMM over the conventional HMMs was shown through single-stroke and multi-stroke character recognition experiments.

DOI： 10.1109/ICPR.2008.4761449

Language：English   Publishing type：Research paper (scientific journal)  

This paper introduces a fast image mosaicing technique that does not require costly search on image domain (e.g., pixel-to-pixel correspondence search on the image domain) and the iterative optimization (e.g., gradient-based optimization, iterative optimization, and random optimization) of geometric transformation parameter. The proposed technique is organized in a two-step manner. At both steps, histograms are fully utilized for high computational efficiency. At the first step, a histogram of pixel feature values is utilized to detect pairs of pixels with the same rare feature values as candidates of corresponding pixel pairs. At the second step, a histogram of transformation parameter values is utilized to determine the most reliable transformation parameter value. Experimental results showed that the proposed technique can provide reasonable mosaicing results in most cases with very feasible computations.

DOI： 10.1093/ietisy/e91-d.11.2701

Language：English   Publishing type：Research paper (other academic)  

Information Embedment with Cross Ratio of Areas for Accurate Camera-Based Character Recognition

Language：Japanese  

A Primary Study on a Data-Embedding Pen

Language：English   Publishing type：Research paper (other academic)  

A wide variety of mathematical expressions printed in scientific and technical reports can be recognized bill analyzing the two-dimensional layout structure. In this paper; the position relation between adjacent characters is analyzed for the purpose of automatic discrimination between baseline, subscript, and superscript characters. This analyzing is one of the most important parts of structure analysis. The proposed method is very promising, as the results reached up to (99.76%) over a very large database by using distribution map. This distribution map is defined by two important features, i.e., relative size and relative position.

DOI： 10.1109/DAS.2008.53

Language：English   Publishing type：Research paper (other academic)  

This paper proposes a novel skew estimation method by instances. The instances to be learned (i.e., stored) are rotation invariants and a rotation. variant for each character category. Using the instances, it is possible to estimate a skew angle of each individual character on a document. This fact implies that the proposed method can estimate the skew angle of a document where characters do not form long straight text lines. Thus, the proposed method will be applicable to various documents such as signboard images captured by a camera. Experimental evaluation using synthetic and real images revealed the expected robustness against various character layouts.

DOI： 10.1109/DAS.2008.22

Language：English   Publishing type：Research paper (other academic)  

This paper presents a new method for simulating the deformation of organ models by using a neural network. The proposed method is based on the idea proposed by Chen et al. [2] that deformed model can be estimated from the superposition of basic deformation modes. The neural network finds a relationship between external forces and the models deformed by the forces. The experimental results show that the trained network can achieve a real-time simulation while keeping the acceptable accuracy compared with the nonlinear FEM computation.

DOI： 10.1007/978-3-540-85990-1_89

Language：English   Publishing type：Research paper (other academic)  

There are many problems to realize camera-based char. acter recognition. One of the problems is that characters in scenes arc often distorted by geometric transformations such as affine distortions. Although some methods that remove the affine distortions have been proposed, they cannot remove a rotation transformation of a character Thus a skew angle of a character has to be determined by examining all the possible angles. However this consumes quite a bit of time. In this paper in order to reduce the processing time for an affine invariant recognition, we propose a set of affine invariant features and a new recognition scheme called "progressive pruning." The progressive pruning gradually prunes less feasible categories and skew angles using multiple classifiers. We confirmed the progressive pruning with the affine invariant features reduced the processing time at least less than half without decreasing the recognition rate.

DOI： 10.1109/DAS.2008.88

Language：English  

Language：Japanese  

HMM for On-Line Handwriting Recognition by Selective Use of Pen-Coordinate Feature and Pen-Direction Feature
本論文では,高精度なオンライン文字認識のために,方向特徴並びに座標特徴を適切に使い分け可能な隠れマルコフモデル(HMM)を提案する.両特徴はいずれもオンライン文字認識の基本的な特徴量でありながら,全く異なった性質を示す.すなわち,線分内で方向特徴が定常的なのに対し,座標特徴は常に非定常である.したがって,HMMの枠組みにおいて両特徴を同等に扱うのは問題が多い.実際従来法では,座標特徴を用いずに方向特徴だけが用いられることが多かった.本論文で提案するHMMでは,方向特徴を状態内自己遷移における出力シンボルとして使用し,座標特徴を状態間遷移における出力シンボルとして使用する.このようにすることで,線分方向が一定した定常的な部分においては方向特微が,線分の方向が変化する過渡的な部分においては座標特徴が評価されることになる.このように特徴を使い分けることで,従来法に比べ認識精度を大幅に向上できることを,多画文字(漢字)の筆順フリー認識実験並びにその詳細な考察を通して示す.

Language：English   Publishing type：Research paper (scientific journal)  

Single-character recognition of mathematical symbols poses challenges from its two-dimensional pattern, the variety of similar symbols that must be recognized distinctly, the imbalance and paucity of training data available, and the impossibility of final verification through spell check. We investigate the use of support vector machines to improve the classification of InftyReader, a free system for the OCR of mathematical documents. First, we compare the performance of SVM kernels and feature definitions on pairs of letters that InftyReader usually confuses. Second, we describe a successful approach to multi-class classification with SVM, utilizing the ranking of alternatives within InftyReader's confusion clusters. The inclusion of our technique in InftyReader reduces its misrecognition rate by 41%. (c) 2008 Elsevier B.V. All rights reserved.

DOI： 10.1016/j.patrec.2008.02.005

Language：English   Publishing type：Research paper (other academic)  

This paper proposes the use of a formal grammar for the verification of mathematical formulae for a practical mathematical OCR. system. Like a C compiler detecting syntax errors in a source file, we, rant to have a verification mechanism to find errors in the output of mathematical OCR. Linear monadic context-free tree grammar (LM-CFTG) was employed as a formal framework to define "well-formed" mathematical formulae. For the purpose of practical evaluation, a verification system for mathematical OCR was developed, and the effectiveness of the system was demonstrated by rising the ground-truthed mathematical document database INIFTY CDB-1.

DOI： 10.1007/978-3-540-85110-3_35

Language：Japanese  

An HMM Representing Stroke Order Variations and Its Application to Online Character Recognition
本論文では,筆順フリーなオンライン文字認識の高精度化を目指し,(i)筆順変動の統計的モデルの構築,及び(ii)その認識における利用,の2点について検討する.一般に筆順フリー化には不自然な画対応の許容による誤認識の問題があるが,提案する筆順変動モデルを用いることでそれらを抑制できる.この筆順変動モデルは,筆順フリー認識のためのグラフモデル(キューブグラフ)の確率的拡張として定式化され,結果的に文字形状に関するゆう度と筆順のゆう度を同時に扱うことが可能な隠れマルコフモデル(HMM)の一種となる.公開されているオンライン文字データベース"HANDS-kuchibue.d-97-06-10"を用いた認識実験により,筆順変動モデル導入の有効性及び妥当性を明らかにした.

Language：English   Publishing type：Research paper (other academic)  

This paper presents a parallel algorithm of the Level Set Method named the Parallel Fast Level Set Method, and its application for real-time 3D reconstruction of human shape and motion. The Fast Level Set Method is an efficient implementation algorithm of the Level Set Method and has been applied to several applications such as object tracking in video images and 3D shape reconstruction using multiple stereo cameras. In this paper, we implement the Fast Level Set Method on a PC cluster and develop a real-time motion capture system for arbitrary viewpoint image synthesis. To obtain high performance on a PC cluster, efficient load-balancing and resource allocation algorithms are crucial problems. We develop a novel optimization technique of load distribution based on the estimation of moving direction of object boundaries. In this technique, the boundary motion is estimated in the framework of the Fast Level Set Method, and the optimum load distribution is predicted and performed according to the estimated boundary motion and the current load balance. Experiments of human shape reconstruction and arbitrary viewpoint image synthesis using the proposed system are successfully carried out.

DOI： 10.1109/ROBOT.2008.4543332

Language：Japanese  

2D/3D Registration by Back Projection and Geometrical Constraints
レンジセンサにより取得した幾何モデルにカラーセンサで撮影したテクスチャ画像を貼り付けて表示するテクスチャマッピングを容易に実現するには,テクスチャ画像と幾何モデルのみからカラー・レンジセンサ間の相対位置関係を知ることが望ましい.本論文では,幾何拘束に基づく大域的手法とエッジの対応付けに基づく局所的手法の組合せにより,センサ間の相対位置・姿勢を初期値の変動にロバストにかつ高精度に推定し,テクスチャ画像と幾何モデルの位置合せを実現する手法を提案する.本手法はまず,テクスチャ画像から稜線と平面領域を抽出する.次に,この稜線と平面領域を幾何モデルに逆投影し,対象における幾何拘束条件を推定しつつ,この拘束条件のもとでセンサ間の相対位置・姿勢の初期推定値を求める.最後に,テクスチャ画像と幾何モデルの各エッジ間の対応付けに基づき,センサ間の相対位置・姿勢を決定する.実験では,エッジ間の対応付けに基づく従来手法と比較して,位置合せの成功率が41%から75%に向上した.

Language：English   Publishing type：Research paper (scientific journal)  

Text recognition captured in multiple frames by a hand-held video camera is a challenging task because it is possible to capture and recognize a longer line of text while improving the quality of the text image by utilizing the redundancy of the overlapping areas between the frames. For this task, the video frames should be registered, i.e., mosaiced, after compensating for their distortions due to camera shakes. In this paper, a mosaicing-by-recognition technique is proposed where the problems of video mosaicing and text recognition are formulated as a unified optimization problem and solved by a dynamic program ming-based optimization algorithm simultaneously and collaboratively. Experimental results indicate that, even if the frames undergo various distortions such as rotation, scaling, translation, and nonlinear speed fluctuation of camera movement, the proposed technique provides fine mosaic image by accurate distortion estimation (around 90% of perfect estimation) and character recognition accuracy (over 95%). (c) 2007 Elsevier Ltd. All rights reserved.

DOI： 10.1016/j.patcog.2007.08.005

Language：English  

This chapter reviews various elastic matching techniques for Handwritten character recognition. Elastic matching is formulated as an optimization problem of planar matching, or pixel-to-pixel correspondence, between two character images under a certain matching model, such as affine and nonlinear. Use of elastic matching instead of rigid matching improves the robustness of recognition systems against geometric deformations in Handwritten character images. In addition, the optimized matching represents the deformation of Handwritten characters and thus is useful for statistical analysis of the deformation. This chapter argues the general property of elastic matching techniques and their classification by matching models and optimization strategies. It also argues various topics and future work related to elastic matching for emphasizing theoretical and practical importance of elastic matching. © 2008, IGI Global.

DOI： 10.4018/978-1-59904-807-9.ch002

Language：English  

Language：Japanese  

Document Skew Estimation by Instance-Based Learning
各文字の回転変形に対する変量と不変量を事例として学習しておき,それらを利用することで文書画像の回転角を推定する方法を提案する.本手法は,文字単位で回転角を効率的に推定するため,文字列が直線的かつ平行にレイアウトされているという仮定が不要であり,したがって様々なレイアウトの文書に利用可能である.

Language：Japanese  

Supplementary Information Embedment with Area Ratio for Camera-Based Character Recognition
ディジタルカメラを入力デバイスとして実環境中の文字を高精度に認識するために,文字画像と同時に認識補助のための付加情報を提示する方法が検討されている.付加情報は,人間にとって自然な形で提示されること,及び,幾何学的変形に対してロバストに抽出できることが要求される.本論文では,これらの要求を満たす手法として,面積比を利用した付加情報提示手法を提案する.すなわち,文字パターンを2色で印字することを前提とし,それぞれの色の領域の面積比を特定の値とするようにデザインする.具体的には,文字に影を付加したり輪郭線を別の色とする.これらは文字パターンのデザインとして既に行われており,提案手法はその線幅や面積を変えるにすぎない.したがって,提案手法は様々な用途に広く応用することが可能である.面積比はアフィン変換に不変であり,アフィン変換を受けた環境においても誤りなく抽出されることが期待される.実際に付加情報を埋め込んだ文字パターンを作成し,ディジタルカメラで撮影された画像中の文字パターンから付加情報を抽出する実験を行い,提案手法の有効性を確認する.また,付加情報を用いて文字を認識する実験を行い,認識精度が向上することを確認する.

Language：English   Publishing type：Research paper (other academic)  

A logical dynamic programming (DP) matching algorithm is proposed for extracting similar subpatterns from two sequential patterns. In the proposed algorithm, local similarity between two patterns is measured by a logical function, called support. The DP matching with the support can extract all similar subpatterns simultaneously while compensating nonlinear fluctuation. The performance of the proposed algorithm was evaluated qualitatively and quantitatively via an experiment of extracting motion primitives, i.e., common subpatterns in gesture patterns of different classes.

DOI： 10.1007/978-3-540-76386-4_59

Language：Others  

Recognition of Engineering Drawing Entities: Review of Approaches.

DOI： 10.1142/S0219467807002878