| 1. |
Cheng Tang, Junkai Ji, Yuki Todo, Atsushi Shimada, Weiping Ding, Akimasa Hirata., Dendritic Neural Network: A Novel Extension of Dendritic Neuron Model, IEEE Transactions on Emerging Topics in Computational Intelligence, 10.1109/TETCI.2024.3367819, 2024.03, The conventional dendritic neuron model (DNM) is a single-neuron model inspired by biological dendritic neurons that has been applied successfully in various fields. However, an increasing number of input features results in inefficient learning and gradient vanishing problems in the DNM. Thus, the DNM struggles to handle more complex tasks, including multiclass classification and multivariate time-series forecasting problems. In this study, we extended the conventional DNM to overcome these limitations. In the proposed dendritic neural network (DNN), the flexibility of both synapses and dendritic branches is considered and formulated, which can improve the model's nonlinear capabilities on high-dimensional problems. Then, multiple output layers are stacked to accommodate the various loss functions of complex tasks, and a dropout mechanism is implemented to realize a better balance between the underfitting and overfitting problems, which enhances the network's generalizability. The performance and computational efficiency of the proposed DNN compared to state-of-the-art machine learning algorithms were verified on 10 multiclass classification and 2 high-dimensional binary classification datasets. The experimental results demonstrate that the proposed DNN is a promising and practical neural network architecture.. |
| 2. |
Zhenyu Song, Cheng Tang, Shuangbao Song, Yajiao Tang, Jinhai Li, Junkai Ji, A complex network-based firefly algorithm for numerical optimization and time series forecasting, Applied Soft Computing, 10.1016/j.asoc.2023.110158, 137, 2023.04, The firefly algorithm (FA) has gained widespread attention and has been widely applied because of its simple structure, few control parameters and easy implementation. As the traditional FA lacks a mutation mechanism, it tends to fall into local optima, leading to premature convergence, thus affecting the optimization accuracy. To address these limitations, from the perspective of population diversity, a complex network-based FA (CnFA) with scale-free properties is proposed in this paper. The scale-free properties of complex networks effectively ensure the diversity of populations to guide the populations in their search, thus avoiding random interactions of information among populations that could lead to superindividuals controlling the entire population. The property of the power-law distribution of nodes in complex networks is exploited to effectively avoid the premature convergence of the FA and falling into local optima. To verify the search performance of CnFA, we compared the FA and its variants, as well as multiple competitive approaches, on 30 different-dimension benchmark function optimization tasks and two time series prediction tasks. The experimental results and statistical analysis show that CnFA achieves satisfactory performance due to the better balance between exploitation and exploration in the search process. Additionally, we extended the proposed method to two other population-based algorithms, and the experimental results verify that the complex network-based mechanism can enhance the performance of not only the FA but also other population-based evolutionary algorithms.. |
| 3. |
Yajiao Tang, Zhenyu Song, Yulin Zhu, Huaiyu Yuan, Maozhang Hou, Junkai Ji, Cheng Tang, Jianqiang Li, A survey on machine learning models for financial time series forecasting, Neurocomputing, 10.1016/j.neucom.2022.09.003, 512, 363-380, 2022.11, Financial time series (FTS) are nonlinear, dynamic and chaotic. The search for models to facilitate FTS forecasting has been highly pursued for decades. Despite major related challenges, there has been much interest in this topic, and many efforts to forecast financial market pricing and the average movement of various financial assets have been implemented. Researchers have applied different models based on computer science and economics to gain efficient information and earn money through financial market investment decisions. Machine learning (ML) methods are popular and successful algorithms applied in the FTS domain. This paper provides a timely review of ML's adoption in FTS forecasting. The progress of FTS forecasting models using ML methods is systematically summarized by searching articles published from 2011 to 2021. Focusing on the analysis of ML methods applied to the theoretical basis and empirical application of FTS data forecasting, this paper provides a relevant reference for FTS forecasting and interdisciplinary fusion research against the background of computational intelligence and big data. The literature survey reveals that the most commonly used models for prediction involve long short-term memory (LSTM) and hybrid methods. The main contribution of this paper is not only building a systematic program to compare the merits and demerits of specific FTS forecasting models but also detecting the importance and differences of each model to help researchers and practitioners make good choices. In addition, the limitations to be addressed and future research directions of ML models’ adoption in FTS forecasting are identified.. |
| 4. |
Yajiao Tang, Zhenyu Song, Yulin Zhu, Maozhang Hou, Cheng Tang, Junkai Ji, Adopting a dendritic neural model for predicting stock price index movement, Expert Systems with Applications, 10.1016/j.eswa.2022.117637, 205, 2022.11, Financial time series forecasting has been an attractive application of machine learning techniques because an advanced forecasting method can help to accurately predict price changes in markets and make good trading profits. In this study, an emerging machine learning approach, named the dendritic neuron model (DNM), is innovatively applied to forecast financial time series. To pursue better prediction performance, a novel scale-free differential evolution (SFDE) is defined as the training algorithm of the DNM, which can well control the balance between exploration and exploitation. In addition, the maximum Lyapunov exponent is used to detect the chaotic property of financial time series; then, the series is reconstructed into a phase space with high dimension before the prediction, where the time delay of the phase space is calculated by a mutual information method and the embedding dimension is separately determined by a false nearest neighbors approach. In our experiments, eight benchmark stock price indices selected from developed markets and emerging markets are used to validate the effectiveness and efficiency of the proposed forecasting model. Overall, the experimental results illustrate that the DNM trained by the SFDE algorithm yields better forecasting performances than other prevailing models and that it can be considered a reliable and satisfactory forecasting tool for predicting price changes in financial markets for practical applications.. |
| 5. |
Bin Li, Yuki Todo, Zheng Tang, Cheng Tang, The mechanism of orientation detection based on color-orientation jointly selective cells, Knowledge-Based Systems, 10.1016/j.knosys.2022.109715, 254, 2022.10, This paper discusses the visual mechanism of global orientation detection and the realization of a mechanism-based artificial visual system for two-dimensional orientation detection tasks. For interpretation and practicability, we introduce the visual mechanism into the design of a detection system. We first propose an orientation detection mechanism according to the color-orientation jointly selectivity cortical neuron character. We assume that part of the orientation detection tasks is completed by the color-orientation jointly selective cells that are only responsible for orientation detection locally. Each cell can only be activated by stimuli with a specific orientation angle and the preferred color. We realize these cells by the McCulloch–Pitts neuron model and extend them to a two-dimensional version. In each local receptive field, there are four separate color-orientation jointly selective cells responsible for orientation detection, and their optimal responsive color corresponds to the central location's color. Every local region connects such a set of cells. Subsequently, by these sets of these cells, we can collect all local information and obtain the global orientation according to the local activations. The type of local orientation angle recognized the most corresponds to the global orientation. Finally, a mechanism-based artificial visual system (AVS) is implemented. Several simulations and comparative experiments are provided to verify the effectiveness and generalization of the proposed orientation detection scheme and the superiority of the AVS to popular classification networks in orientation detection tasks. In addition, the feature extraction ability of AVS is shown to accelerate the learning and noise immunity of neural networks.. |
| 6. |
Junkai Ji, Cheng Tang, Jiajun Zhao, Zheng Tang, Yuki Todo., A survey on dendritic neuron model: Mechanisms, algorithms and practical applications, Neurocomputing, 10.1016/j.neucom.2021.08.153, 2022.06, Research on dendrites has been conducted for decades, providing valuable information for the development of dendritic computation. Creating an ideal neuron model is crucial for computer science and may also provide robust guidance for understanding our brain’s underlying mechanisms and principles. This paper aims to review the related studies regarding a newly emerging, non-spiking and biologically inspired model, the dendritic neuron model (DNM). By mimicking the biological phenomena of neurons in vivo, the DNM incorporates a neural pruning scheme to eliminate superfluous synapses and dendrites, simplifying its architecture and forming unique neuron morphology for a specific task. Furthermore, the simplified structure can be transformed into logic circuits consisting of the comparators and logic AND, OR and NOT gates, without sacrificing model accuracy. The rapidity of binary operations in hardware implementation gives the DNM a distinct advantage to handle high-speed data streams. The advent of the big data era has led to an exponential explosion in the amount and variety of available information. The appealing properties of the DNM lead us to believe that it is worthy of more attention and that it might be a promising data mining technique. This article presents an in-depth analysis of the pruning and transformation mechanisms and a comprehensive review of the learning algorithms and real-world applications of the DNM. It also presents an empirical comparison of the optimization performance of different algorithms. Finally, we outline some critical issues and future works of the DNM. All the source code of DNM is available at http://www.dnm.net.cn/.. |
| 7. |
Cheng Tang, Yuki Todo, Junkai Ji, Zheng Tang, A novel motion direction detection mechanism based on dendritic computation of direction-selective ganglion cells, Knowledge-Based Systems, 10.1016/j.knosys.2022.108205, 241, 2022.04, The visual system plays a vital role when the brain receives and processes information. Approximately ninety percent of the information received by the brain comes from the visual system, and motion detection is a crucial part of processing visual information. To further understand the generation of direction selectivity, we propose a novel apparent motion detection mechanism using direction-selective ganglion cells (DSGCs). Considering the simplicity of neural computation, each neuron is responsible for detection in a specific direction. For example, eight neurons are employed to detect movements in eight directions, and local information is collected by scanning. The global motion direction is obtained according to the degree of activation of the neurons. We report that this method not only has striking biological similarities with hypercomplex retinal cells, but can also make accurate discriminations. The pioneering mechanism may lead to a new technique for understanding more complex principles of the visual nervous system.. |
| 8. |
Cheng Tang, Shuangbao Song, Junkai Ji, Yajiao Tang, Zheng Tang, Yuki Todo, A cuckoo search algorithm with scale-free population topology, Expert Systems with Applications, 10.1016/j.eswa.2021.116049, 188, 2022.02, The scale-free network is an important type of complex network. The node degrees in a scale-free network follow the power-law distribution. In the skeleton of a scale-free network, there exists a few nodes which own huge neighborhood size and play an important role in information transmission of the entire network, while most of the network nodes have few connections whose influences of information exchange are limited to a relatively low level. In this paper, we introduce a scale-free population topology into the cuckoo search (CS) algorithm to propose a novel variant, which is termed the scale-free cuckoo search (SFCS) algorithm. Unlike other CS algorithms where the individuals exchange information randomly, two properties of the scale-free network can improve the SFCS algorithm in two aspects: the possibility that the information of competent individuals quickly floods the whole population is reduced significantly, which guarantees population diversity; and the corrupt individuals can learn from competent individuals with greater probability, which is beneficial for convergence. Thus, SFCS can obtain a better trade-off between exploitation and exploration during the search process. To evaluate the effectiveness of the proposed SFCS, 58 benchmark functions with different dimensions (10-D, 30-D, and 50-D), and 21 real-world optimization problems are employed in our experiment. We compare SFCS with the basic CS algorithm, two CS variants, and five state-of-the-art optimization algorithms, and the experimental results and statistical analysis verify the superiority of SFCS in terms of solution quality and convergence speed. Furthermore, we compare SFCS with a scale-free fully informed particle swarm optimization algorithm (SFIPSO) and the results prove our scale-free idea is effective despite its simplicity. We also introduce the scale-free population topology into the differential evolution (DE) and the firefly algorithm (FA) and the experimental results show that the scale-free population topology enhance the search ability of the DE and FA. These lead us to believe that the scale-free population topology may be a new technique for improving the performance of the population-based algorithms.. |
| 9. |
Cheng Tang, Yuki Todo, Junkai Ji, Qiuzhen Lin, Zheng Tang, Artificial immune system training algorithm for a dendritic neuron model, Knowledge-Based Systems, 10.1016/j.knosys.2021.107509, 233, 2021.12, Dendritic neuron model (DNM), which is a single neuron model with a plastic structure, has been applied to resolve various complicated problems. However, its main learning algorithm, namely the back-propagation (BP) algorithm, suffers from several shortages, such as slow convergence rate, being easy to fall into local minimum and over-fitting problems. That largely limits the performances of the DNM. To address this issue, another bio-inspired learning paradigm, namely the artificial immune system (AIS) is employed to train the weights and thresholds of the DNM, which is termed AISDNM. These two methods have advantages on different issues. Due to the powerful global search capability of the AIS, it is considered to be efficient in improving the performance of the DNM. To evaluate the performance of AISDNM, eight classification datasets and eight prediction problems are adopted in our experiments. The experimental results and statistical analysis confirm that the AISDNM can exhibit superior performance in terms of accuracy and convergence speed when compared with the multilayer perceptron (MLP), decision tree (DT), the support vector machine with the linear kernel (SVM-l), the support vector machine with the radial basis function kernel (SVM-r), the support vector machine with the polynomial kernel (SVM-p) and the conventional DNM. It can be concluded that the reasonable combination of two different bio-inspired learning paradigms is efficient. Furthermore, for the classification problems, empirical evidence also validates the AISDNM can delete superfluous synapses and dendrites to simplify its neural structure, then transform the simplified structure into a logic circuit classifier (LCC) which is suitable for hardware implementation. The process does not sacrifice accuracy but significantly improves the classification speed. Based on these results, both the AISDNM and the LCC can be regarded as effective machine learning techniques to solve practical problems.. |
| 10. |
Minhui Dong, Cheng Tang, Junkai Ji, Qiuzhen Lin, Ka Chun Wong, Transmission trend of the COVID-19 pandemic predicted by dendritic neural regression, Applied Soft Computing, 10.1016/j.asoc.2021.107683, 111, 2021.11, In 2020, a novel coronavirus disease became a global problem. The disease was called COVID-19, as the first patient was diagnosed in December 2019. The disease spread around the world quickly due to its powerful viral ability. To date, the spread of COVID-19 has been relatively mild in China due to timely control measures. However, in other countries, the pandemic remains severe, and COVID-19 protection and control policies are urgently needed, which has motivated this research. Since the outbreak of the pandemic, many researchers have hoped to identify the mechanism of COVID-19 transmission and predict its spread by using machine learning (ML) methods to supply meaningful reference information to decision-makers in various countries. Since the historical data of COVID-19 is time series data, most researchers have adopted recurrent neural networks (RNNs), which can capture time information, for this problem. However, even with a state-of-the-art RNN, it is still difficult to perfectly capture the temporal information and nonlinear characteristics from the historical data of COVID-19. Therefore, in this study, we develop a novel dendritic neural regression (DNR) method to improve prediction performance. In the DNR, the multiplication operator is used to capture the nonlinear relationships between input feature signals in the dendrite layer. Considering the complex and large landscape of DNR's weight space, a new scale-free state-of-matter search (SFSMS) algorithm is proposed to optimize the DNR, which combines the state-of-matter search algorithm with a scale-free local search. The SFSMS achieves a better global search ability and thus can effectively reduce the possibility of falling into local minima. In addition, according to Takens's theorem, phase space reconstruction techniques are used to discover the information hidden in the high-dimensional space of COVID-19 data, which further improves the precision of prediction. The experimental results suggest that the proposed method is more competitive in solving this problem than other prevailing methods.. |
| 11. |
Cheng Tang, Junkai Ji, Yajiao Tang, Shangce Gao, Zheng Tang, Yuki Todo, A novel machine learning technique for computer-aided diagnosis, Engineering Applications of Artificial Intelligence, 10.1016/j.engappai.2020.103627, 92, 2020.06, The primary motivation of this paper is twofold: first, to employ a heuristic optimization algorithm to optimize the dendritic neuron model (DNM) and second, to design a tidy visual classifier for computer-aided diagnosis that can be easily implemented on a hardware system. Considering that the backpropagation (BP) algorithm is sensitive to the initial conditions and can easily fall into local minima, we propose an evolutionary dendritic neuron model (EDNM), which is optimized by the gbest-guided artificial bee colony (GABC) algorithm. The experiments are performed on the Liver Disorders Data Set, the Wisconsin Breast Cancer Data Set, the Haberman's Survival Data Set, the Diabetic Retinopathy Debrecen Data Set and Hepatitis Data Set, and the effectiveness of our model was rigorously validated in terms of the classification accuracy, the sensitivity, the specificity, the F_measure, Cohen's Kappa, the area under the receiver operating characteristic curve (AUC), convergence speed and the statistical analysis of the Wilcoxon signed-rank test. Moreover, after training, the EDNM can simplify its neural structure by removing redundant synapses and superfluous dendrites by the neuronal pruning mechanism. Finally, the simplified structural morphology of the EDNM can be replaced by a logic circuit (LC) without sacrificing accuracy. It is worth emphasizing that once implemented by an LC, the model has a significant advantage over other classifiers in terms of speed when handling big data. Consequently, our proposed model can serve as an efficient medical classifier with excellent performance.. |
| 12. |
Junkai Ji, Shuangbao Song, Cheng Tang, Shangce Gao, Zheng Tang, Yuki Todo, An artificial bee colony algorithm search guided by scale-free networks, Information Sciences, 10.1016/j.ins.2018.09.034, 473, 142-165, 2019.01, Many optimization algorithms have adopted scale-free networks to improve the search ability. However, most methods have merely changed their population topologies into those of scale-free networks; their experimental results cannot verify that these algorithms have superior performance. In this paper, we propose a scale-free artificial bee colony algorithm (SFABC) in which the search is guided by a scale-free network. The mechanism enables the SFABC search to follow two rules. First, the bad food sources can learn more information from the good sources of their neighbors. Second, the information exchange among good food sources is relatively rare. To verify the effectiveness of SFABC, the algorithm is compared with the original artificial bee colony algorithm (ABC), several advanced ABC variants, and other metaheuristic algorithms on a wide range of benchmark functions. Experimental results and statistical analyses indicate that SFABC obtains a better balance between exploration and exploitation during the optimization process and that, in most cases, it can provide a competitive performance of the benchmark functions. We also verify that scale-free networks can not only improve the optimization performance of ABC but also enhance the search ability of other metaheuristic algorithms, such as differential evolution (DE) and the flower pollination algorithm (FPA).. |
