A Modified Echo State Network Model Using Non-Random Topology

• Main Author: Arroyo, Diana Carolina Roca
• Publication Date: 2023
• Format: Doctoral thesis
• Language: eng
• Source: Biblioteca Digital de Teses e Dissertações da USP
• Download full: https://www.teses.usp.br/teses/disponiveis/55/55134/tde-12092023-210015/
• topic: Clustering; Clusterização; Computação de reservatório; Echo state network; Recurrent neural network; Redes de estado de eco; Redes neurais recorrentes; Reservoir computing

Reservoir Computing is a type of recurrent neural networks suitable for temporal/sequential data processing. One of the representative Reservoir Computing models is the Echo State Network (ESN), which maps the input-output patterns through a high dimensional nonlinear projection, called reservoir. In the classic model of ESN, the reservoir is compound of a large number of neurons with random connections, forming a Erdös-Rényi random complex network. In this way, the reservoir maps the input data into a higher dimensional space to overcome the linear inseparable problem by applying a simple linear regression algorithm in the training phase. Inspired by the modular structure of a human brain, we propose new ESNs using non-random typology as reservoir by complex network models and clustering models. The connectivity topology based on complex networks in the reservoir are: random networks, scale-free networks, and small-world networks. To generate the clustered reservoirs, we propose to use the classic data clustering algorithms: K-means, Partitioning Around Medoids, and Ward algorithm to simulate community structures. We also generate the clustered scale-free and small-world networks as reservoirs. The main hypothesis of this approach is that the non-random network structures, especially, the clustered networks as reservoirs can better capture the information of different classes of the training data in such a way that each of a group of network communities may be served to encode a certain class of data. Numerical experiments in signal prediction applications show that the proposed models have an improvement in its performance and lower computational cost compared to the classical ESNs. The utility of our ESN-based methods has been shown by considering the tissue classification on hematoxylin and eosin (H&E) stained histopathological whole slide images (WSI). The proposed models to classify tissue components on WSI provide results that overcome classical and the state-of-the-art techniques.