Delayed Transfer Entropy applied to Big Data

• Ano de defesa: 2018
• Autor(a) principal: Dourado, Jonas Rossi
• Tipo de documento: Dissertação
• Tipo de acesso: Acesso aberto
• Idioma: eng
• Instituição de defesa: Biblioteca Digitais de Teses e Dissertações da USP
• Palavras-chave em Português: Análise de Big Data; Big Data analysis; Causalidade; Causality; Cluster heterogêneo de computadores; Delayed Transfer Entropy; Estratégias de paralelismo; Heterogeneous computer cluster; Parallelism strategies; Surrogate
• Link de acesso: http://www.teses.usp.br/teses/disponiveis/18/18153/tde-19022019-134228/

Recent popularization of technologies such as Smartphones, Wearables, Internet of Things, Social Networks and Video streaming increased data creation. Dealing with extensive data sets led the creation of term big data, often defined as when data volume, acquisition rate or representation demands nontraditional approaches to data analysis or requires horizontal scaling for data processing. Analysis is the most important Big Data phase, where it has the objective of extracting meaningful and often hidden information. One example of Big Data hidden information is causality, which can be inferred with Delayed Transfer Entropy (DTE). Despite DTE wide applicability, it has a high demanding processing power which is aggravated with large datasets as those found in big data. This research optimized DTE performance and modified existing code to enable DTE execution on a computer cluster. With big data trend in sight, this results may enable bigger datasets analysis or better statistical evidence.