Digital signal processing and machine learning for coherent optical systems: a Python approach

• Autor(a) principal: Zadorozhnyy, Denys
• Data de Publicação: 2023
• Tipo de documento: Dissertação
• Idioma: eng
• Título da fonte: Repositórios Científicos de Acesso Aberto de Portugal (RCAAP)
• Texto Completo: http://hdl.handle.net/10773/41696
• topic: Autoencoder; Coherent Optical System; Digital Signal Processing; Geometric Shaping

Since their inception, optical communications have gone through a lot of changes, with constant innovation and evolution. The introduction of Digital Signal Processing (DSP) enabled the immense and rapid progression of coherent optics. This, in turn, allowed the use of advanced modulation formats, such as Quadrature Amplitude Modulation (QAM), which made this kind of system much more efficient in comparison to their Intensity Modulation with Direct Detection (IM-DD) counterparts. DSP allows for the mitigation of imperfections in the Local Oscillator (LO) at the receiver and in the transmission channel in the digital domain, which eliminates the need for specific physical components to do so. These fast and highly reliable optical communication systems have truly revolutionized the way information is transmitted. However, the world keeps evolving. And with it, the demands for even more efficient and secure data transmission continue to grow. With that in mind, Machine Learning (ML) appeared as a means to further push optical communications to new levels. It has already shown to be very effective and groundbreaking in many other fields. In that sense, this thesis explores the intersection of coherent optical fiber systems with ML. It was already shown that an Autoencoder (AE) could learn full end-to-end mapping and demapping of a transmission system. In this work, the main objective is to develop a complete simulation tool using the Python programming language, which integrates classical DSP approaches with the innovative ideas of ML. First, going step by step, a simple simulator is made with the base components of a communications system, such as a mapper, demapper, and Additive White Gaussian Noise (AWGN) channel. After its validation more complex DSP functionalities are added to the receiver side: Adaptive Equalization, Carrier-Frequency Recovery (CFR), and Carrier-Phase Recovery (CPR). Different types of algorithms are explored and compared. Bandwidth limitations, mismatches in frequency, and laser phase noise can all deteriorate the transmitted signal. Those effects were applied to the system under study, and various tests were done in order to validate the developed algorithms. Using the developed simulation tool, the compensation of each one of these effects was demonstrated. Finally, the AE is introduced, and the chosen architecture is explained. In this work, it was opted to use a Neural Network (NN) at the encoder and a minimum distance demapper at the decoder. This way the computational complexity is reduced, and the geometrically shaped constellations can be easily exported and integrated into the developed Python simulator. Those new constellations had a visible gain over the regular square ones. At last, residual laser phase noise is introduced in the AE, and it was demonstrated that the optimization of the constellation geometry can effectively contribute to enhance the performance of phase noise impaired coherent optical systems.