Spatial-temporal reasoning in symbolic neural network for semantic interpretation of videos - Tenório, Milena Rodrigues (2022)
The Semantic Video Interpretation field of study looks for ways to model the information in videos. Existing methods can be divided into generic and specialized methods; the former can efficiently categorize information while the latter does not perform well for generic data. One way for researchers to deal with this impasse, in other fields of study, is to use the knowledge and basic restrictions on it. For this, we use neural-symbolic reasoning. Our hypothesis is to use a neural-symbolic network to extract information from images in a video to model this information, and finally perform reasoning to extract the semantic description. For this purpose, three main steps were chosen: (1) identification of the objects in the video images, (2) identification of the spatial relations in frame groups, and (3) analysis of the temporal relations found.