Lexicon annotation with LLM: a proof of concept with ChatGPT

• Autor(a) principal: Marcondes, Francisco Supino
• Data de Publicação: 2025
• Outros Autores: Gala, Adelino de C.O.S., Rodrigues, Manuel, Almeida, J. J., Novais, Paulo
• Idioma: eng
• Título da fonte: Repositórios Científicos de Acesso Aberto de Portugal (RCAAP)
• Texto Completo: https://hdl.handle.net/1822/95171
• topic: ChatGPT; Lexicon annotation; LLMs; NLP; Ciências Naturais::Ciências da Computação e da Informação

Lexicon annotation is a critical yet time-consuming task that can hold back the progress of language-intensive projects. This paper explores the potential of Large Language Models (LLMs) to automate lexicon annotation, traditionally performed by humans. We present a proof of concept by evaluating ChatGPT's performance on annotating VADER's sentiment lexicon. Our findings demonstrate that ChatGPT achieves fair performance in this task, suggesting that LLMs can operate as a valuable tool for initial annotations, with subsequent refinements by domain specialists. This approach could significantly accelerate lexicon development and maintenance while balancing efficiency and accuracy. Our study provides insights into the capabilities and limitations of LLMs in lexicon annotation, leading the way for further research in automating linguistic resources development.