Ai-driven software development: how has the rise of Chatgpt impacted Github users? - Impact on gode quality

• Autor(a) principal: Menezes, Guilherme Nascimento Ortega de Paiva
• Data de Publicação: 2023
• Tipo de documento: Dissertação
• Idioma: eng
• Título da fonte: Repositórios Científicos de Acesso Aberto de Portugal (RCAAP)
• Texto Completo: http://hdl.handle.net/10362/176254
• topic: Software development; Chatgpt; Github; Code quality; Differences in differences; Llm; Machine learning; Data analysis; Text classification; Data curation; Domínio/Área Científica::Ciências Sociais::Economia e Gestão

This thesis analyses AI-driven Software Development practices, investigating how the ban of ChatGPT in Italy affected GitHub activity using a Difference in Difference analysis. It covers the process of retrieving and processing data from GitHub Archive to form a four-week dataset spanning the period from 17/03/2023 to 14/04/2023. A scoring and selection approach to identify users from Italy (Treatment Group) and Germany (Control Group) resulted in a dataset comprising 244,401 commits from 10,520 individual GitHub users. Results suggest that users affiliated with organizations show a 12.12% increase in GitHub events, implying a decrease in coding efficiency after the ban of ChatGPT. In contrast, individual users' activities remain largely unaffected by the ban. Coding errors rose by 8.91% on business days, further indicating a reduction in code quality, while results for weekends and public holidays were insignificant. Lastly, organization-related users active on business days exhibited a 13.39% increase in GitHub events post-ban, suggesting a reduction in coding efficiency, and a 20.6% increase in coding errors, pointing to a decline in code quality. No empirical evidence is found for the bans’ effects on collaboration practices. These findings suggest that ChatGPT is well integrated into the daily software development workflow and actively used to assist in writing and debugging code, especially in professional settings.