Assessment of Argumentation Quality

The article published in issue 29(4) of the Journal of Language Studies (RELIN) presents the entire process of collecting, defining guidelines and annotating the corpus. The article is divided into 6 sections. The first section presents an assessment of the quality of argumentation as a research area and addresses some aspects of the style of messages on the social network Twitter, the data source for this work. The second section presents the works related to this research. The third section presents the aspects of the quality of argumentation applied in this work. The fourth section presents, in detail, the entire process of constructing the corpus. The fifth section presents the process of annotating the corpus according to the guidelines established for this purpose. This section also presents examples of annotation and the spreadsheet used by the annotators, in addition to the annotation statistics and analysis of agreement between annotators. The sixth and last section presents the final considerations and future directions.
Cássio Faria da et al. Quality of argumentation in political tweets: what is and how to measure it. Revista de Estudos da Linguagem, [S.l.], v. 29, n. 4, p. 2537-2586, Jul. 2021. ISSN 2237-2083. Available at: http://dx.doi.org/10.17851/2237-2083.29.4.2537-2586.

The article published in issue 15(1) of the Revista Linguamática describes the experiments carried out with the aim of producing a computational model capable of automatically evaluating the quality of argumentation in political messages written in Portuguese. This article presents the algorithms used and the results of the experiments carried out on the corpus of messages. The first section presents the evaluation of the quality of argumentation as a research area and presents the research questions that this study seeks to answer. The second section presents the works related to this research, including the taxonomy proposed by Wachsmuth et al. (2017). The third section presents, in summary form, the process of elaboration and annotation of the corpus, in addition to the details of how the scores for each of the aspects of the taxonomy were calculated. The fourth section presents the experiments carried out with feature-based ML and the experiments with BERTimbau and RoBERTaTwitterBR. The fifth section presents the results found and the qualitative and quantitative analyses. The sixth section presents the conclusions, limitations and future work.
SILVA, Cássio Faria da; NERIS, Vânia Paula de Almeida; CASELI, Helena de Medeiros. Argument Quality Assessment in Brazilian political tweets. Linguamática, v. 15, n.1, p. 102-127, 7 Jul. 2023. ISSN 1647–0818. Available at: https://doi.org/10.21814/lm.15.1.387.

Research in the area of argumentation, inherent to human beings and essential for both spoken and written communication, dates back to the 4th century BC. The argument is multidisciplinary and covers several fields of research, including computer science. Communication has evolved to social networks, which are a considerable source of argumentative texts in various domains, such as politics. The automatic evaluation of arguments is the subject of recent studies in the area of Natural Language Processing and computational models that seek to perform this task are being proposed, especially based on algorithms based on Support Vector Machines and Deep Learning. In parallel, corpus of argumentative texts in English are being produced. As a way of contributing to research related to the assessment of the quality of argumentation in Portuguese, this work aims to propose, implement and validate a computational approach, for the automatic assessment of the quality of argumentation in the rhetorical dimension in tweets related to Brazilian politics. The approach involves developing a computational model, an annotated corpus with policy-related messages, and task-specific annotation guidelines. The studies carried out here showed that the most appropriate way to assess the quality of arguments in tweets related to Brazilian politics was using a neural model generated from the fine-tuning of BERTimbau. The proposed model was able to predict with 100% accuracy instances of the High quality argumentation class.
SILVA, Cássio Faria da. Abordagem computacional para avaliação automática da qualidade da argumentação na dimensão retórica de tweets no domínio da política brasileira. 2023. Tese (Doutorado em Ciência da Computação) – Universidade Federal de São Carlos, São Carlos, 2023. Available at: https://repositorio.ufscar.br/handle/ufscar/18253.