GUILHERME DA FRANCA COUTO FERNANDES DE ALMEIDA
Lattes
ORCID
ScopusProjetos de Pesquisa
Unidades Organizacionais
Resumo profissional
Área de pesquisa
Nome para créditos
2 resultados
Resultados de Busca
Agora exibindo 1 - 2 de 2
Artigo Científico Epistemology Goes AI: A Study of GPT-3?s Capacity to Generate Consistent and Coherent Ordered Sets of Propositions on a Single-Input-Multiple-Outputs Basis(2024) Araújo, Marcelo de; GUILHERME DA FRANCA COUTO FERNANDES DE ALMEIDA; Nunes, José LuizThe more we rely on digital assistants, online search engines, and AI systems to revise our system of beliefs and increase our body of knowledge, the less we are able to resort to some independent criterion, unrelated to further digital tools, in order to asses the epistemic reliability of the outputs delivered by them. This raises some important questions to epistemology in general and pressing questions to applied to epistemology in particular. In this paper, we propose an experimental method for the assessment of GPT-3’s capacity to generate consistent and coherent sets of outputs. When several outputs to one and the same input are very repetitive they tend to be consistent with each other, that is they do not contradict each other. But consistency does not make the set of outputs as a whole more informative than the outputs considered individually. We argue that the less informative a set of outputs is, the less coherent it is. We establish a conceptual distinction between consistency and coherence in the light of what some epistemologists refer to as a coherence theories of truth and justification. While much attention has been given to GPT-3’s capacity to produce internally coherent individual outputs, we argue, instead, that more attention should be given to its capacity to produce consistent and coherent outputs generated on a single-input-multiple-outputs basis.Artigo Científico Exploring the psychology of LLMs’ moral and legal reasoning(2024) GUILHERME DA FRANCA COUTO FERNANDES DE ALMEIDA; Nunes, José Luiz; Engelmann, Neele; Wiegmann, Alex; Araújo, Marcelo deLarge language models (LLMs) exhibit expert-level performance in tasks across a wide range of different domains. Ethical issues raised by LLMs and the need to align future versions makes it important to know how state of the art models reason about moral and legal issues. In this paper, we employ the methods of experimental psychology to probe into this question. We replicate eight studies from the experimental literature with instances of Google's Gemini Pro, Anthropic's Claude 2.1, OpenAI's GPT-4, and Meta's Llama 2 Chat 70b. We find that alignment with human responses shifts from one experiment to another, and that models differ amongst themselves as to their overall alignment, with GPT-4 taking a clear lead over all other models we tested. Nonetheless, even when LLM-generated responses are highly correlated to human responses, there are still systematic differences, with a tendency for models to exaggerate effects that are present among humans, in part by reducing variance. This recommends caution with regards to proposals of replacing human participants with current state-of-the-art LLMs in psychological research and highlights the need for further research about the distinctive aspects of machine psychology