Exploring the psychology of LLMs’ moral and legal reasoning

Página do item simplificado
dc.contributor.authorGUILHERME DA FRANCA COUTO FERNANDES DE ALMEIDA
dc.contributor.authorNunes, José Luiz
dc.contributor.authorEngelmann, Neele
dc.contributor.authorWiegmann, Alex
dc.contributor.authorAraújo, Marcelo de
dc.creatorNunes, José Luiz
dc.creatorEngelmann, Neele
dc.creatorWiegmann, Alex
dc.creatorAraújo, Marcelo de
dc.date.accessioned2024-09-18T22:22:57Z
dc.date.available2024-09-18T22:22:57Z
dc.date.issued2024
dc.description.abstractLarge language models (LLMs) exhibit expert-level performance in tasks across a wide range of different domains. Ethical issues raised by LLMs and the need to align future versions makes it important to know how state of the art models reason about moral and legal issues. In this paper, we employ the methods of experimental psychology to probe into this question. We replicate eight studies from the experimental literature with instances of Google's Gemini Pro, Anthropic's Claude 2.1, OpenAI's GPT-4, and Meta's Llama 2 Chat 70b. We find that alignment with human responses shifts from one experiment to another, and that models differ amongst themselves as to their overall alignment, with GPT-4 taking a clear lead over all other models we tested. Nonetheless, even when LLM-generated responses are highly correlated to human responses, there are still systematic differences, with a tendency for models to exaggerate effects that are present among humans, in part by reducing variance. This recommends caution with regards to proposals of replacing human participants with current state-of-the-art LLMs in psychological research and highlights the need for further research about the distinctive aspects of machine psychologyen
dc.formatDigital
dc.format.extent242 p.
dc.identifier.doi10.1016/j.artint.2024.104145
dc.identifier.issn1872-7921
dc.identifier.issn0004-3702
dc.identifier.urihttps://repositorio.insper.edu.br/handle/11224/6965
dc.language.isoen
dc.relation.ispartofArtificial Intelligence
dc.subjectAI Ethicsen
dc.subjectExperimental jurisprudenceen
dc.subjectEthics of artificial intelligenceen
dc.subjectMachine Behavioren
dc.subjectMoral psychologyen
dc.subjectMachine psychologyen
dc.subjectLarge language modelsen
dc.titleExploring the psychology of LLMs’ moral and legal reasoning
dc.typejournal article
dspace.entity.typePublication
local.identifier.sourceUrihttps://www.sciencedirect.com/science/article/pii/S000437022400081X?via%3Dihub
local.publisher.countryNão Informado
local.subject.cnpqCIENCIAS SOCIAIS APLICADAS
local.typeArtigo Científico
publicationvolume.volumeNumber333
relation.isAuthorOfPublication8575f912-24df-44e3-8512-a288b848e951
relation.isAuthorOfPublication.latestForDiscovery8575f912-24df-44e3-8512-a288b848e951
Arquivos
Pacote Original
Agora exibindo 1 - 2 de 2
N/D
Nome:
ACESSO_RESTRITO_Artigo_2024_Exploring_the_psychology_of_LLMs_moral_and_legal_reasoning_TC.pdf
Tamanho:
2.17 MB
Formato:
Adobe Portable Document Format
Baixar
N/D
Nome:
Acesso_Primeira Pagina_Exploring the psychology of LLMs’ moral and legal reasoning.pdf
Tamanho:
237.32 KB
Formato:
Adobe Portable Document Format
Baixar
Licença do Pacote
Agora exibindo 1 - 1 de 1
N/D
Nome:
license.txt
Tamanho:
236 B
Formato:
Item-specific license agreed upon to submission
Descrição:
Baixar
Coleções
Coleção de Artigos Acadêmicos