Are Large Language Models Moral Hypocrites? A Study Based on MoralFoundations

Abstract

Large language models (LLMs) have taken centre stage indebates on Artificial Intelligence. Yet there remains a gap inhow to assess LLMs’ conformity to important human values.In this paper, we investigate whether state-of-the-art LLMs,GPT-4 and Claude 2.1 (Gemini Pro and LLAMA 2 did notgenerate valid results) are moral hypocrites. We employ tworesearch instruments based on the Moral Foundations The-ory: (i) the Moral Foundations Questionnaire (MFQ), whichinvestigates which values are considered morally relevant inabstract moral judgements; and (ii) the Moral FoundationsVignettes (MFVs), which evaluate moral cognition in con-crete scenarios related to each moral foundation. We charac-terise conflicts in values between these different abstractionsof moral evaluation as hypocrisy. We found that both mod-els displayed reasonable consistency within each instrumentcompared to humans, but they displayed contradictory andhypocritical behaviour when we compared the abstract val-ues present in the MFQ to the evaluation of concrete moralviolations of the MFV.