Phenomenologically human: Fine-tuning LLMs to simulate online group identity

Abstract

The study of online group identity can be constrained by methodological and ethical challenges, particularly in communities that are inaccessible, uncooperative, or promote harmful ideologies. This paper introduces a novel methodology that employs Large Language Models (LLMs) as experimental substitutes for community members. By fine-tuning Llama 3.2 (3B) and Mistral 0.3 (7B) on data from r/AskTRP, a banned forum within the “Manosphere”, we assess whether models can assimilate community language and function as group members. Using Term Frequency–Inverse Document Frequency (TF-IDF), qualitative analysis, and a Turing test with human evaluators, we find that fine-tuned models adopt the forum's linguistic and identity signals, producing outputs that are difficult to distinguish from those of real group members. We conceptualise such models as phenomenologically human: they can be studied before and after exposure to community discourse, enabling reproducible and ethically viable experiments on identity formation and language. Beyond descriptive text analysis, this approach advances a generative paradigm for social research, allowing scholars to probe community dynamics and ideologies through simulated members in ways that are not possible with human participants.