General-purpose agents in human-machine teams: Can we preserve meaningful human control?

Abstract

As General-Purpose AI (GPAI) grows more capable of cross-domain reasoning and multi-agent coordination, it reshapes the design space of military human-machine teams and introduces new opportunities for adaptability as well as new risks for Meaningful Human Control (MHC). The paper presents an integrated delegation framework for preserving MHC in GPAI-enabled human-machine teams. It combines play-based task structuring, normative constraints, LLM-mediated interaction, and an exploratory empirical evaluation with military professionals. Part of this framework is DIALOG, an LLM-mediated interaction component that translates natural language into doctrine-aligned plays, and builds up a context with ethical, legal, and operational constraints. We evaluate this framework in TNO's vehicle simulation facility, where five military professionals executed tactical missions using closed play, open play, and goal-based delegation modes. Findings reveal a nuanced autonomy-control trade-off: operators initially favored higher-autonomy modes but reverted to more structured control when confronted with unexpected or non-doctrinal agent behaviors. The results highlight implications for MHC, transparency, and trust calibration, and point toward research directions for enhancing tactical competence, communication fidelity, and verifiability in GPAI-enabled human-machine teams.