Publikationen

Autor	Titze, Sophie; Halvani, Oren
Datum	2025
Art	Conference Paper
Abstrakt	This submission addresses Subtask 1 of the Voight-Kampff Generative AI Detection task, which is part of the PAN 2025 lab. The goal of the subtask is to distinguish AI-generated texts from human-written ones, even when the machine-generated texts have been intentionally obfuscated to appear more human-like. As Large Language Models (LLMs) continue to improve in fluency and coherence, this distinction becomes increasingly difficult and requires robust detection strategies. This submission introduces a zero-shot method based on token-level statistics, which are extracted from two pre-trained LLMs: a base model and an instruction-tuned model. This method LOG-AID computes five core features: mean surprisal under each model, Jensen-Shannon divergence between their predictive distributions, average entropy difference, the mean entropy of the base model and the average logarithmic rank of the ground-truth tokens. These features are combined into a fixed-size vector and classified using a logistic regression model. On the official test set, the proposed system achieved a mean score of 0.827 across five metrics, surpassing strong baselines such as Binoculars (0.818) and PPMd Compression (0.758). In particular, the combination of uncertainty-based measures (surprisal, entropy) and rank-based features (log-rank) enhances discriminative power. This contribution offers a simple, interpretable and self-contained classification approach that does not require any fine-tuning. The method relies solely on internal probability structures of pre-trained models and may serve as a lightweight baseline for future work in AI text detection.
Konferenz	Conference and Labs of the Evaluation Forum 2025
Url	https://publica.fraunhofer.de/handle/publica/497951