Research on GPT-4 Turbo and Political Communication
A study published in the journal PLOS One reveals that OpenAI's GPT-4 Turbo model can craft responses on behalf of British politicians that people find more authentic, logical, and relevant than the actual statements made by those public figures. Researchers fed the AI 520 answers from participants of the political talk show Question Time, then had the model generate fresh replies to the same questions—this time incorporating each speaker's biography.
Some 948 British participants evaluated the responses using three criteria: how closely they matched a specific person's speech patterns, their logical coherence, and their appropriateness. One group saw only a single answer, another viewed both the real and AI-generated versions side by side, while a third group also received a short biography of the speaker. Results showed that adding the biography had almost no effect on the ratings. Crucially, none of the respondents knew that some answers were produced by a neural network.
Persuasiveness and Ethical Questions Around AI
The AI's answers proved highly persuasive, though about 13% of the generated responses contradicted what the speaker had actually said, according to the study. Moreover, politicians dodged direct questions 28% of the time, while the AI did so far less frequently. Notably, the AI almost never used phrasing that signaled personal uncertainty.
These findings confirm that language models like GPT-4 Turbo tend to confidently guess answers rather than admit a lack of knowledge. An analysis of OpenAI's own research, covered in the journal Science, also highlighted this tendency, raising doubts about the accuracy and justification of responses provided by such models. As a result, this study opens new avenues for understanding how AI interacts with human communication and perception.
The implications of this research could reshape how we view AI's role in political discourse. Since AI models can produce more logical and relevant replies, ethical questions arise about their use in public debates and election campaigns. Future studies may help clarify how AI's perceived reliability will affect trust in political leaders and their communication strategies.