Home »
Academic Articles »
C »
The fluency-based semantic network of LLMs differs from humans
The fluency-based semantic network of LLMs differs from humans
Ye Wang
(State Key Laboratory of Media Convergence and Communication, Communication University of China, No. 1 of Dingfuzhuang East Street, Chaoyang District, Beijing, 100024, China; Neuroscience and Intelligent Media Institute, Communication University of China, No. 1 of Dingfuzhuang East Street, Chaoyang District, Beijing, 100024, China)
Yaling Deng
(State Key Laboratory of Media Convergence and Communication, Communication University of China, No. 1 of Dingfuzhuang East Street, Chaoyang District, Beijing, 100024, China; Neuroscience and Intelligent Media Institute, Communication University of China, No. 1 of Dingfuzhuang East Street, Chaoyang District, Beijing, 100024, China)
Ge Wang
(School of Computer and Cyber Sciences, Communication University of China, No. 1 of Dingfuzhuang East Street, Chaoyang District, Beijing, 100024, China)
Tong Li
(School of Information and Communication Engineering, Communication University of China, No. 1 of Dingfuzhuang East Street, Chaoyang District, Beijing, 100024, China)
Hongjiang Xiao
(State Key Laboratory of Media Convergence and Communication, Communication University of China, No. 1 of Dingfuzhuang East Street, Chaoyang District, Beijing, 100024, China; Corresponding author.)
Yuan Zhang
(State Key Laboratory of Media Convergence and Communication, Communication University of China, No. 1 of Dingfuzhuang East Street, Chaoyang District, Beijing, 100024, China)
Journal: Computers in Human Behavior: Artificial Humans
•
• Elsevier
• ISSN: 2949-8821
Abstract
Modern Large Language Models (LLMs) exhibit complexity and granularity similar to humans in the field of natural language processing, challenging the boundaries between humans and machines in language understanding and creativity. However, whether the semantic network of LLMs is similar to humans is still unclear. We examined the representative closed-source LLMs, GPT-3.5-Turbo and GPT-4, with open-source LLMs, LLaMA-2-70B, LLaMA-3-8B, LLaMA-3-70B using semantic fluency tasks widely used to study the structure of semantic networks in humans. To enhance the comparability of semantic networks between humans and LLMs, we innovatively employed role-playing to generate multiple agents, which is equivalent to recruiting multiple LLM participants. The results indicate that the semantic network of LLMs has poorer interconnectivity, local association organization, and flexibility compared to humans, which suggests that LLMs have lower search efficiency and more rigid thinking in the semantic space and may further affect their performance in creative writing and reasoning.
Ye Wang, Yaling Deng, Ge Wang, et al. (2025). "The fluency-based semantic network of LLMs differs from humans". Computers in Human Behavior: Artificial Humans. https://doi.org/10.1016/j.chbah.2024.100103