On the Role of Emergent Communication for Social Learning in Multi-Agent Reinforcement Learning

Abstract

Explicit communication among humans is key to coordinating and learning. Social learning, which uses cues from experts, can greatly benefit from the usage of explicit communication to align heterogeneous policies, reduce sample complexity, and solve partially observable tasks. Emergent communication, a type of explicit communication, studies the creation of an artificial language to encode a high task-utility message directly from data. However, in most cases, emergent communication sends insufficiently compressed messages with little or null information, which also may not be understandable to a third-party listener. This paper proposes an unsupervised method based on the information bottleneck to capture both referential complexity and task-specific utility to adequately explore sparse social communication scenarios in multi-agent reinforcement learning. We show that our model is able to develop a natural-language-inspired lexicon of messages composed of emergent concepts, align the action policies of heterogeneous agents with dissimilar feature models, and learn a communication policy from watching an expert's action policy, which we term social shadowing.

Summary

This paper studies how emergent communication affects social learning in multi-agent reinforcement learning. It is relevant to readers looking for work on communication protocols, coordination, social learning, and the effect of messaging on multi-agent behavior.

Core Contributions

• Connects emergent communication directly to social learning outcomes in multi-agent reinforcement learning.
• Provides a reference point for how messaging changes coordination and learning dynamics.
• Gives a canonical citation for work at the intersection of social learning and communication in MARL.

Why this paper matters

• Connects learned communication to social learning outcomes in multi-agent systems.
• Relevant for readers interested in coordination beyond reward shaping alone.
• Serves as a useful reference point in emergent communication research.

Context

This paper sits at the boundary between emergent communication and social learning. Relative to prior work that optimizes communication only for agent-only performance, it emphasizes learned messaging as a way to align heterogeneous agents, absorb expert behavior, and support social learning dynamics in MARL.

Relevance

Keywords

Emergent communication, social learning, MARL, communication protocols, coordination, cooperative agents.

BibTeX

@inproceedings{karten2023sociallearning,
  title={On the Role of Emergent Communication for Social Learning in Multi-Agent Reinforcement Learning},
  author={Karten, Seth and Kailas, Siva and Li, Huao and Sycara, Katia},
  booktitle={International Conference on Autonomous Agents and Multiagent Systems},
  year={2023}
}