How to Analyze Communication Data from Laboratory Experiments Without Being a Machine Learning Specialist
Recently, the analysis of communication has gained attention in experimental research. One important question is whether certain types of communication affect decisions differently than others. In this regard, Houser & Xiao (2011) present an approach for the classification of natural language messages. The primary limitation of their approach is its limited applicability to large message datasets. Therefore, Penczynski (2019) extends the methodological instruments by applying a machine learning classifier to experimental communication data. This is accompanied by the problem of a dearth of machine learning knowledge among experimenters. Hence, this paper presents an approach that employs a publicly available machine learning text analysis application. This makes it possible to analyze larger datasets based on small training datasets classified beforehand by human evaluators. As a first step, I use primary communication data reported by Charness and Dufwenberg (2006) to generate both training and test datasets. Following this approach, I am able to substantially replicate the original classification results obtained by Charness and Dufwenberg. The second step again involves messages from Charness and Dufwenberg as training data, while I take messages from a related trust game published by Deck et al. (2013) as a test, dataset. Promisingly, I am also able to replicate the classification results obtained by the external evaluators, as reported by Deck et al. The findings suggest that machine learning can be used to analyze large message datasets, both if the artificial intelligence is trained with data from the very same experiment and if it is trained with message data from a comparable experiment.
Copyright (c) 2021 Benjamin Wegener
This work is licensed under a Creative Commons Attribution 4.0 International License.
Author (s) should affirm that the material has not been published previously. It has not been submitted and it is not under consideration by any other journal. At the same time author (s) need to execute a publication permission agreement to assume the responsibility of the submitted content and any omissions and errors therein. After submission of a revised paper in the light of suggestions of the reviewers, editorial team edits and formats manuscripts to bring uniformity and standardization in published material.
This work will be licensed under Creative Commons Attribution 4.0 International (CC BY 4.0) and under condition of the license, users are free to read, copy, remix, transform, redistribute, download, print, search or link to the full texts of articles and even build upon their work as long as they credit the author for the original work. Moreover, as per journal policy author (s) hold and retain copyrights without any restrictions.