A Pilot Test of AI Coding of Open-Ended Survey Responses


This white paper details results of a pilot test using artificial intelligence to code open-ended survey responses into quantitative categories. Our aim was to see if our use of AI could simplify the time-consuming task of coding open-ends without sacrificing data quality.

Compared with human coding, we encountered poor category creation, misclassification and an inability to detect nuance or valence. While our investigation was limited in scope, results suggest the need for caution in using AI for open-end coding when data quality is a priority.

Further advances and additional testing may produce better results. We continue to monitor and test this and other potential applications of AI in our research practice.

 

Communication and Motivation Strategies


Our research partners often ask us not simply to measure public attitudes or consumer behavior but also to identify their underlying motivations. Those insights, in turn, inform strategies to communicate effectively with groups and encourage their action.

Successful communication and motivation strategies require not only strong data and analysis, but also an understanding of underlying theories developed by academic researchers over the decades. Our work builds actionable results on the foundation of that accumulated knowledge. This white paper describes the conceptual bases for our communication and motivation work.