Computer Vision Tools for Protest Analysis

Dr. Michelle Torres

August 31, 2024

Course Overview and Objectives

Political science has changed dramatically in the last decade. Until recently, it has never been easier to retrieve and get access to massive amounts of data with different content and format: from written to spoken speeches, videos of political campaigns, voting history of millions of citizens, real time political event data, and much more. In particular, visual content, a key component of the political communication process, is now easier to obtain, store, and analyze.

Technological advances have not only allowed researchers, especially in the computer science f ield, to access the aforementioned visual data but also to develop and use methods that were unthinkable a few years ago. While these data and methods have the potential to revolutionize the study of politics by allowing the emergence of new questions and the possibility of new ways of understanding and explaining political puzzles, the traditional social science toolkit does not suffice for their analysis. Therefore, in this workshop we cover the application of two popular computer vision tools to the study of questions relevant to social scientists studying protests.

Workshop content

1. Image basics: pixels, features, and image structure

2. From text to images: similarities and differences • Finding tokens and features in images • Challenges

3. Convolutional Neural Networks

• Motivation: image classification

• The UCLA Protest Data

• Re-training and using a model for image classification: violence, composition, magnitude, and more

4. Transformers

• Motivation: facial identification for size estimation

• Identifying faces using HuggingFaces

• More than just a face: demographics, emotions, etc.

5. To infinity and beyond!

• Other tools • Challenges and costs

• Things to consider and further research

Useful texts

• Won, Donghyeon, Zachary C. Steinert-Threlkeld, and Jungseock Joo. 2017. “Protest activity detection and perceived violence estimation from social media images.” Proceedings of the 25th ACM international conference on Multimedia.

• Torres, Michelle. 2023. “A framework for the unsupervised and semi-supervised analysis of images.” Political Analysis.

• Torres, Michelle, and Francisco Cant´ u. 2022. “Learning to see: Convolutional neural networks for the analysis of social science data.” Political Analysis 30(1): 113-131.

• Webb Williams, Nora, Andreu Casas, and John D. Wilkerson. 2020. Images as data for social science research: An introduction to convolutional neural nets for image classification. Cambridge, UK: Cambridge University Press.

Requirements

Please check the ReadMe file in the following GitHub repository smtorres, and complete the steps indicated there. Please bring your laptop to the workshop; we will use it to do some “live” coding.