My current research focus is hypergraph-based learning models on datasets with many modalities (e.g., text, image, and video), with applications in online news reporting and social media analysis. By extracting multimodal features using natural language processing (NLP) and computer vision (CV) techniques, I analyze the media’s responsibility of the biased representation of social minorities focusing on gender, race, migrants, and refugees. I have also worked on developing Bayesian deep learning on time series forecasting and implementing convolutional neural networks (CNN) for the neural mechanisms of political ideology using the fMRI brain dataset.
BEYOND PAIRWISE RELATIONSHIP: HYPERGRAPH
AS A NEW GRAPH-BASED PARADIGM IN POLITICAL NETWORKS
Multimodal datasets contain a huge amount of information from diverse modalities, such as text, audio, and video. Big data revolution and artificial intelligence have created unprecedented research opportunities for political scientists to delve deeply into huge amounts of multimedia objects. A growing number of scholars realize the need to develop a data-driven method to combine diverse unstructured data such as text, audio, and video in a unified model and encode complex relationships between them. To address such challenging issues and search for new methods, this project introduces a hypergraph as a new graph-based paradigm to handle multimodal datasets and model rich patterns of complex relationships among multimedia objects. Specifically, I introduce three hypergraph-based learning models and various applications ranging from political communication to legislative politics.
A PICTURE IS WORTH A THOUSAND WORDS.
MACHINE-LEARNING VISUAL FRAMING ANALYSIS
This paper presents an automated machine learning method to jointly explore word phrases and visual features of photographs in an unsupervised manner to measure media bias in contemporary media sources. I develop a scalable hypergraph regularized tensor decomposition that maps multimedia items stored in a three-order tensor into a low dimensional semantic space to uncover hidden topic structures in media coverages. Analyzing 173,204 articles with news photographs from 145 online newspapers for political bias in news reporting about abortion and immigration,
my method examines the patterns of news reporting on the visual and verbal level and identifies politically charged phrases and visual characteristics.
IMAGE WITH TEXT:
MULTIMODAL FRAMING ANALYSIS OF ONLINE NEWS COVERAGE
ON THE EUROPEAN REFUGEE CRISIS
In news reporting about conflict and crisis, photographs convey stories that generate emotions of all kinds that words cannot always deliver. While the rapid growth of online news photographs has created unprecedented research opportunities, quantitative approaches that deal with the volume, variety, and complexity of both images and texts have lagged behind in social science. To address such challenging issues and search for new methods, this paper introduces a new method for quantitative framing research to examine the patterns of news reporting on the visual and verbal level and explore image-text relations in news stories. Specifically, I introduce hypergraph as a new graph-based method to integrate the various types of data and model their complex relationships in a network. Unlike conventional graph structures that capture exclusively dyadic relationships, a hypergraph constructs rich patterns of more than dyadic relationships to better convey the underneath geometrical structure in various data types. Using hypergraph, I develop a hypergraph regularized topic model that fuses the visual, textual, and other multimedia features simultaneously to find the latent topic representation in media coverage during the European refugee crisis in 2015. Using this new topic model, I confirm that the visual and textual representation of the refugee crisis varies between right-biased and left-biased news sources. This paper aims to provide scholars with a new quantitative tool to investigate not only what news stories get mentioned but also how they are described visually and verbally.
FUNCTIONAL CONNECTIVITY PREDICTS POLITICAL IDEOLOGY
A growing body of evidence suggests that political behavior is grounded in biology and that liberals and conservatives exhibit different cognitive and emotional patterns. The goal of this study is to investigate the extent to which the functional connectivity of the brain can predict political ideology and to explore the neural mechanisms of political ideology across a range of tasks. To investigate the predictive ability of functional connectivity, we apply a convolutional neural network framework to fMRI data from 174 healthy individuals across eight tasks and resting state. Individuals self-reported their political ideology as well as other socio-economic indicators in a survey associated in the study. Our findings demonstrate that (i) functional connectivity captures biological signatures associated with political orientation, (ii) data from separate tasks provide complementary insights into political ideology, and (iii) functional connectivity arising from reward, affect, and empathy tasks is highly predictive of political ideology.
BAYESIAN DEEP LEARNING FOR IDENTIFYING GRANGER CAUSAL GRAPHS AND FORECASTING POLITICAL DYNAMICS
Time series modeling and Forecasting conflicts has traditionally been made using regression models of different types with parametric assumptions in political science. Current pre-assumed regression models for time series forecasts still face limitations in many empirical applications. First, classical Bayesian time series models do not scale. With the advent of Big Data, we now have many alternative ways to forecast conflicts by extracting insights from massive high-quality data. Second, identifying a suitable forecasting model for a particular time series beforehand is not possible due to the lack or incompleteness of our domain knowledge in many cases. To address such challenges, we propose Bayesian scalable causal graph learning (BSCGL). BSCGL models find the form of mapping function between input and output directly from data and capture the nonlinearities that traditional linear/nonlinear statistical models cannot fully develop. Thus, more complex relationships between time series can be discovered without relying on domain knowledge and any distribution assumption, resulting in better in-sample and out-of-sample prediction performances. Our proposed model also discovers non-linearities in the underlying granger causal mechanisms in time series.