Warning: This election contains language some people may find offensive

When Andrew Jackson ran against incumbent John Quincy Adams in 1828, things got nasty. Adams accused Jackson of adultery and his wife of bigamy. Jackson’s supporters countered by calling Adams a ‘pimp’ and spreading rumors that he had procured an American girl to give sexual services to the Russian czar.

The 1860 campaign between Abraham Lincoln and the candidates of three other parties was filled with insults, name-calling and race-baiting. During the 1876 campaign, Samuel J. Tilden’s opposition called him everything from a briber and a thief to a drunken syphilitic.

Politics has had a tendency to ‘excite the passions of its citizenry’ since time immemorial. The difference these days is that it happens on social media where we can capture and mine it.

A new tool called the Electome, developed by the MIT Media Lab’s Laboratory for Social Machines, now allows us to track incivility in election-related conversations on Twitter in real time. The tool uses natural language processing and semantic analysis to identify and classify election-related tweets by topic and by candidate. In April, the team trained it to identify mean tweets. Since then, it’s captured 12,000 to 50,000 tweets per day that contain insults, profanities, vulgarity and threats of violence related to the election. The number spikes to 100,000 following a debate.

Here’s an overview of what they found: Incivility peaked in March, driven mostly by discussion of racial issues, guns and abortion. It spiked following the outbreaks of violence at Trump rallies and in the run-up to major contests such as the New York primaries.

Data source: Lab for Social Machines, MIT Media Lab, ‘Electome’ civility data for the period November through May. Rolling 10-day average of uncivil tweets by topic.

On average, for the period they collected data, uncivil tweets account for some 7% of all election-related tweets each day. Granted, Twitter’s audience skews male and isn’t representative of the population as a whole. Still mudslinging appears to have no party preference. Left-leaning and right-leaning tweeters are equally crude, but many of the nastiest tweets are in response to Trump’s campaign. Of the six topics tracked by the researchers, racial issues, guns and abortion prompted the greatest proportion of hateful tweets.

According to the lab, the tool identifies uncivil tweets with 94% accuracy, but looking at a set of sample tweets, I see plenty of otherwise substantive tweets get picked up because they contain the words ‘damn,’ ‘crap’ or ‘hell.’  Likewise, the algorithm sometimes gets a little confused semantically, by the name Dick for example. Hey, no algorithm is perfect.

Not surprisingly, profanity is the most common form of incivility in tweets, followed by insults, violence and ethnic and sexual slurs. Interestingly tweets containing threats of violence seemed to spike on Twitter only after incidents happen on the ground, such as the outbreak of violence at the Trump rallies in Fayetteville, North Carolina, and Chicago in early March. Likewise, uncivil conversation appears to spike during and after debates. By contrast, key voting days such as Super Tuesday and the New York primaries saw a drop in incivility as these major contests came to a close.

So, what triggers incivility and why does it matter? Some people think incivility is a good thing, calling it a “crucial rhetorical weapon.” But one concern is that incivility during election cycles may lead to a breakdown in civil society. Research on factors contributing to the Rwandan genocide, for example, showed that hate speech on radio outlets, where the majority of Rwandans got their news, rose dramatically in the time leading up to that country’s civil war. Could we face such a crisis here? It seems unlikely, but tracking incivility in elections could help us better understand its impact on other functions of civil society, including the growing polarization in Congress.

Data source: Voteview.com, The Polarization of the Congressional Parties. Updated: March 21, 2015.

Anecdotal evidence suggests that local politics are becoming fractious too. Consider this report of a city councilmember censored for verbally attacking a colleague, or this report of a slap on the hand by a councilmember which snowballed into assault charges.

The new ability to track and understand the massive national conversation that takes place each election cycle may help us understand its effect on voter efficacy (the belief that one’s vote counts) and voter engagement (the willingness to participate in a political process).

“We have the ability through the Electome to see what people are saying as opposed to the candidates or the media and that’s an interesting new dimension,” said Andrew Heyward, who is careful to point out that the lab does not take a position on whether incivility is good or bad. “We’re not wagging our finger. We’re just shedding light, but as a human I do think it matters.”

Data source: Lab for Social Machines, MIT Media Lab, ‘Electome’ civility data for the period November through May. Proportion of uncivil tweets by type of incivility to all uncivil tweets for each topic tracked.

In addition to the work being done by MIT’s Lab for Social Machines to track election-related incivility, other groups have conducted research in recent years that may help us to better understand what causes rude and vulgar behavior. What they’ve found is that incivility is very predictable. It can be triggered by a few basic conditions, including anonymity, a contest for scarce resources, and being attached to an arbitrary group label—which are fundamental parts of our presidential elections, making them a perfect storm for insults, profanity, violence and slurs.

Research into online communication more broadly, may also help explain what triggers incivility on social media— especially tweets like this one identified by MIT’s tool.

If you’ve ever noticed that people often share or comment on a post to debunk it, you’re seeing something researchers call the third-person effect. It can explain why conversation around extremist issues can escalate and spread. The theory posits that people tend to perceive that mass media messages have a greater effect on others than on themselves  This can lead to users shouting things like “DON”T BELIEVE WHAT YOU READ. THE PEOPLE SPREADING THESE LIES ARE IDIOTS WHO DON’T CARE ABOUT OUR COUNTRY” in the comments section of a politically-inflammatory post and then sharing said post along with a hyperbolic warning to as many people as they can. Research shows this effect increases with age.

Conversely, and perhaps more importantly, research can also help us understand what fosters effective civil discourse. Ravi Iyer is a data scientist and moral psychologist who, along with his colleague Jonathan Haidt, has been gathering and analyzing data to help foster civil discourse in politics through social science. The website for their nonprofit, civilpolitics.org offers a trove of recent research on the issue.

“We don’t define civility as agreement,” says Iyer. In fact, respectful dissent can be valuable to decision-making. “It’s one thing if you disagree with someone and think they have reasonable values and goals. It’s another thing if you demonize them, which can lead to vulgarity, insults, threats of violence or even violence. That’s the dividing line.”

Perceived group affiliation can drive people apart, but personal contact between groups can bring them back together. We get along better if we see individuals from our group getting along with people in the other group (known as the extended contact effect), such as when we work towards a larger common goal. Take right-wing Charles Koch collaborating with left-wing George Soros to tackle prison reform.

To help people interested in improving the quality of political discourse civilpolitics.org provides a summary of evidence-based practices that can measurably influence civil outcomes.

I asked Iyer if we should be concerned about the rise of incivility and more generally, the rise of polarization in politics. After all, our democracy has survived violent protests, brokered conventions and contested election outcomes in the past.

“Yes,” he says. “We’re heading to new levels of polarization. It’s sort of uncharted territory. Now we increasingly are living in communities that all agree with us. We have fewer inter-ideology conversations.”

“The fact that we have muddled on in the past is not predictive that we will continue on in the future,” Iyer warns.

Which makes the work of MIT’s  Lab for Social Machines and others ever more urgent. When it comes to presidential elections, incivility may be here to stay, but at least we may be able understand it—and hopefully each other— better.

Credits: The ‘Electome’ is a project of the Laboratory for Social Machines, MIT Media Lab, with funding by the John S. and James L. Knight Foundation. Soroush Vosoughi and Prashanth Vijayaraghavan, researchers at the Laboratory for Social Machines, MIT Media Lab, prepared the data used in this story.

Kate Stohr is a data journalist and community builder based in San Francisco, CA.