Left Menu
Development News Edition

New AI tool to spot spoilers for you!

Researchers have developed an AI-based system that can figure out spoilers in online reviews of books and TV shows.

ANI | Washington DC | Updated: 13-07-2019 23:00 IST | Created: 13-07-2019 22:53 IST
New AI tool to spot spoilers for you!
Representative image Image Credit: ANI

Researchers have developed an AI-based system that can figure out spoilers in online reviews of books and TV shows. "Spoilers are everywhere on the Internet and are very common on social media. As Internet users, we understand the pain of spoilers, and how they can ruin one's experience," said one of the paper's senior authors, Ndapa Nakashole.

Some websites allow people to manually flag their posts with tags that serve as 'spoiler ahead' warning signs. But this doesn't always happen. So researchers, who presented the study at Association for Computational Linguistics, wanted to develop an artificial intelligence tool powered by neural networks to automatically detect spoilers. They named the tool SpoilerNet. On a theoretical level, researchers want to better understand how people write spoilers and what kind of linguistic patterns and common knowledge mark a sentence as a spoiler.

The tool the researchers developed could be used to build a browser extension to shield people from spoilers. To train and test SpoilerNet, researchers went looking for large datasets of sentences containing spoilers. Spoiler alert! They found none. So they created their own by collecting more than 1.3 million book reviews annotated with spoiler tags by book reviewers. The tags encompass sentences that include spoilers and hide them behind a "view spoiler" link in the text. The reviews were collected from Goodreads, a social networking site that allows people to track what they read, and share thoughts and reviews with other readers.

"To our knowledge, this is the first dataset with spoiler annotations at this scale and at such a fine-grained granularity," said the paper's first author, Mengting Wan. Researchers found that spoiler sentences tend to clump together in the latter part of reviews. But they also found that different users had different standards to tag spoilers, and neural networks needed to be carefully calibrated to take this into account.

In addition, the same word may have different semantic meanings in different contexts. For example, 'green' is just color in one book review, but it can be the name of an important character and a signal for spoilers in another book. Identifying and understanding these differences is challenging, Wan said. Researchers trained SpoilerNet on 80 percent of the reviews on Goodreads, running the text through several layers of neural networks. The system could detect spoilers with 89 to 92 percent accuracy.

They also ran SpoilerNet on a dataset of more than 16,000 single-sentence reviews of about 880 TV shows. The accuracy of the tool to detect spoilers was 74 to 80 percent. Most of the errors came from the system getting distracted by words that are usually loaded and revelatory for example murder or killed.

Looking forward, the Goodreads dataset can be used as a powerful tool to train algorithms to detect spoilers in different types of content said tweets containing spoilers.


TRENDING

OPINION / BLOG / INTERVIEW

How UK’s 'best prepared' healthcare system failed to gauge COVID-19

The UK is proud of their public health system and its unlike any other country as around 90 percent of British public supports the founding principles of National Health Service. But without accurate data being available to stakeholders in ...

Poor on IHR capacity progress in 2019, WHO says Cambodia tops COVID-19 response

Despite being in proximity to Hubei, the original epicenter of COVID-19 pandemic, Cambodia has reported just 226 confirmed cases and zero deaths. After seeing the data, WHO appreciated Cambodias healthcare information system but experts dou...

Loopholes in Healthcare Information System may have failed Singapore COVID-19 model

In the initial days of the COVID-19 outbreak, Singapore was in the limelight for its effective healthcare system and pandemic response plan. However, Singapore has now joined the list of the worst-hit nations and the situation is even worse...

Australia's COVID-19 response: Digital infrastructure of help but implementation remains a challenge

Australias ongoing plans to upgrade its health information system helped by the Digital Health Strategy seem even more practical due to the pandemic. But as evident during the pandemic, administrative lapses and the complex matrix of power ...

Videos

Latest News

Made in India Video Conferencing App 'Lauk' Launched

Respite to the privacy concerns, Lauk comes with an end-to-end encryption trully, a vocal for local call New Delhi, Delhi, India NewsVoir Senior Journalist turned Entrepreneur Anuranjan Jha has launched Lauk, a video conferencing and web ...

Masked messaging: Pandemic changes means of expression, ‘gags’ do the talking

Beneath this mask there is more than flesh. Beneath this mask there is an idea, Mr. Creedy, and ideas are bulletproof, anarchist V tells corrupt politician Creedy as a pandemic ravages Europe in the 2005 film V for Vendetta. In 2020, the wo...

BCCI not to punish players if they admit to age fudging, two-year ban otherwise

The BCCI on Monday said it will grant amnesty to registered players who voluntarily declare any age fraud but those found to be in breach of the unusual scheme could be slapped with a two-year suspension. The measures will be applicable to ...

Bank of India Q1 net profit rises over three-fold to Rs 844 cr

State-owned Bank of India BoI on Monday reported over three-fold rise in net profit to Rs 843.60 crore for the quarter ended June 30, as pressure of bad loans eased. The bank had registered a net profit of Rs 242.60 crore in the April-June ...

Give Feedback