Jinfo: How to scrape the web for news and adverse-media monitoring

How to scrape the web for news and adverse-media monitoring
Jinfo Community

18th November 2025

Description

The Jinfo Report, "Optimising licensed content, web scraping and GenAI for adverse-media monitoring" includes five strategic actions for information managers from this Community session.

Read a summary of the findings in our blog item "Actions to balance the risks of adverse-media monitoring and web scraping for news content".

How to scrape the web for news and adverse-media monitoring The appetite for news continues to grow, and social media platforms continue to outstrip traditional news media channels.

Web scraping is an increasingly attractive approach to gather news, but it's not without its challenges. Scraping the web for adverse-media, or negative news, further complicates the issue.

So, how can you build a comprehensive news and adverse-media monitoring capability?

In this Community session, Toby Cook from Opoint discussed the practicalities of web-scale news scraping, and adverse-media monitoring, including:

What is a scraper and how does it work?
How are AI and AI agents enhancing scraping?
How do you balance the fear of missing critical stories against drowning in false positives and misinformation?
What are the practicalities of using the scraped search results? (e.g. Paywalls and handling results from multiple sites).

Slide presentation and notes

This event has passed, and a recording and/or slide deck is available through a Jinfo Subscription.

If your organisation has a Jinfo Subscription, please sign in here so that we can check your account: