Skip to main content

[1804.03056v1] Automated Discovery of Internet Censorship by Web Crawling

[1804.03056v1] Automated Discovery of Internet Censorship by Web Crawling: Censorship of the Internet is widespread around the world. As access to the web becomes increasingly ubiquitous, filtering of this resource becomes more pervasive. Transparency about specific content that citizens are denied access to is atypical. To counter this, numerous techniques for maintaining URL filter lists have been proposed by various individuals and organisations that aim to empirical data on censorship for benefit of the public and wider censorship research community.
We present a new approach for discovering filtered domains in different countries. This method is fully automated and requires no human interaction. The system uses web crawling techniques to traverse between filtered sites and implements a robust method for determining if a domain is filtered. We demonstrate the effectiveness of the approach by running experiments to search for filtered content in four different censorship regimes. Our results show that we perform better than the current state of the art and have built domain filter lists an order of magnitude larger than the most widely available public lists as of Jan 2018. Further, we build a dataset mapping the interlinking nature of blocked content between domains and exhibit the tightly networked nature of censored web resources.

Popular posts from this blog

(26) Post | LinkedIn

(26) Post | LinkedIn : ► Trump was first compromised by the Russians back in the 80s. In 1984, the Russian Mafia began to use Trump real estate to launder money and it continued for decades. In 1987, the Soviet ambassador to the United Nations, Yuri Dubinin, arranged for Trump and his then-wife, Ivana, to enjoy an all-expense-paid trip to Moscow to consider possible business prospects. Only seven weeks after his trip, Trump ran full-page ads in the Boston Globe, the NYT and WaPO calling for, in effect, the dismantling of the postwar Western foreign policy alliance. The whole Trump/Russian connection started out as laundering money for the Russian mob through Trump's real estate, but evolved into something far bigger. ► In 1984, David Bogatin — a Russian mobster, convicted gasoline bootlegger, and close ally of Semion Mogilevich, a major Russian mob boss — met with Trump in Trump Tower right after it opened. Bogatin bought five condos from Trump at that meeting. Those condos were...