A good story, from Fast Company, on social media sites’ discrimination against researchers:
Independent researchers have used web scraping to reveal large-scale disinformation operations, horrifying malfunctions in platform algorithms, and more. But scraped data isn’t only beneficial for public interest research—it also has enormous commercial value. Scraping is the bread and butter of the “social listening” industry, which collects and analyses social media data on behalf of companies who want to find out what people think of their brand and keep tabs on trends that might impact their business. Multimillion-dollar companies like Brandwatch and Meltwater use a variety of methods to collect this data, including web scraping, and sell access to their data tools though subscriptions that cost thousands of dollars per month. Yet while researchers are routinely served cease-and-desist letters for the same practices, social listening companies are considered trusted partners of social media companies.
As the author—researcher Brandi Geurnik—notes, firms like Facebook often weaponize privacy to shut down scholars’ access. Now the machine-learning boom is leading sites to aggressively monetize API access—pricing researchers out.