Tag Archives: data

There’s a Multibillion-Dollar Market for Your Phone’s Location Data

Above: Photo / Adobe Stock

A huge but little-known industry has cropped up around monetizing people’s movements

Companies that you likely have never heard of are hawking access to the location history on your mobile phone. An estimated $12 billion market, the location data industry has many players: collectors, aggregators, marketplaces, and location intelligence firms, all of which boast about the scale and precision of the data that they’ve amassed.

Location firm Near describes itself as “The World’s Largest Dataset of People’s Behavior in the Real-World,” with data representing “1.6B people across 44 countries.” Mobilewalla boasts “40+ Countries, 1.9B+ Devices, 50B Mobile Signals Daily, 5+ Years of Data.” X-Mode’s website claims its data covers “25%+ of the Adult U.S. population monthly.”

In an effort to shed light on this little-monitored industry, The Markup has identified 47 companies that harvest, sell, or trade in mobile phone location data. While hardly comprehensive, the list begins to paint a picture of the interconnected players that do everything from providing code to app developers to monetize user data to offering analytics from “1.9 billion devices” and access to datasets on hundreds of millions of people. Six companies claimed more than a billion devices in their data, and at least four claimed their data was the “most accurate” in the industry.

The Location Data Industry: Collectors, Buyers, Sellers, and Aggregators

The Markup identified 47 players in the location data industry

Created by Joel Eastwood and Gabe Hongsdusit. Source: The Markup. (See our data, including extended company responses, here.)

“There isn’t a lot of transparency and there is a really, really complex shadowy web of interactions between these companies that’s hard to untangle,” Justin Sherman, a cyber policy fellow at the Duke Tech Policy Lab, said. “They operate on the fact that the general public and people in Washington and other regulatory centers aren’t paying attention to what they’re doing.” 

Occasionally, stories illuminate just how invasive this industry can be. In 2020, Motherboard reported that X-Mode, a company that collects location data through apps, was collecting data from Muslim prayer apps and selling it to military contractors. The Wall Street Journal also reported in 2020 that Venntel, a location data provider, was selling location data to federal agencies for immigration enforcement. 

A Catholic news outlet also used location data from a data vendor to out a priest who had frequented gay bars, though it’s still unknown what company sold that information. 

Many firms promise that privacy is at the center of their businesses and that they’re careful to never sell information that can be traced back to a person. But researchers studying anonymized location data have shown just how misleading that claim can be. 

The truth is, it’s hard to know all the ways in which your movements are being tracked and traded. Companies often reveal little about what apps serve as the sources of data they collect, what exactly that data consists of, and how far it travels. To piece together a picture of the ecosystem, The Markup reviewed the websites and marketing language of each of the 47 companies we identified as operating in the location data industry, as well as any information they revealed about how the data got to them. (See our methodology here.)

How the Data Leaves Your Phone

Most times, the location data pipeline starts off in your hands, when an app sends a notification asking for permission to access your location data. 

Apps have all kinds of reasons for using your location. Map apps need to know where you are in order to give you directions to where you’re going. A weather, waves, or wind app checks your location to give you relevant meteorological information. A video streaming app checks where you are to ensure you’re in a country where it’s licensed to stream certain shows. 

But unbeknownst to most users, some of those apps sell or share location data about their users with companies that analyze the data and sell their insights, like Advan Research. Other companies, like Adsquare, buy or obtain location data from apps for the purpose of aggregating it with other data sources. Companies like real estate firms, hedge funds and retail businesses might then turn and use the data for their own advertising, analytics, investment strategy, or marketing purposes. 

Serge Egelman, a researcher at UC Berkeley’s ​​International Computer Science Institute and CTO of AppCensus, who has researched sensitive data permissions on mobile apps, said it’s hard to tell which apps on your phone simply use the data for their own functional purposes and which ones release your data into the economic ether.

“When the app asks for location, in the moment, because maybe you click the button to find stuff near you and you get a permission dialog, you might reasonably infer that ‘Oh, that’s to service that request to provide that functionality,’ but there’s no guarantee of that,” Egelman said. “And there’s certainly usually never a disclosure that says that the data is going to be limited to that purpose.”

Companies that trade in this data are reluctant to share which apps they get data from. 

The Markup asked spokespeople from all the companies on our list where they get the location data they obtain. 

Companies like Adsquare and Cuebiq told The Markup that they don’t publicly disclose what apps they get location data from to keep a competitive advantage but maintained that their process of obtaining location data was transparent and with clear consent from app users. 

“It is all extremely transparent,” said Bill Daddi, a spokesperson for Cuebiq.

He added that consumers must know what the apps are doing with their data because so few consent to share it. “The opt-in rates clearly confirm that the users are fully aware of what is happening because the opt-in rates can be as low as less than 20%, depending on the app,” Daddi said in an email. 

Yiannis Tsiounis, the CEO of the location analytics firm Advan Research, said his company buys from location data aggregators, who collect the data from thousands of apps—but would not say which ones. Tsiounis said the apps he works with do explicitly say that they share location data with third parties somewhere in the privacy policies, though he acknowledged that most people don’t read privacy policies. 

“There’s only so much you can squeeze into the notification message. You get one line, right? So you can’t say all of that in the notification message,” Tsiounis said. “You only get to explain to the user, ‘I need your location data for X, Y, and Z.’ What you have to do is, there has to be a link to the privacy policy.”  

Only one company spokesperson, Foursquare’s Ashley Dawkins, actually named any specific apps—Foursquare’s own products, like Swarm, CityGuide, and Rewards—as sources for its location data trove. 

But Foursquare also produces a free software development kit (SDK)—a set of prebuilt tools developers can use in their own apps—that can potentially track location through any app that uses it. Foursquare’s Pilgrim SDK is used in apps like GasBuddy, a service that compares prices at nearby gas stations, Flipp, a shopping app for coupons, and Checkout 51, another location-based discount app. 

GasBuddy, Flipp, and Checkout 51 didn’t respond to requests for comment.

A search on Mighty Signal, a site that analyzes and tracks SDKs in apps, found Foursquare’s Pilgrim SDK in 26 Android apps. 

While not every app with Foursquare’s SDK sends location data back to the company, the privacy policies for Flipp, Checkout 51, and GasBuddy all disclose that they share location data with the company.

Foursquare’s method of obtaining location data through an embedded SDK is a common practice. Of the 47 companies that The Markup identified, 12 of them advertised SDKs to app developers that could send them location data in exchange for money or services.

Placer.ai says in its marketing that it does foot traffic analysis and that its SDK is installed in more than 500 apps and has insights on more than 20 million devices. 

“We partner with mobile apps providing location services and receive anonymized aggregated data. Very critically, all data is anonymized and stripped of personal identifiers before it reaches us,” Ethan Chernofsky, Placer.ai’s vice president of marketing, said in an email. 

Into the Location Data Marketplace 

Once a person’s location data has been collected from an app and it has entered the location data marketplace, it can be sold over and over again, from the data providers to an aggregator that resells data from multiple sources. It could end up in the hands of a “location intelligence” firm that uses the raw data to analyze foot traffic for retail shopping areas and the demographics associated with its visitors. Or with a hedge fund that wants insights on how many people are going to a certain store.

“There are the data aggregators that collect the data from multiple applications and sell in bulk. And then there are analytics companies which buy data either from aggregators or from applications and perform the analytics,” said Tsiounis of Advan Research. “And everybody sells to everybody else.” 

Some data marketplaces are part of well-known companies, like Amazon’s AWS Data Exchange, or Oracle’s Data Marketplace, which sell all types of data, not just location data. Oracle boasts its listing as the “world’s largest third-party data marketplace” for targeted advertising, while Amazon claims to “make it easy to find, subscribe to, and use third-party data in the cloud.” Both marketplaces feature listings for several of the location data companies that we examined.

Amazon spokesperson Claude Shy said that data providers have to explain how they gain consent for data and how they monitor people using the data they purchase.

“Only qualified data providers will have access to the AWS Data Exchange. Potential data providers are put through a rigorous application process,” Shy said. 

Oracle declined to comment.

Other companies, like Narrative, say they are simply connecting data buyers and sellers by providing a platform. Narrative’s website, for instance, lists location data providers like SafeGraph and Complementics among its 17 providers with more than two billion mobile advertising IDs to buy from. 

But Narrative CEO Nick Jordan said the company doesn’t even look at the data itself. 

“There’s a number of companies that are using our platform to acquire and/or monetize geolocation data, but we actually don’t have any rights to the data,” he said. “We’re not buying it, we’re not selling it.” 

To give a sense of how massive the industry is, Amass Insights has 320 location data providers listed on its directory, Jordan Hauer, the company’s CEO, said. While the company doesn’t directly collect or sell any of the data, hedge funds will pay it to guide them through the myriad of location data companies, he said.

“The most inefficient part of the whole process is actually not delivering the data,” Hauer said. “It’s actually finding what you’re looking for and making sure that it’s compliant, making sure that it has value and that it is exactly what the provider says it is.”

Oh, the Places Your Data Will Go

There are a whole slew of potential buyers for location data: investors looking for intel on market trends or what their competitors are up to, political campaigns, stores keeping tabs on customers, and law enforcement agencies, among others.

Data from location intelligence firm Thasos Group has been used to measure the number of workers pulling extra shifts at Tesla plants. Political campaigns on both sides of the aisle have also used location data from people who were at rallies for targeted advertising.

Fast food restaurants and other businesses have been known to buy location data for advertising purposes down to a person’s steps. For example, in 2018, Burger King ran a promotion in which, if a customer’s phone was within 600 feet of a McDonalds, the Burger King app would let the user buy a Whopper for one cent.

The Wall Street Journal and Motherboard have also written extensively about how federal agencies including the Internal Revenue Service, Customs and Border Protection, and the U.S. military bought location data from companies tracking phones. 

Of the location data firms The Markup examined, the offerings are diverse. 

https://bookshop.org/a/565/9781479837243

Advan Research, for instance, uses historical location data to tell its customers, largely retail businesses or their private equity firm owners, where their visitors came from, and makes guesses about their income, race, and interests based on where they’ve been. 

“For example, we know that the average income in this neighborhood by census data is $50,000. But then there are two devices—one went to Dollar General, McDonald’s, and Walmart, and the other went to a BMW dealer and Tiffany’s … so they probably make more money,” Advan Research’s Tsiounis said.

Others combine the location data they obtain with other pieces of data gathered from your online activities. Complementics, which boasts data on “more than a billion mobile device IDs,” offers location data in tandem with cross-device data for mobile ad targeting.

The prices can be steep. 

Outlogic (formerly known as X-Mode) offers a license for a location dataset titled “Cyber Security Location data” on Datarade for $240,000 per year. The listing says “Outlogic’s accurate and granular location data is collected directly from a mobile device’s GPS.” 

At the moment, there are few if any rules limiting who can buy your data. 

Sherman, of the Duke Tech Policy Lab, published a report in August finding that data brokers were advertising location information on people based on their political beliefs, as well as data on U.S. government employees and military personnel. 

“There is virtually nothing in U.S. law preventing an American company from selling data on two million service members, let’s say, to some Russian company that’s just a front for the Russian government,” Sherman said. 

Existing privacy laws in the U.S., like California’s Consumer Privacy Act, do not limit who can purchase data, though California residents can request that their data not be “sold”—which can be a tricky definition. Instead, the law focuses on allowing people to opt out of sharing their location in the first place. 

​​The European Union’s General Data Protection Regulation has stricter requirements for notifying users when their data is being processed or transferred. 

But Ashkan Soltani, a privacy expert and former chief technologist for the Federal Trade Commission, said it’s unrealistic to expect customers to hunt down companies and insist they delete their personal data.

 “We know in practice that consumers don’t take action,” he said. “It’s incredibly taxing to opt out of hundreds of data brokers you’ve never even heard of.”  

Companies like Apple and Google, who control access to the app stores, are in the best position to control the location data market, AppCensus’s Egelman said. 

“The real danger is the app gets booted from the Google Play store or the iOS app store,” he said.” As a result, your company loses money.” 

Google and Apple both recently banned app developers from using location reporting SDKs from several data companies.  

Researchers found, however, that the companies’ SDKs were still making their way into Google’s app store. 

Apple didn’t respond to a request for comment. 

“The Google Play team is always working to strengthen privacy protections through both product and policy improvements. When we find apps or SDK providers that violate our policies, we take action,” Google spokesperson Scott Westover said in an email.

Digital privacy has been a key policy issue for U.S. senator Ron Wyden, a Democrat from Oregon, who told The Markup that the big app stores needed to do more. 

“This is the right move by Google, but they and Apple need to do more than play whack-a-mole with apps that sell Americans’ location information. These companies need a real plan to protect users’ privacy and safety from these malicious apps,” Wyden said in an email. 

This article was originally published on The Markup and written by By: Jon Keegan and Alfred Ng was republished under the Creative Commons Attribution-NonCommercial-NoDerivatives license (CC BY-NC-ND 4.0).

Related Articles:


Find books on Big Tech and many other topics at our sister site: Cherrybooks on Bookshop.org

Enjoy Lynxotic at Apple News on your iPhone, iPad or Mac.

Lynxotic may receive a small commission based on any purchases made by following links from this page

What Does It Actually Mean When a Company Says, “We Do Not Sell Your Data”?

Above: Photo Credit / Unsplash

Experts say the privacy promise—ubiquitous in online services and apps—obscures the truth about how companies use personal data

You’ve likely run into this claim from tech giants before: “We do not sell your personal data.” 

Companies from Facebook to Google to Twitter repeat versions of this statement in their privacy policies, public statements, and congressional testimony. And when taken very literally, the promise is true: Despite gathering masses of personal data on their users and converting that data into billions of dollars in profits, these tech giants do not directly sell their users’ information the same way data brokers directly sell data in bulk to advertisers

But the disclaimers are also a distraction from all the other ways tech giants use personal data for profit and, in the process, put users’ privacy at risk, experts say. 

Lawmakers, watchdog organizations, and privacy advocates have all pointed out ways that advertisers can still pay for access to data from companies like Facebook, Google, and Twitter without directly purchasing it. (Facebook spokesperson Emil Vazquez declined to comment and Twitter spokesperson Laura Pacas referred us to Twitter’s privacy policy. Google did not respond to requests for comment.)

And focusing on the term “sell” is essentially a sleight of hand by tech giants, said Ari Ezra Waldman, a professor of law and computer science at Northeastern University.

“[Their] saying that they don’t sell data to third parties is like a yogurt company saying they’re gluten-free. Yogurt is naturally gluten-free,” Waldman said. “It’s a misdirection from all the other ways that may be more subtle but still are deep and profound invasions of privacy.”

Those other ways include everything from data collected from real-time bidding streams (more on that later), to targeted ads directing traffic to websites that collect data, to companies using the data internally.

How Is My Data at Risk if It’s Not Being Sold? 

Even though companies like Facebook and Google aren’t directly selling your data, they are using it for targeted advertising, which creates plenty of opportunities for advertisers to pay and get your personal information in return.

The simplest way is through an ad that links to a website with its own trackers embedded, which can gather information on visitors including their IP address and their device IDs. 

Advertising companies are quick to point out that they sell ads, not data, but don’t disclose that clicking on these ads often results in a website collecting personal data. In other words, you can easily give away your information to companies that have paid to get an ad in front of you.

If the ad is targeted toward a certain demographic, then advertisers would also be able to infer personal information about visitors who came from that ad, Bennett Cyphers, a staff technologist at the Electronic Frontier Foundation, said. 

For example, if there’s an ad targeted at expectant mothers on Facebook, the advertiser can infer that everyone who came from that link is someone Facebook believes is expecting a child. Once a person clicks on that link, the website could collect device IDs and an IP address, which can be used to identify a person. Personal information like “expecting parent” could become associated with that IP address.  

“You can say, ‘Hey, Google, I want a list of people ages 18–35 who watched the Super Bowl last year.’ They won’t give you that list, but they will let you serve ads to all those people,” Cyphers said. “Some of those people will click on those ads, and you can pretty easily figure out who those people are. You can buy data, in a sense, that way.” 

Then there’s the complicated but much more common way that advertisers can pay for data without it being considered a sale, through a process known as “real-time bidding.” 

Often, when an ad appears on your screen, it wasn’t already there waiting for you to show up. Digital auctions are happening in milliseconds before the ads load, where websites are selling screen real estate to the highest bidder in an automated process. 

Visiting a page kicks off a bidding process where hundreds of advertisers are simultaneously sent data like an IP address, a device ID, the visitor’s interests, demographics, and location. The advertisers use this data to determine how much they’d like to pay to show an ad to that visitor, but even if they don’t make the winning bid, they have already captured what may be a lot of personal information.  

With Google ads, for instance, the Google Ad Exchange sends data associated with your Google account during this ad auction process, which can include information like your age, location, and interests.

The advertisers aren’t paying for that data, per se; they’re paying for the right to show an advertisement on a page you visited. But they still get the data as part of the bidding process, and some advertisers compile that information and sell it, privacy advocates said.

In May, a group of Google users filed a federal class action lawsuit against Google in the U.S. District Court for the Northern District of California alleging the company is violating its claims to not sell personal information by operating its real-time bidding service.

The lawsuit argues that even though Google wasn’t directly handing over your personal data in exchange for money, its advertising services allowed hundreds of third parties to essentially pay and get access to information on millions of people. The case is ongoing. 

“We never sell people’s personal information and we have strict policies specifically prohibiting personalized ads based on sensitive categories,” Google spokesperson José Castañeda told the San Francisco Chronicle in May

Real-time bidding has also drawn scrutiny from lawmakers and watchdog organizations for its privacy implications.

In January, Simon McDougall, deputy commissioner of the United Kingdom’s Information Commissioner’s Office, announced in a statement that the agency was continuing its investigation of real-time bidding (RTB), which if not properly disclosed, may violate the European Union’s General Data Protection Regulation.

“The complex system of RTB can use people’s sensitive personal data to serve adverts and requires people’s explicit consent, which is not happening right now,” McDougall said. “Sharing people’s data with potentially hundreds of companies, without properly assessing and addressing the risk of these counterparties, also raises questions around the security and retention of this data.”

And in April, a bipartisan group of U.S. senators sent a letter to ad tech companies involved in real-time bidding, including Google. Their main concern: foreign companies and governments potentially capturing massive amounts of personal data about Americans. 

“Few Americans realize that some auction participants are siphoning off and storing ‘bidstream’ data to compile exhaustive dossiers about them,” the letter said. “In turn, these dossiers are being openly sold to anyone with a credit card, including to hedge funds, political campaigns, and even to governments.” 

On May 4, Google responded to the letter, telling lawmakers that it doesn’t share personally identifiable information in bid requests and doesn’t share demographic information during the process.

“We never sell people’s personal information and all ad buyers using our systems are subject to stringent policies and standards, including restrictions on the use and retention of information they receive,” Mark Isakowitz, Google’s vice president of government affairs and public policy, said in the letter.

What Does It Mean to “Sell” Data?

Advocates have been trying to expand the definition of “sell” beyond a straightforward transaction. 

The California Consumer Privacy Act, which went into effect in January 2020, attempted to cast a wide net when defining “sale,” beyond just exchanging data for money. The law considers it a sale if personal information is sold, rented, released, shared, transferred, or communicated (either orally or in writing) from one business to another for “monetary or other valuable consideration.” 

And companies that sell such data are required to disclose that they’re doing so and allow consumers to opt out. 

“We wrote the law trying to reflect how the data economy actually works, where most of the time, unless you’re a data broker, you’re not actually selling a person’s personal information,” said Mary Stone Ross, chief privacy officer at OSOM Products and a co-author of the law. “But you essentially are. If you are a social media company and you’re providing advertising and people pay you a lot of money, you are selling access to them.” 

But that doesn’t mean it’s always obvious what sorts of personal data a company collects and sells. 

In T-Mobile’s privacy policy, for instance, the company says it sells compiled data in bulk, which it calls “audience segments.” The policy states that audience segment data for sale doesn’t contain identifiers like your name and address but does include your mobile advertising ID. 

Mobile advertising IDs can easily be connected to individuals through third-party companies.  

Nevertheless, T-Mobile’s privacy policy says the company does “not sell information that directly identifies customers.”

T-Mobile spokesperson Taylor Prewitt didn’t provide an answer to why the company doesn’t consider advertising IDs to be personal information but said customers have the right to opt out of that data being sold. 

So What Should I Be Looking for in a Privacy Policy? 

The next time you look at a privacy policy, which few people ever really do, don’t just focus on whether or not the company says it sells your data. That’s not necessarily the best way to assess how your information is traveling and being used. 

And even if a privacy policy says that it doesn’t share private information beyond company walls, the data collected can still be used for purposes you might feel uncomfortable with, like training internal algorithms and machine learning models. (See Facebook’s use of one billion pictures from Instagram, which it owns, to improve its image recognition capability.)

Consumers should look for deletion and retention policies instead, said Lindsey Barrett, a privacy expert and until recently a fellow at Georgetown Law. These are policies that spell out how long companies keep data, and how to get it removed. 

She noted that these statements hold a lot more weight than companies promising not to sell your data. 

“People don’t have any meaningful transparency into what companies are doing with their data, and too often, there are too few limits on what they can do with it,” Barrett said. “The whole ‘We don’t sell your data’ doesn’t say anything about what the company is doing behind closed doors.” 

This article was originally published on The Markup and was republished under the Creative Commons Attribution-NonCommercial-NoDerivatives license.


Find books on Sustainable Energy Solutions and Climate Science and many other topics at our sister site: Cherrybooks on Bookshop.org

Enjoy Lynxotic at Apple News on your iPhone, iPad or Mac.

Lynxotic may receive a small commission based on any purchases made by following links from this page

T-Mobile Data Breach affects over 47 Million people

Photo by Mika Baumeister on Unsplash

Data stolen include names, dob, SSN, and much more!

The investigation of the ongoing T-Mobile data breach has revealed some staggering information regarding the number of customers affected. As per a new article from Engadget, T-Mobile has confirmed roughly 47.8 million current and former customers have been affected by the cyberattack.

The company issued a press statement regarding the data breach and below are some of the immediate steps they are taking:

  • As a result of this finding, we are taking immediate steps to help protect all of the individuals who may be at risk from this cyberattack. Communications will be issued shortly to customers outlining that T-Mobile is:
    • Immediately offering 2 years of free identity protection services with McAfee’s ID Theft Protection Service.
    • Recommending all T-Mobile postpaid customers proactively change their PIN by going online into their T-Mobile account or calling our Customer Care team by dialing 611 on your phone. This precaution is despite the fact that we have no knowledge that any postpaid account PINs were compromised.
    • Offering an extra step to protect your mobile account with our Account Takeover Protection capabilities for postpaid customers, which makes it harder for customer accounts to be fraudulently ported out and stolen.
    • Publishing a unique web page later on Wednesday for one stop information and solutions to help customers take steps to further protect themselves.

As a T-Mobile customer myself, this is quite worrisome. The data stolen includes personal information like names (first and last), date of birth, social security numbers and driver license numbers. It is unclear at the moment if the stolen files have information that would contain financial account numbers or passwords.

Read at:


Find books on Politics and many other topics at our sister site: Cherrybooks on Bookshop.org

Enjoy Lynxotic at Apple News on your iPhone, iPad or Mac and subscribe to our newsletter.

Lynxotic may receive a small commission based on any purchases made by following links from this page

Why Web Scraping Is Vital to Democracy

Photo Credit / Fabio / Unsplash

Journalists have used scrapers to collect data that rooted out extremist cops, tracked lobbyists, and uncovered an underground market for adopted children

By: The Markup Staff

The fruits of web scraping—using code to harvest data and information from websites—are all around us.

People build scrapers that can find every Applebee’s on the planet or collect congressional legislation and votes or track fancy watches for sale on fan websites. Businesses use scrapers to manage their online retail inventory and monitor competitors’ prices. Lots of well-known sites use scrapers to do things like track airline ticket prices and job listings. Google is essentially a giant, crawling web scraper.

Scrapers are also the tools of watchdogs and journalists, which is why The Markup filed an amicus brief in a case before the U.S. Supreme Court this week that threatens to make scraping illegal.

The case itself—Van Buren v. United States—is not about scraping but rather a legal question regarding the prosecution of a Georgia police officer, Nathan Van Buren, who was bribed to look up confidential information in a law enforcement database. Van Buren was prosecuted under the Computer Fraud and Abuse Act (CFAA), which prohibits unauthorized access to a computer network such as computer hacking, where someone breaks into a system to steal information (or, as dramatized in the 1980s classic movie “WarGames,” potentially start World War III).

In Van Buren’s case, since he was allowed to access the database for work, the question is whether the court will broadly define his troubling activities as “exceeding authorized access” to extract data, which is what would make it a crime under the CFAA. And it’s that definition that could affect journalists.

Or, as Justice Neil Gorsuch put it during Monday’s oral arguments, lead in the direction of “perhaps making a federal criminal of us all.”

Investigative journalists and other watchdogs often use scrapers to illuminate issues big and small, from tracking the influence of lobbyists in Peru by harvesting the digital visitor logs for government buildings to monitoring and collecting political ads on Facebook. In both of those instances, the pages and data scraped are publicly available on the internet—no hacking necessary—but sites involved could easily change the fine print on their terms of service to label the aggregation of that information “unauthorized.” And the U.S. Supreme Court, depending on how it rules, could decide that violating those terms of service is a crime under the CFAA.

“A statute that allows powerful forces like the government or wealthy corporate actors to unilaterally criminalize newsgathering activities by blocking these efforts through the terms of service for their websites would violate the First Amendment,” The Markup wrote in our brief.

What sort of work is at risk? Here’s a roundup of some recent journalism made possible by web scraping:

  • The COVID tracking project, from The Atlantic, collects and aggregates data from around the country on a daily basis, serving as a means of monitoring where testing is happening, where the pandemic is growing, and the racial disparities in who’s contracting and dying from the virus.
  • This project, from Reveal, scraped extremist Facebook groups and compared their membership rolls to those of law enforcement groups on Facebook—and found a lot of overlap.
  • Reveal also used scrapers to find that hundreds of millions of dollars in property taxes should have never been charged to Detroit residents who then lost their homes through foreclosure.
  • The Markup’s recent investigation into Google’s search results found that it consistently favors its own products, leaving some websites from which the web giant itself scrapes information struggling for visitors and, therefore, ad revenue. The U.S. Department of Justice cited the issue in an antitrust lawsuit against the company. 
  • In Copy, Paste, Legislate, USA Today found a pattern of cookie-cutter laws, pushed by special interest groups, circulating in legislatures around the country.
  • Reuters scraped social media and message boards to find an underground market for adopted children whose parents, who had usually adopted the children from abroad, decided the children were too much for them. A couple featured in the piece was later convicted of kidnapping as a result of the investigation.
  • Gizmodo was able to use similar tools to find the probable locations of tens of thousands of Ring surveillance cameras.
  • The Trace and The Verge, using scrapers, found people using an online market to sell guns without a license and without performing background checks.

This article was originally published on The Markup and was republished under the Creative Commons Attribution-NonCommercial-NoDerivatives license.

Recent Articles:


Find books on Music, Movies & Entertainment and many other topics at our sister site: Cherrybooks on Bookshop.org

Enjoy Lynxotic at Apple News on your iPhone, iPad or Mac.

Lynxotic may receive a small commission based on any purchases made by following links from this page

Apple’s Ad Spotlights in a Hilarious Tour de Force: Why Privacy Matters on your iPhone

Yes that includes Netflix -and-chill text invites

We all constantly interact with our iPhone in a myriad of ways. We use our phones for directions, searching for information, snapping a photo, communicating with work-family-friends-lovers, buying (drunk-dailing), accessing our financial data, enter and compiling health data…. the list goes on and on!

Read More: Apple Privacy in iOS 14 and Big Sur: Safari to offer deep pervasive control of personal data

Apple’s newest ad highlights how important our privacy really is. Although funny, the ad also depicts what the worst case scenario could look like.

Privacy is extremely high on Apple’s list

No one would knowingly would scream out their password, but through this clever comedic device the video spotlights the scary truth that if your valuable information were to be shared continuously, the situation would be grave, indeed. Apple wants to make it clear that privacy should be a “given” and that they have taken extra steps to ensure users information will be safe.

Subscribe to our YouTube Channel


Check out all Lynxotic Apple Coverage

Subscribe to our newsletter for all the latest updates directly to your inBox.

Find books on Big TechSustainable EnergyEconomics and many other topics at our sister site: Cherrybooks on Bookshop.org

Enjoy Lynxotic at Apple News on your iPhone, iPad or Mac.

Lynxotic may receive a small commission based on any purchases made by following links from this page.