Computerworld – Overshadowing Apple’s earnings news this past week was the publicity surrounding the discovery that iPhones and 3G iPads track users’ locations and store the data in an unencrypted file. The discovery was made by O’Reilly researchers Alasdair Allan and Pete Warden, and it caused quite an uproar.
The file, named “consolidated.db,” is an unencrypted SQLite database that can be found in the devices’ file systems and in the iOS backup files created and updated by iTunes every time an iPhone or 3G iPad is synced. Although the file isn’t immediately accessible on the device itself, it can be accessed on a jailbroken device via the iTunes-generated backup file. It also could potentially be accessed using other tools that allow you to explore an iOS device’s file system while it’s attached to a computer.
The file contains location data about cell towers the device accessed and Wi-Fi networks that it was within range of, plus other information, like the direction a device was facing as determined by the digital compass that became standard on the iPhone 3GS. Other data points appear to be supported by the database file but don’t appear to be used in its current iteration.
One type of data included in the file noted by Allan and Warden is related to so-called geofencing. Geofencing allows a business or organization to create a virtual “fence” around a location that can provide information to mobile devices. A coffee shop could use a geofence technology to broadcast daily specials, or a school could use it to create a perimeter that would allow a phone to indicate that a child has safely arrived for class or is headed home.
Allan and Warden created a proof-of-concept Mac OS X app that can pull information from the database of a user’s iPhone backup and display it on a map — clearly showing where a device has been used.
It’s worth noting that an iPhone’s position isn’t being continuously tracked. When I ran the app, for example, it showed a number of sporadic entries between upstate N.Y. where I live and the location in Virginia where two of my friends got married last August. If my iPhone had been recording my location constantly, there would’ve been a solid line of entries through New Jersey, Delaware, and Maryland. The only entries along the route I drove, however, were at places where I used a location-related feature or app — to look up directions, to check the distance to the next rest area, to snap photos in D.C., or to check in at restaurants.
This clearly implies that the file records data when and where the iOS location services are used (although all manner of apps use location services, potentially generating a lot of entries).
It’s also important to realize that this file doesn’t have actual GPS data. It contains location data based on other, less accurate, sources — like cell tower triangulation and a database of known Wi-Fi hot spots.
While this week’s news about the location-tracking file generated a lot of angst — and prompted members of Congress to ask Apple what’s going on — this isn’t really new information. Data forensics specialists have known about this file for some time, along with a file called h-cells.plist that stored similar location data in previous iOS versions. It was much more difficult to extract that file or its data, however.
O'Reilly researchers Alasdair Allan and Pete Warden created an app that creates a map showing where an iPhone has been, based on the data in the consolidated.db file. (The information displayed here is from an iPhone owner living in New England.)
Not surprisingly, the consolidated.db file and the earlier version of it have been used in data forensic investigations by law enforcement agencies. Although that may raise the hackles of some privacy advocates, similar information can be retrieved from cell carriers by court order during an investigation.
Just what is this file for?
So what is consolidated.db doing on the iPhone? This is the probably the biggest unanswered question. Not being an Apple iOS engineer, I have no inside knowledge, but I’m fairly confident that Apple’s goal is not to maliciously spy on the whereabouts of every iPhone owner. Apple has more important things to do with its time and resources. Also, it appears that this file itself is never actually transmitted back to Apple, though information about a phone’s location apparently is, according to the Wall Street Journal.
My assumption is that the file is related to Apple’s collection of location data about available Wi-Fi networks. Apple’s iOS devices have three ways to determine your location: They can collect GPS data (provided the device supports GPS and can get a signal from enough GPS satellites), utilize cell tower triangulation (provided we’re talking about an original iPhone or a 3G iPad and a cell connection can be established), or refer to a database of known Wi-Fi networks.
Even though my Wi-Fi-only iPad doesn’t support GPS or cellular communication, it very often nails my location with surprising accuracy based on nearby networks — regardless of which one I’m actually connected to. That’s because it has access to a broad global database of known public and private networks and their locations.
Up through iOS 3.1, Apple relied on a database known as Skyhook. Beginning with the introduction of iOS 3.2 a year ago, the company quietly began using its own database of Wi-Fi networks and their locations. This change became public in a letter (download PDF) Apple’s chief counsel sent to Reps. Ed Markey (D-Mass.) and Joe Barton (R-Texas) about changes to the company’s privacy policy last July. Apple essentially said that any information it collects about a particular user or device is kept private unless the user consents to sharing it.
Here’s what Apple’s privacy policy says about location-based services:
“To provide location-based services on Apple products, Apple and our partners and licensees may collect, use, and share precise location data, including the real-time geographic location of your Apple computer or device. This location data is collected anonymously in a form that does not personally identify you and is used by Apple and our partners and licensees to provide and improve location-based products and services. For example, we may share geographic location with application providers when you opt in to their location services.”
“Some location-based services offered by Apple, such as the MobileMe ‘Find My iPhone’ feature, require your personal information for the feature to work.”
As Apple builds its own global database of Wi-Fi networks and locations, collecting data from iOS devices worldwide is an ideal way to maintain and update that database. Note: Apple’s not alone. Smartphones based on Google‘s Android operating system do the same thing, as the Wall Street Journal noted in its story.
However, forensic specialists familiar with the consolidated.db file claim that the file is not transmitted to Apple by either an iOS device or iTunes — although that doesn’t mean that some of the data stored in the file isn’t transmitted on its own. If that’s the case, one likely explanation for its presence is to provide third-party apps with easy access to a cache of past location information. This is one reason forensic experts believe Apple moved away from using the private — and difficult to access — h-cells.plist file.
To provide security, all third-party iOS apps run in a sandboxed environment within the iOS file system and onboard memory. Apple made a large number of additional features available to developers in iOS 4, including expanded location capabilities and the ability for some tasks to run in the background. That meant that Apple needed to make location data accessible to a part of the file system that apps can access. John Gruber at Daring Fireball backs this view with what appears to be deeper knowledge of the situation.
Another possibility: Apple may be trying to capture information about the device or, perhaps, carrier performance — the theory expressed by blogger Andy Ihnatko. Given the rap the iPhone got as a result of AT&T’s network problems, I wouldn’t discount the idea that Apple may have wanted firm data about how well its devices are actually working in the real world.
Why is it so vast?
Even absent malicious intent, why does an iPhone or 3G iPad store months and months of data — and why is it carried over from one device to another as Allan and Warden discovered?
I agree with the consensus view on why iOS isn’t purging older data — it’s probably a bug. Simply for performance and space reasons, it would make sense that a location cache be cleaned out periodically — just as any cache file on any desktop or mobile platform should be cleaned out. The fact that data isn’t being culled from the file means it likely got overlooked among other iOS engineering issues over the past year or two.
So why maintain the data across devices? That’s easy: When you replace an iPhone or iPad, you’re given the option of setting your shiny new one up using a backup from its predecessor. To do that, iTunes copies the existing backup to the device, including all your music, apps and preferences — and apparently that consolidated.db file.
What now?
I strongly suspect that the next iOS update will secure this file and probably add automatic culling of older data. Whether Apple will explicitly say that in the release notes — or even acknowledge the situation — is unknown. Though with Congress asking questions, it seems likely that Apple will have to offer up some kind of response. Until then, I recommend you turn on the option to encrypt your iPhone/iPad backups in iTunes and be prepared to use Apple’s Find My iPhone to remotely wipe the device if it gets lost or stolen.
If you’ve jailbroken your iPhone, there’s already a tool available on Cydia that will automatically wipe consolidated.db on a continuous basis.
For IT professionals who support the iPhone and iPad, there are security policies you can enable using Apple’s iPhone Configuration Utility or a third-party management console to remotely wipe a device after a set number of failed login attempts. Management consoles, like Exchange, can also initiate a remote wipe at any time as needed.
Ryan Faas is a freelance writer and technology consultant specializing in Mac and multiplatform network issues. He has been a Computerworld columnist since 2003 and is a frequent contributor to Peachpit.com. Faas is also the author of iPhone for Work (Apress 2009). You can find out more about him at RyanFaas.com and follow him on Twitter (@ryanfaas).
Read more about Smartphones in Computerworld’s Smartphones Topic Center.