The Complications of Deep Packet Inspection

By David Bruggeman
July 23, 2008

Update July 27
The Washington Post ran an article on Friday describing a case of an internet service provider conducting deep packet inspection on customers in Kansas. Notice was affected through a change in the company’s privacy policy on its website. Subscribers were offered the opportunity to opt out of the test, but some lawmakers are of the opinion that is insufficient in these situations, that the intrusion on privacy is sufficient to require participants to opt in to the process.

You can read the letter to the internet service provider from the House Energy and Commerce Committee, as well as the ISP’s response letters, online.

Original Post
As computing power and data storage increase, the limits of what are possible with computer science change and new capacities emerge. Part of the new trends in online advertising appears to take advantage of one such change – the relative ease of deep packet inspection (DPI).

A brief description of packets and DPI (those more familiar with the topic can skip ahead). Internet traffic is typically broken down into packets – bursts of data that include routing information. So, if we were looking at an email message, for example, the message wouldn’t travel from sender to receiver as one piece, but as a number of packets, which were routed through the network by routers. Packets are already inspected by routers to determine where they should be sent. This is shallow packet inspection, and isn’t that far off from a post office or shipping service doing spot checks of pieces of mail to make sure they are headed to where they are supposed to go.

With DPI, both the content and the routing information of the packet is inspected. There are legitimate uses for DPI, which can include detecting network threats, various forms of viruses, and intellectual property, as well as managing network congestion. None of these requires a particularly deep inspection of the packet, but does assess the content to some degree, looking for patterns rather than individual behaviors.

Now there are forms of DPI equipment that could dig deeper into the packets, and can be used to construct profiles to facilitate targeted advertisements. If this information is retained, the amount of personal data and web browsing history that can be collected and mined is significant. This represents a shift in consumer expectations. Absent notice, no one really anticipates that their email, commercial transactions, browser history and other internet activity could be monitored and examined as closely as they can be with DPI. This expectation is likely to carry forward, even as more sensitive information (like health and financial data) is more commonly sent over the internet.

So this increased capability of DPI provides new policy issues, most of them revolving around fair information practices. While the equipment is still being used in limited deployment, now would be the critical time to determine what DPI users must provide in terms of consent, access, data retention, and other information use practices to make sure that consumers’ privacy is maintained and that the use of DPI can continue without causing a chilling effect in the use of the internet.

These issues were addressed in a recent hearing before the House Energy and Commerce Committee. You can read the written witness testimony online. A transcript of the July 17 hearing is not yet available, though you can watch the archived video.