Google Glass

A new gadget joined the P3F team. We are finally proud owners of the Google Glass Explorer version. 🙂

Besides playing around and exploring the Glass API, we are currently working on a proposal to apply for a netidee follow-up grant. We would really like to take P3F to the next level and include Glass feature.

Wearable such as Glass are exciting – but preserving bystanders’ privacy is challenging!




P3F @ SOUPS conference

Last week, I visited SOUPS (Symposium on Usable Privacy and Security) in Menlo Park, CA, USA at the Facebook HQ. Besides listening to many interesting talks, I had the chance to meet researchers from all over the world to exchange ideas and to spread the word about P3F.


Panel Discussion @ SOUPS


Facebook Campus in Menlo Park

After the conference, I spent some days in San Francisco and also visited the EFF, an international non-profit digital rights group to participate in a Usable Crypto Hackathon.



Manikin Monika is helping out


New staff for our project! ‘Monika’ is helping out with test shots for our project. Katharina needs to focus more on programing again (seems to make her happy). This week she will visit SOUPS (Symposium on usable privacy and security) at Facebook headquarters. Let’s see, how our project is received there and what the newest developments in the sector are.

P3F @ Netidee Spring Talk

In course of the netidee spring talk, the P3F project team had the opportunity to present their first results and discuss their challenges with other teams that are funded by the netidee grant program. After an afternoon full of workshops and presentations, a delicious buffet was served 😉



New Challenges for Wearable Privacy

According to a news article by Gizmodo Google has formally announced that it’s opening sales of Google Glass to everyone in the U.S. Even though the price ($1,500) will still detain some potential costumers from buying this promising new device, the permanent video record of humans in public spaces is a potential scenario that we will encounter in the next years. Google Glass therefore has several privacy implications on the people that are recorded by these devices.

This development highlights the need for privacy enhancing techniques that are reliable and easy to use. The P3F framework has the ability to provide such a usable and efficient countermeasure against contiuous video recording and mass surveillance.

The Framework Architecture

After evaluating the algorithms to be used in the P3F framework, we defined the general architecture of our framework and started with the prototype implementation in Matlab.

Architecture of the P3F Framework

Architecture of the P3F Framework

The overall goal is to modify an input image in a way so all persons on the respective image are presented as defined by their privacy policies. The first step consists of preprocessing (i.e. removal of distortions, calculation of integral/greyscale images, …). After preprocessing the image is segmented. In the P3F framework, image segmentation is performed by a face locator, a person locator and a data locator and decoder to obain a list of persons and tags. From these lists, the privacy policies are deducted and furthermore enforced.


Testing Person Correlation

Our first tests with person correlation where performed with QR-Codes. In this example, we used QR codes to encode privacy policies. The overall goal of this experimental setup was to test person correlation inherently to the artifacts in which the privacy policies are encoded. We are not intending to use QR codes in a later stage of the project as they are big, intrusive and everything else but aesthetic. However, due to their feasibility for this particular test on person correlation they serve as an excellent baseline for investigation.

One of the essentials of the P3F framework is efficient and reliable face detection in order to correlate the persons on the picture with the artifacts used to encode the privacy policies.

Our results have shown, that false positives from the face detector can lead to very unpleasant effects for the users. In case of a fale positive, the P3F tags are assigned away from real persons to falsely detected faces that coincidentely happen to be nearer. Thus, they weaken the personal protection of the actual persons in the picture. We will address this problem by revising our framework and by testing several face detection algorithms that have been investigated in scientific publications and used in industrial applications.


Readings on Human Body Segmentation

One of the essential components of the P3F framework is designed to perform human body segmentation. Human body segmentation is essential to correlate the privacy policy obtained via a certain artifact with the face of a specific person shown on the picture. Especially when it comes to pictures with a “noisy” background, or a crowd where some humans in the front cover other humans in the back of the image, precise human body segmentation is difficult to perform. The following pictures shows an example of such a scene:

Shibuya Crossing, Tokyo

Shibuya Crossing, Tokyo

To improve the body segmenter module of the P3F framework, we are currently evaluating algorithms that have been published in scientific literature. We are currently reading the following two papers and evaluating the proposed algorithms with respect to their applicability for the P3F framework:

Juang, Chia-Feng, et al. “Computer vision-based human body segmentation and posture estimation.” Systems, Man and Cybernetics, Part A: Systems and Humans, IEEE Transactions on 39.1 (2009): 119-133.:

Mori, Greg, et al. “Recovering human body configurations: Combining segmentation and recognition.” Computer Vision and Pattern Recognition, 2004. CVPR 2004. Proceedings of the 2004 IEEE Computer Society Conference on. Vol. 2. IEEE, 2004.



Criteria for Marking Patterns

Technical requirements:

a) Illumination stability:
The code should be decodable under a wide range of lighting conditions. However, under conditions making face identification impossible, a decoding failure is tolerable.
b) Blurriness tolerance:
Picture blurriness can arise from sub-optimal auto-focus mechanisms because the photographer actually focused on another object or person or moved the camera during exposure (a common problem with amateur photographers).
c) Size and clipping invariance:
The code should be decodable from shots with different fields of view. Therefore, it should be so redundant that a partial capture in a close-up produces results as good as those in a wide shot. Furthermore, in a wide shot, a larger part of the code is recorded but with a reduced resolution compared to a close-up. Fine encoding that repeats multiple times is better for close-up shots while coarse encoding is better for wide shots. Ideally, a code unifies both traits.
d) Distortion stability:
People do not always face the camera head-on, especially when they are being photographed unintentionally. Furthermore, the human body is not a flat board, and loose clothing tends to fold and wrinkle. Another faults may arise from lense distortion or improper washing or drying of the person’s clothing.
e) Noise robustness:
Another artifact introduced by cameras is noise, especially in low-light and low-contrast situations due to the automatic camera gain amplifying the sensors background noise.
f) Computational weight:
The detection algorithm should be lightweight because operators of publishing systems will most likely demand one that conserves computational resources.
g) Compression stability:
Digital photography greatly depends on picture compression algorithms. They commonly destroy details in pictures and introduce artifacts. These algorithms are often based on a psycho-visual model of human visual perception and are therefore not optimized for computer vision purposes. The most common compression method for photographs on the Internet is JPEG.
h) Blind decoding ability:
The decoder should have the ability to decode the data without prior knowledge of the original pattern used to encode the data or the data that is being looked for (a common prerequisite for some watermarking techniques).
i) Detection accuracy:
Detection accuracy should be high with a slight bias toward false positives since people typically feel more comfortable with more privacy than with less. False positives can still be overridden by the publisher if necessary.
j) Error detection and correction:
The encoding scheme should have an error detection or correction code to avoid producing erroneous results.

Aesthetic Demands:

a) Dress code:
Dress codes are often imposed by society, the employer, or another external entity. The coding scheme should thus produce markings and patters that blends into the imposed dress code.
b) Fashion:
People additionally often have their own fashion demands. The coding scheme should thus produce markings that blends into the individual’s fashion style.
c) Adaptive:
Clothing is sold in many different colors and shapes. The code should thus be versatile and work with many different colors and shapes.
d) Unobtrusive:
The application of P3F should require only a slight adjustment in clothing style. The code should be subtle with low visual impact. It should be unrecognizable by other people, thus minimizing social complications.