virtual contact sheets - cs.cmu.eduworichte/talks/pdlretreat... · a contact sheet approach to...

Virtual Contact Sheets Wolfgang Richter

Kiryong Ha, Alok Shankar, Ardalan Amiri Sani*, Jan Harkes, Lin Zhong*, Mahadev Satyanarayanan

PARALLEL DATA LABORATORY Carnegie Mellon University

*Rice University

Crowdsourcing the News [1] “When every cell phone can record

video and take pictures, everyone is a potential news source.”

Wolfgang Richter © November 11 http://www.pdl.cmu.edu/ 2

Who killed Ian Tomlinson?

CNN: Caught Red Handed [2]


I’m a sneaky thief.


Problem

Cloud

Recent, not uploaded

Older, uploaded

Fundamental Mobile Challenges • Real-time sync not possible:

• Energy •  Infinite battery life desired

• Bandwidth •  Increasingly metered and charged


How far can we degrade the objects in a search before it loses meaning?

Just-in-Time Search 1.  Check virtual contact sheet

•  Collection of fidelity-reduced images in the cloud

2.  Pull full fidelity image on hit •  Search full fidelity; present full fidelity results

to searcher


Cloud


Near Real-Time Sync (1)   Search hit (2)   Retrieve

Contact Sheet Analogy


Evaluating Feasibility [3]


Face Detection, RGB Histogram, Brightness Filter

Benchmark

Using Varying Resolution and JPEG Quality Factor

Energy Savings


0

1000

2000

3000

4000

5000

6000

7000

Full 640x480 320x240 160x120 80x60 40x30 20x20

En

erg

y C

on

sum

ptio

n (

mJ)

Fidelity

Energy

10 images = 5 seconds call time (Droid)

Bandwidth Savings


0

50

100

150

200

250

300

350

400

Full 640x480 320x240 160x120 80x60 40x30 20x20

Bandw

idth

(K

B)

Fidelity

Bandwidth

Orders of magnitude savings in bandwidth.

What about Search Accuracy?


0

0.2

0.4

0.6

0.8

1

640x480 320x240 160x120 80x60

Filt

er

Acc

ura

cy

Fidelity

0.650

0.424

10095755025

No one-size-fits-all sweet spot.

Face Detection

Conclusion & Future Work •  Searches retain meaning

•  No one-size-fits-all, but •  Orders of magnitude bandwidth savings •  Significant energy savings

•  More search primitives? •  Perceptual hashing •  Face recognition •  Texture Filters

•  Better fidelity reduction? •  Keep faces at high fidelity


Further Reading [1] TED. http://www.ted.com/talks/paul_lewis_crowdsourcing_the_news.html.

[2] CNN. http://www.cnn.com/2010/CRIME/08/24/new.jersey.theft.photo/index.html?hpt=C1.

[3] SimpleCV. http://simplecv.org/.

Jan Harkes, Mahadev Satyanarayanan, Ardalan Amiri Sani, and Benjamin Gilbert. A Contact Sheet Approach to Searching Untagged Images on Smartphones. CMU Technical Report CMU-CS-11-132.

Ardalan Amiri Sani, Wolfgang Richter, Xuan Bao, Trevor Narayan, Mahadev Satyanarayanan, Lin Zhong, and Romit Roy Choudhury. Opportunistic Content Search of Smartphones. Rice University Technical Report TR0627-2011.

Xuan Bao, Trevor Narayan, Ardalan Amiri Sani, Wolfgang Richter, Romit Roy Choudhury, Lin Zhong, and Mahadev Satyanarayanan. The Case for Context-Aware Compression. HotMobile 2011.

Jason Flinn and Mahadev Satyanarayanan. Energy-aware Adaptation for Mobile Applications. SOSP 1999.

Y. Nimmagadda, K. Kumar, and Y. H. Lu. Energy-efficient image compression in mobile devices for wireless transmission. ICME 2009.


Full Fidelity

Wolfgang Richter © November 11 http://www.pdl.cmu.edu/ 15 12,000+ Flickr dataset

Average photo size: 374 KB

Reduce Resolution



JPEG Quality Factor



Bandwidth Savings Table


Resolution 100 95 75 50 25 5

640x480 221 KB 114 KB 47 KB 31 KB 19 KB 6 KB

320x240 61 KB 33 KB 14 KB 10 KB 6 KB 2 KB

160x120 17 KB 10 KB 5 KB 3 KB 2 KB 826 B

80x60 5 KB 3 KB 2 KB 1 KB 892 B 460 B

40x30 2 KB 1 KB 725 B 588 B 484 B 345 B

20x20 1000 B 727 B 501 B 436 B 385 B 316 B

virtual contact sheets - cs.cmu.eduworichte/talks/pdlretreat... · a contact sheet approach to...

Documents