ChromaCode: A Fully Imperceptible Screen-Camera Communication System
Overview
Hidden screen-camera communication techniques emerge as a new paradigm that embeds data imperceptibly into regular videos while remaining unobtrusive to human viewers. Three key goals on imperceptible, high rate, and reliable communication are desirable but conflicting, and existing solutions usually made a trade-off among them. In this paper, we present the design and implementation of ChromaCode, a screen-camera communication system that achieves all three goals simultaneously. In our design, we consider for the first time color space for perceptually uniform lightness modifications. On this basis, we design an outcome-based adaptive embedding scheme, which adapts to both pixel lightness and regional texture. Last, we propose a concatenated code scheme for robust coding and devise multiple techniques to overcome various screen-camera channel errors. Our prototype and experiments demonstrate that ChromaCode achieves remarkable raw throughputs of >700 kbps, data goodputs of 120 kbps with BER of 0.05, and with fully imperceptible flicker for viewing proved by user study, which significantly outperforms previous works.
Paper
ChromaCode: A Fully Imperceptible Screen-Camera Communication System
Kai Zhang*, Chenshu Wu*, Chaofan Yang, Yi Zhao, Kehong Huang, Chunyi Peng, Yunhao Liu, Zheng Yang (*: co-primary authors)
ACM International Conference on Mobile Computing and Networking (MobiCom), 2018
Video
Materials
In the following, we provide the high resolution versions of several figures (Figure 3) in the paper for better viewing.
Impact of color space
The below figures depict the user-perceptual uniformity under HSL, a non-uniform color space, and CIELAB, a uniform color space respectively. In the first two figures (Hue palettes), the original colors (middle of the each sector) are of the same saturation (100%) and lightness (50%), but are different in hue ranging from 0 to 300. For each original color, we increase and decrease the lightness (outer and inner sector) by the same amount (∆L = 4%). As seen, given the same lightness changes, the resulted color differences are considerably more uniform (meaning that the perceptually color changes are more close under the same amount of lightness changes) in CIELAB space, regardless of the hue values. Similar results can be observed in the later two figures (Lightness palettes) where we keep hue (120) and saturation (100%) unchanged and alter lightness dimension from 10% to 90%.
 |
 |
| Hue palette in HSL color space[PDF] | Hue palette in LAB color space with CIEDE2000[PDF] |
 |
 |
| Lightness palette in HSL color space[PDF] | Lightness palette in LAB color space with CIEDE2000[PDF] |
Impact of texture
The below shows the impacts of video texture on perceptual lightness changes. As seen, smooth regions usually expose noticeable flicker more easily while textured regions help hide flicker.
 |
| Texture plate[PDF] |