High-Fidelity Single-Pixel Imaging Using Multi-Head Attention Mechanism

Hui Lu, Xinrui Zhan, Liheng Bian*

*Corresponding author for this work

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

Abstract

Single-pixel imaging has gained prominence for its wide working wavelength and high sensitivity. Deep learning-based single-pixel imaging shows superiority in real-time reconstruction, particularly with limited resources. In this work, we report a novel encoder-decoder method for single-pixel imaging, which aims at enhancing imaging quality from extremely low measurement amounts. First, we encode the high-dimensional target information into one-dimensional measurements using globally optimized modulation patterns, implemented by a fully connected or convolutional layer. Second, we integrate a U-Net neural network with an advanced multi-head self-attention mechanism and a pyramid pooling module to decode the measurements and reconstruct high-fidelity images. Under such a strategy, the skip connections within the U-Net structure enhance the preservation of fine image features, and the incorporation of the multi-head self-attention mechanism and pyramid pooling module effectively captures contextual dependencies among low-dimensional measurements, thereby extracting significant image features and enhancing reconstruction quality. The simulation results conducted on the STL-10 dataset validate the efficiency of the reported technique. With a resolution of 96 × 96 pixels and an ultra-low sampling rate of 1%, we consistently achieved the highest image fidelity compared to traditional single-pixel reconstruction methods for both grayscale and color images.

Original languageEnglish
Title of host publicationOptoelectronic Imaging and Multimedia Technology XI
EditorsJinli Suo, Zhenrong Zheng
PublisherSPIE
ISBN (Electronic)9781510682061
DOIs
Publication statusPublished - 2024
EventOptoelectronic Imaging and Multimedia Technology XI 2024 - Nantong, China
Duration: 13 Oct 202415 Oct 2024

Publication series

NameProceedings of SPIE - The International Society for Optical Engineering
Volume13239
ISSN (Print)0277-786X
ISSN (Electronic)1996-756X

Conference

ConferenceOptoelectronic Imaging and Multimedia Technology XI 2024
Country/TerritoryChina
CityNantong
Period13/10/2415/10/24

Keywords

  • multi-head self-attention
  • pyramid pooling modules
  • single-pixel imaging
  • ultra-low sampling rate

Cite this