Elena Goi, Assoc. Prof., University of Shanghai for Science and Technology, Institute of photonic chips

Library, A.2.500, Staudtstr. 2

Location details

Abstract:

Diffractive neural networks (DNNs) exploit the nature of light propagation in free space and the interaction of a light field with metasurfaces designed with machine learning methods to implement inference passively in the optical domain¹. In this way, DNNs can achieve all-optical machine learning tasks such as image classification or decryption². When nanoprinted with two photon nanolithography (TPN) methods, DNNs can express their full potential achieving a neuron density of >500 million neurons per square centimeter, operative wavelength in the near infrared or visible wavelength regime and on-chip integration with imaging sensors, such as commercial complementary metal-oxide–semiconductor (CMOS) sensors³. This co-integration is particularly useful, because it allows to compensate the shortcoming of nanoprinted DNNs, which have neither the means to be reconfigured or tuned after fabrication nor any direct way to implement hidden optical nonlinear layers that could help solve more complex tasks and achieve better performances. By co-integrating DNNs with a CMOS imaging sensor, it is possible to create a hybrid integrated optoelectronic modules, where large matrix multiplications are executed in the optical domain by the diffractive elements, followed by a nonlinear optoelectronic conversion that can be modelled as a shifted ReLU function. At this point, the digitalized output of the hybrid integrated optoelectronic module can be processed even further by digital neural networks. For certain tasks, such hybrid optoelectronic neural network architectures exhibit performance comparable to purely digital networks, while requiring up to a factor of >10 less computational resources in the digital domain⁴. This talk will show that the reduction in computational complexity in the digital domain observed for the hybrid neural network architectures is partially due to the optical networks capability of compressing the incoming information into a reduced number of readout pixels. This represents an interesting perspective also in terms of all-optical information processing with hybrid diffractive and integrated neural networks, where the number of input channels to the integrated network is typically limited⁵.

References

1. Lin, X. et al. All-optical machine learning using diffractive deep neural networks. Science 361, 1004–1008 (2018).
2. Goi, E. et al. Nanoprinted high-neuron-density optical linear perceptrons performing near-infrared inference on a CMOS chip. Light Sci. Appl. 10, 40 (2021).
3. Goi, E., Schoenhardt, S. & Gu, M. Direct retrieval of Zernike-based pupil functions using integrated diffractive deep neural networks. Nat. Commun. 13, 7531 (2022).
4. Chen, M., Schoenhardt, S., Gu, M. & Goi, E. Quantitative comparison of the computational complexity of optical, digital and hybrid neural network architectures for image classification tasks. Opt Express 31, 44474–44485 (2023).
5. Xu, Z. et al. Large-scale photonic chiplet Taichi empowers 160-TOPS/W artificial general intelligence. Science 384, 202–209 (2024).