The deep neural networks that drive today’s most advanced machine learning applications have reached such significant levels of size and complexity that they are beginning to challenge the capabilities of traditional electronic computing hardware. Photonic hardware, which utilizes light for machine learning computations, emerges as a faster and more energy-efficient alternative. However, certain types of neural network calculations remain beyond the capabilities of photonic devices, necessitating the use of off-chip electronics or other methods that can compromise overall speed and efficiency.
Building on years of extensive research, scientists from MIT, alongside other institutions, have introduced an innovative photonic chip designed to surmount these obstacles. They have successfully demonstrated a fully integrated photonic processor capable of executing all essential calculations of a deep neural network optically, directly on the chip. Remarkably, this optical device achieved the necessary computations for a machine learning classification task in under half a nanosecond while maintaining over 92% accuracy, a level of performance that rivals conventional hardware.
The chip consists of interconnected modules that form an optical neural network and is fabricated using commercial foundry methods, making it potentially scalable and integrable with existing electronic systems. In the long term, this photonic processor could revolutionize deep learning applications across various domains, including lidar, scientific research in areas like astronomy and particle physics, and high-speed telecommunications.
Saumil Bandyopadhyay, who leads this research and is a visiting scientist in the Quantum Photonics and AI Group at the Research Laboratory of Electronics (RLE) at MIT, notes, "In many scenarios, the performance of the model is not the only factor; speed in obtaining results is also crucial. With our end-to-end system capable of running a neural network utilizing optics at nanosecond time scales, we can shift our focus to higher-level applications and algorithms." Bandyopadhyay co-authored the paper alongside a group of notable researchers, including Alexander Sludds, Nicholas Harris, Darius Bunandar, and Stefan Krastanov, among others, with their findings published in Nature Photonics.
Advancing Machine Learning with Light
Deep neural networks function through interconnected layers of nodes, or neurons, that process input data to produce results. A critical operation within these networks involves linear algebra techniques, particularly matrix multiplication, which facilitates data transformation as it progresses through layers. In addition to linear operations, deep neural networks also carry out nonlinear operations that enable the model to recognize more complex patterns. These nonlinear functions, such as activation functions, provide neural networks with the capability to tackle intricate challenges.
Back in 2017, Englund’s team, in collaboration with researchers from Marin Soljačić’s lab, showcased an optical neural network on a single photonic chip that performed matrix multiplication with light. However, at that time, the device was unable to handle nonlinear operations natively on the chip. Instead, optical data needed to be converted to electrical signals for processing by a digital processor, which limited speed and efficiency.
"The challenge of nonlinearity in optics stems from the difficulty of photons interacting with each other, requiring significant energy to trigger optical nonlinearities. Consequently, designing a scalable system that effectively implements these operations becomes challenging," explains Bandyopadhyay. To address this, the team developed devices known as nonlinear optical function units (NOFUs), which merge electronic and optical components to execute nonlinear operations directly on the chip.
The researchers constructed an optical deep neural network on the photonic chip featuring three layers that accommodate both linear and nonlinear operations. Their system begins by encoding the parameters of the deep neural network into light, followed by the utilization of programmable beamsplitters to perform matrix multiplication on the inputs. The data is subsequently routed to the programmable NOFUs, which implement nonlinear functions by diverting a small portion of light to photodiodes that transform optical signals into electric current. This innovative process reduces energy consumption by eliminating the need for an external amplifier.
"We maintain the entire operation within the optical domain until the final step when results are read out, which leads to ultra-low latency," Bandyopadhyay remarks. This remarkably low latency allowed for efficient training of a deep neural network directly on the chip, a method known as in situ training, which ordinarily incurs considerable energy costs in digital systems.
The photonic system achieved over 96% accuracy during training tests and more than 92% accuracy during inference, which is competitive with traditional hardware performance, with key computations completed in less than half a nanosecond. “This work illustrates that computing—at its core, the mapping of inputs to outputs—can be reimagined with new architectures utilizing linear and nonlinear physics, allowing for fundamentally different scaling laws of computation in relation to the effort required,” Englund elaborates.
Notably, the entire circuit was produced using the same infrastructure and foundry processes that manufacture CMOS computer chips. This advance implies the possibility of scalable production while minimizing errors in fabrication.
Moving forward, scaling up this device and integrating it with real-world electronics, such as cameras and telecommunications systems, will be a primary focus for the research team. Additionally, they aim to investigate algorithms that optimize the advantages of optics for faster and more energy-efficient system training.
This research received funding from several entities, including the U.S. National Science Foundation, the U.S. Air Force Office of Scientific Research, and NTT Research, positioning it at the forefront of innovation in photonic machine learning applications.
