As generative AI fashions develop extra highly effective, their power use is changing into a critical bottleneck. A brand new totally optical generative AI chip might assist by working superior picture and video technology duties at speeds and efficiencies orders of magnitude past as we speak’s {hardware}.
Coaching generative AI fashions requires an unlimited quantity of computing energy and power. However as demand explodes, the method of truly working the fashions to create pictures, textual content, or video—often called inference—is rapidly changing into a fair larger drain on sources.
Video and picture technology fashions are significantly power intensive. Whereas the effectivity of those fashions is consistently enhancing, a 2023 examine discovered that producing 1,000 pictures utilizing a number one mannequin produced carbon emissions equal to driving a gas-powered automotive greater than 4 miles.
One promising method for slashing power use is photonic computing, the place processors use gentle as a substitute of electrical energy. It’s a tactic a number of well-funded startups are pursuing in earnest. However most advances have been restricted to easier duties like picture classification or textual content technology.
Now, researchers from Shanghai Jiao Tong College and Tsinghua College in China have demonstrated an all-optical chip they name LightGen that’s greater than 100 instances sooner and extra power environment friendly than a number one Nvidia GPU on duties like video and picture technology.
“LightGen supplies a brand new approach to bridge the brand new chip architectures to every day sophisticated AI with out impairment of efficiency and with pace and effectivity which can be orders of magnitude larger,” the researchers write in a latest paper on the chip in Science.
A key side of the brand new design is its density. Generative fashions sometimes require hundreds of thousands of parameters to supply high-quality outputs, however earlier photonic chips have had, at most, a couple of thousand synthetic neurons. Utilizing 3D packaging, nevertheless, LightGen integrates greater than two million onto a tool measuring only a quarter of a sq. inch.
The ensuing processing enhance permits the chip to work with pictures at resolutions as much as 512-by-512 pixels. Older photonic chips sometimes broke up high-resolution pictures into smaller patches to course of them. This not solely takes longer but in addition reduces a mannequin’s capacity to attract statistical correlations between the completely different patches.
The researchers additionally innovated one thing known as an “optical latent house.” Generative AI fashions work, partly, by compressing high-dimensional knowledge into easier representations. This forces them to take away much less necessary data and solely retain the bits which can be integral to the enter.
These condensed representations are then saved in a multi-dimensional map of ideas known as a latent house. Fashions use these representations to generate new outputs when given a immediate.
LightGen’s builders replicated this course of fully optically. Of their chip, a full-resolution picture is transmitted via an optical encoder made up of a number of metasurfaces—ultra-thin buildings designed to govern gentle—after which coupled into an array of optical fibers.
This course of naturally filters out higher-order knowledge, successfully condensing the data into easier representations, that are then saved within the fiber array because the optical latent house. One other set of metasurfaces on the different finish of the system, which may be switched relying on the duty, then take the output from this latent house and use it to generate high-resolution pictures.
The researchers additionally got here up with a novel coaching method. Right here, the chip learns probabilistic representations of coaching knowledge, which makes it doable to sort out extra complicated duties, like creating novel outputs. It is a promising improvement. To this point, most photonic chips have targeted on inference not coaching.
The crew examined their chip on a number of demanding duties, together with the technology of high-resolution pictures of animals, changing pictures into completely different creative types, and even turning 2D pictures into 3D fashions. Notably, the chip achieved speeds and power efficiencies greater than two orders of magnitude higher than Nvidia’s A100 GPU, one of many firm’s strongest AI chips.
The brand new optical chip isn’t prepared to interrupt out of the lab simply but. It nonetheless depends on cumbersome lasers and spatial gentle modulators to generate enter indicators, and the metasurfaces central to its design are presently made with specialised processes quite these you would possibly discover in commonplace chip factories.
Nonetheless, with additional improvement, the work suggests optical processors may very well be a quick, energy-efficient approach to energy the cutting-edge of an more and more power-hungry AI business.
