As generative AI fashions develop extra highly effective, their power use is turning into a critical bottleneck. A brand new totally optical generative AI chip may assist by operating superior picture and video technology duties at speeds and efficiencies orders of magnitude past immediately’s {hardware}.
Coaching generative AI fashions requires an infinite quantity of computing energy and power. However as demand explodes, the method of really operating the fashions to create photographs, textual content, or video—referred to as inference—is shortly turning into an excellent larger drain on assets.
Video and picture technology fashions are significantly power intensive. Whereas the effectivity of those fashions is continually bettering, a 2023 examine discovered that producing 1,000 photographs utilizing a number one mannequin produced carbon emissions equal to driving a gas-powered automobile greater than 4 miles.
One promising strategy for slashing power use is photonic computing, the place processors use gentle as a substitute of electrical energy. It’s a tactic a number of well-funded startups are pursuing in earnest. However most advances have been restricted to less complicated duties like picture classification or textual content technology.
Now, researchers from Shanghai Jiao Tong College and Tsinghua College in China have demonstrated an all-optical chip they name LightGen that’s greater than 100 occasions quicker and extra power environment friendly than a number one Nvidia GPU on duties like video and picture technology.
“LightGen supplies a brand new approach to bridge the brand new chip architectures to each day sophisticated AI with out impairment of efficiency and with velocity and effectivity which might be orders of magnitude higher,” the researchers write in a current paper on the chip in Science.
A key side of the brand new design is its density. Generative fashions usually require thousands and thousands of parameters to provide high-quality outputs, however earlier photonic chips have had, at most, a number of thousand synthetic neurons. Utilizing 3D packaging, nevertheless, LightGen integrates greater than two million onto a tool measuring only a quarter of a sq. inch.
The ensuing processing increase permits the chip to work with photographs at resolutions as much as 512-by-512 pixels. Older photonic chips usually broke up high-resolution photographs into smaller patches to course of them. This not solely takes longer but in addition reduces a mannequin’s capability to attract statistical correlations between the totally different patches.
The researchers additionally innovated one thing known as an “optical latent house.” Generative AI fashions work, partially, by compressing high-dimensional information into less complicated representations. This forces them to take away much less necessary data and solely retain the bits which might be integral to the enter.
These condensed representations are then saved in a multi-dimensional map of ideas known as a latent house. Fashions use these representations to generate new outputs when given a immediate.
LightGen’s builders replicated this course of totally optically. Of their chip, a full-resolution picture is transmitted by way of an optical encoder made up of a number of metasurfaces—ultra-thin buildings designed to control gentle—after which coupled into an array of optical fibers.
This course of naturally filters out higher-order information, successfully condensing the data into less complicated representations, that are then saved within the fiber array because the optical latent house. One other set of metasurfaces on the different finish of the gadget, which may be switched relying on the duty, then take the output from this latent house and use it to generate high-resolution photographs.
The researchers additionally got here up with a novel coaching strategy. Right here, the chip learns probabilistic representations of coaching information, which makes it attainable to sort out extra complicated duties, like creating novel outputs. This can be a promising improvement. Up to now, most photonic chips have targeted on inference not coaching.
The crew examined their chip on a number of demanding duties, together with the technology of high-resolution photographs of animals, changing photographs into totally different inventive types, and even turning 2D photographs into 3D fashions. Notably, the chip achieved speeds and power efficiencies greater than two orders of magnitude higher than Nvidia’s A100 GPU, one of many firm’s strongest AI chips.
The brand new optical chip isn’t prepared to interrupt out of the lab simply but. It nonetheless depends on cumbersome lasers and spatial gentle modulators to generate enter indicators, and the metasurfaces central to its design are at the moment made with specialised processes relatively these you may discover in normal chip factories.
Nonetheless, with additional improvement, the work suggests optical processors may very well be a quick, energy-efficient approach to energy the cutting-edge of an more and more power-hungry AI business.
