The instructor and the coed
Our strategy revolves round an idea referred to as data distillation, which makes use of a “instructor–scholar” mannequin coaching methodology. We begin with a “instructor” — a big, highly effective, pre-trained generative mannequin that’s an skilled at creating the specified visible impact however is much too sluggish for real-time use. The kind of instructor mannequin varies relying on the purpose. Initially, we used a custom-trained StyleGAN2 mannequin, which was skilled on our curated dataset for real-time facial results. This mannequin could possibly be paired with instruments like StyleCLIP, which allowed it to govern facial options primarily based on textual content descriptions. This offered a robust basis. As our challenge superior, we transitioned to extra subtle generative fashions like Google DeepMind’s Imagen. This strategic shift considerably enhanced our capabilities, enabling higher-fidelity and extra various imagery, better inventive management, and a broader vary of kinds for our on-device generative AI results.
The “scholar” is the mannequin that in the end runs on the person’s gadget. It must be small, quick, and environment friendly. We designed a scholar mannequin with a UNet-based structure, which is great for image-to-image duties. It makes use of a MobileNet spine as its encoder, a design identified for its efficiency on cellular gadgets, paired with a decoder that makes use of MobileNet blocks.
