October 15, 2024
1 Solar System Way, Planet Earth, USA
Gaming

Edify helps developers create custom models

Editor's note: This post is part of the AI Decoded Serieswhich demystifies AI by making the technology more accessible and showcases new hardware, software, tools, and accelerations for RTX PC users.

Content creators, whether producing language, 2D images, 3D models, or videos, are providing the creative community with tools that enable visions to come to life more quickly.

To help developers create these new generative AI tools, NVIDIA has created NVIDIA AI FoundryHelps businesses train generative AI models with their own authoritative data using NVIDIA Edifya multimodal AI architecture that can use simple text prompts to generate images, videos, 3D assets, 360-degree high dynamic range images, and physically-based rendering (PBR) materials. With AI Foundry, companies can train custom AI models to generate any of these assets.

Key elements of Edify include its ability to generate multiple types of content, its superior training efficiency, which allows it to produce high-quality content while training on fewer images, and its ability to fine-tune models to match style or learn characters or objects.

One of the best examples of services built with NVIDIA AI Foundry and Edify is Generative artificial intelligence from Getty Imagesa commercially secure generative photography service. The combination of AI Foundry and Edify allows users to control their training datasets, so they can build models that fit their needs.

To avoid copyright issues, Getty Images used Edify to train the service with its own licensed content, ensuring that there are no famous people or products in the dataset. The company also shares a portion of the profits with contributors, creating a new revenue stream for creators who contribute to the model.

Asset generation with Edify

Edify can be trained to generate a variety of image types including images, 3D assets, and 360-degree HDRi environment maps.

Edify Image can generate four high-quality 1K images in about six seconds, doubling the performance of the previous model. Images can also be upscaled to 4K with a generative upscaler that adds additional detail.

4K image generation trained on NVIDIA Edify using commercially secure creative libraries.

Images are highly controllable thanks to advanced cue adherence, camera controls to specify focal length or depth of field, and ControlNets to guide generation. ControlNets include Sketch, which allows users to provide a sketch to follow or copy the composition of an image, and Depth, to copy the composition of an image.

Images can also be edited with Edify Image. InPaint allows users to add or modify content in an image. Replace (a strict InPaint) can change details like clothing. And OutPaint can stretch an image to match different aspect ratios. And all of this is made simpler with Segment, a feature that can mask objects with just a text prompt.

Edify can also create artist-ready 3D meshes. Meshes come with clean quad-based topology, up to 4K PBR materials, and automatic UV mapping for easier texture editing. A quick preview mode provides results in as little as 10 seconds, which can then be converted into a full 3D mesh.

Meshes are perfect for prototyping scenes, generating background objects for set decoration, or as a starting point for 3D sculpting.

Edify 360 HDRi generates ambient maps of natural landscapes that can be used to light a scene, generate reflections, and even serve as a background. The model can generate up to 16K HDRi images from text or image cues. With the desired backplane in hand, users can create a matching custom HDRi instead of spending hours searching for one.

High dynamic range, 360-degree panoramas from text prompts.

Edify's multimodal capability is unique and allows for advanced workflows that combine different asset types. Used in conjunction with an agent, for example, Edify allows users to prototype an entire scene in a couple of minutes with a simple text message, as in the NVIDIA Research SIGGRAPH Demo which showcased the assisted 3D world creation capabilities of models powered by NVIDIA Edify and the NVIDIA Omniverse platform.

Another use case is combining Edify 3D and 360 HDRi with Image to give users full control over image generation. By generating the scene in 3D, artists can move objects and frame the desired shot, then use Edify Image to turn the prototype into a photorealistic image.

Generative artificial intelligence from Getty Images

Getty Images is one of the largest content service providers and suppliers of creative imagery, editorial photography, video and music, and is one of the first places people turn to discover, purchase and share impactful visual content from the world's best photographers and videographers.

Getty Images used NVIDIA AI Foundry to train an NVIDIA Edify image model to power its generative AI service. Available via Generative artificial intelligence from Getty Images for companies and iStock's generative artificial intelligence For small businesses and hobbyist creators, the service allows users to generate and modify images using models powered by NVIDIA Edify.

Generative AI from Getty Images (or iStock) offers a variety of licensed content.

Getty Images and iStock have recently upgraded to the latest version of Edify Image, allowing for faster renders and greater adherence and exposure to camera controls.

Updated camera controls in Getty Images' Generative AI.

Users can now also use generative AI tools on pre-designed creative content, allowing them to edit and modify iStock's library of visual assets to quickly iterate and refine content. Those same capabilities will be available soon on Gettyimages.com.

Try Getty Images' generative AI on ai.nvidia.com.

Generative AI is transforming gaming, video conferencing, and interactive experiences of all kinds. Find out what's new and coming by subscribing to the AI Decoded Newsletter.

    Leave feedback about this

    • Quality
    • Price
    • Service

    PROS

    +
    Add Field

    CONS

    +
    Add Field
    Choose Image
    Choose Video
    X