While building AI research projects here at Aliases, we constantly explore and experiment with various models to find the best tools for our projects. Over time, we've come across several AI models on Replicate.com that stand out from the crowd. In this Aliases Dev Diary entry, we'll dive into our top 5 favorite models, discussing what makes each one unique and indispensable for our development process.

What's Replicate.com?

Not familiar with Replicate? It's an amazing AI platform where developers can access and deploy AI models with ease. Whether you'd like to use text-to-image, image-to-image, generate video, generate audio, or enhance existing media, Replicate offers a diverse range of models that can significantly enhance your projects. It’s also super affordable for testing many models. We’ve used these models while building AIdentity prototypes for Next.js & iOS (Swift):

There are libraries for Node.js, Python, Google Colab, Next.js, SwiftUI, Discord Bots, and Elixir, with API calls as simple as this:

import Replicate from "replicate";
const replicate = new Replicate();
const input = {
prompt: "Woman, detailed face, sci-fi RGB glowing, cyberpunk",
light_source: "Left Light",
subject_image: "https://replicate.delivery/pbxt/KtCKrs9sxPF3HciwoWL0TTVM9Nde7ySDWpO9S2flTiyi9Pp3/i3.png" };

Our Top 5 Favorite Models

1. fofr / live-portrait: Portrait animation using a driving video source.

   This model uses a reference image and a reference green screen video to animate faces and generate MP4 videos. By leveraging the driving video source, the model maps the facial movements and expressions from the green screen video onto the reference image, creating a dynamic and realistic animation. The resulting output is an MP4 video where the subject in the reference image appears to come to life, mimicking the movements and expressions from the driving video.
   Here's a fun example:

   
   
   

2. zsxkib / pulid: PuLID: Pure and Lightning ID Customization via Contrastive Alignment.

   This model is by far the most versatile and fast model for generating virtually any style of image from an existing photo of a person with a simple prompt. It allows users to transform a single photo into a wide variety of artistic styles and contexts, providing unparalleled flexibility and creativity in image generation. Whether you need a professional headshot, a cartoon version, or a fantasy-themed portrait, this model can deliver stunning results in no time.
   Here's examples showing off the flexibility:

   

   
   
   

3. zsxkib / ic-light: Prompts to auto-magically relights your images.

   This model does an incredible job taking an existing portrait and completely relighting the lighting source. It can also change the background scenery and enhance the quality of the photo based on the prompt. The ability to adjust lighting and background elements allows for greater creative control and results in stunning, professional-quality images.
   Here's a fun example taking the original creation on the left changing the light type and direction:
   
   

   

   
   

4. batouresearch / high-resolution-controlnet-tile: Open-source implementation of an efficient ControlNet 1.1 tile for high-quality upscales.

   This model can upscale an existing photo to a much higher resolution, providing exceptional clarity and detail. With this improved resolution, you gain the flexibility to tweak various aspects of the image, such as enhancing resemblance to the original subject and applying HDR (High Dynamic Range) effects. This allows for professional-level adjustments that can bring out the best in your photos, making them look more vibrant and true-to-life.
   Here's a quick before & after:

   

   
   
   

5. stability-ai / stable-diffusion-3: A text-to-image model with greatly improved performance in image quality, typography, complex prompt understanding, and resource-efficiency.

   No surprise here, but Stable Diffusion 3 (SD3) is extremely versatile, offering high-quality output for text-to-image generation. Whether you're starting from scratch with just a text prompt or using an input image, SD3 delivers impressive results. One of the standout features is its improved handling of text within images, producing clear and readable typography that was often a challenge in earlier models. This makes SD3 a go-to choice for projects that require both visual and textual elements to be seamlessly integrated.

Unlocking Creative AI Workflows

The variation of models like this combined with the affordable, easy access to APIs and SDKs from Replicate make Hybrid Centered Design exploration at Aliases extremely fun and easy by removing barriers and letting us focus on creativity to builder our own AI workflows. Here’s a fun little exploration of using multiple models in a UX flow in our prototype app: