Change your drawing (ss one svg) to photographic image with Stable Diffusion

This is a test data a-powerbook-100.svg that I drew with ss one:

This image was generated with Stable Diffusion:

Step1

First step, change the a-powerbook-100.svg to png data like this:

This image was resized as 320x320.
I use this svg a-powerbook-100_768x768.png for Stable Diffusion.

Step2

And then, do stable diffusion magic with this prompt: A powerbook 100 on the wood table

Another one, this prompt is A powerbook 100, plain background, by color pencil drawing art by Leonardo DaVinci:

It's awsome.

Code

A python code to convert:

import torch
import requests
from PIL import Image, ImageOps
from io import BytesIO
from diffusers import StableDiffusionControlNetPipeline, ControlNetModel

def load_image(image_file_path):
    with open(image_file_path, "rb") as f:
        image = Image.open(f).convert("RGB")
    return image


device = "cuda"

ct_model_id = "lllyasviel/sd-controlnet-scribble"
sd_model_id = "runwayml/stable-diffusion-v1-5"
seed = 42
input_image_file  = "a-powerbook-100.png"
output_image_file = "a-powerbook-100_sd.png"
prompt = "A powerbook 100 on the wood table"

controlnet = ControlNetModel.from_pretrained(
    ct_model_id,
    torch_dtype=torch.float16)

pipe = StableDiffusionControlNetPipeline.from_pretrained(
    sd_model_id,
    controlnet=controlnet,
    torch_dtype=torch.float16).to(device)


init_image = load_image(input_image_file)
init_image_inverted = ImageOps.invert(init_image)
#init_image_inverted.thumbnail((768, 768))

generator = torch.Generator(device=device).manual_seed(seed)
image = pipe(
    prompt=prompt,
    image=init_image_inverted,
    guidance_scale=7.5,
    generator=generator).images[0]

image.save(output_image_file)