bindstablediff

README

Bindstablediff

stable-diffusion.cpp bindings for golang for running stable diffusion on CPU

Goal of this library is provide simple interface for running stable diffusion process on golang programs in situations where no GPU is not available (like cloud service).

This library is under development at the moment. Now this is only thin wrapper around C++ code. Goal is to allow user tap into diffusion process and expand existing algorithm. (zooming, custom tokenizer weighting etc.. tricks)

Converting models

Original stable-diffusion.cpp site have some guidance how to do that.

Basic use case is to just use float16 and convert .ckpt to ggml binary by conversion script

	cd convert
    python convert.py sd-v1-4.ckpt --out_type f16

Using library

Basic idea is to include library (and do go mod tidy)

import "github.com/hjkoskel/bindstablediff"

Then create StableDiffusionModel with function InitStableDiffusion for loading model

func InitStableDiffusion(fname string, nThreads int, schedule EnumSchedule) (StableDiffusionModel, error) {

Then collect parameters to struct and call txt2img

par := bindstablediff.TextGenPars{
		Prompt:         "cute dog",
		NegativePrompt: "",
		CfgScale:       7,
		Width:          512,
		Height:         512,
		SampleMethod:   bindstablediff.HEUN,
		SampleSteps:    *pSteps,
		Seed:           -1}

resultImg, errGen := engine.Txt2Img(par)

Example dogandcat

Directory ./cmd/dogandcat have minimal example how to use this library.

Example usage stbdiff

directory ./cmd/stbdiff contains simple commandline utility for batch running.

There are two basic ways to use. Directly from command line or by using json file containing list of jobs. Its own README.md have more details

Documentation

Index

• func SavePng(fname string, img image.Image) error
• type EnumRNGType
• type EnumSDLogLevel
• type EnumSampleMethod
  • func ParseSampleMethod(s string) (EnumSampleMethod, error)
• type EnumSchedule
  • func ParseSchedule(s string) (EnumSchedule, error)
• type StableDiffusionModel
  • func InitStableDiffusion(fname string, nThreads int, schedule EnumSchedule) (StableDiffusionModel, error)
  • func (p *StableDiffusionModel) Img2Img(startImage image.Image, parameters TextGenPars) (image.Image, error)
  • func (p *StableDiffusionModel) Txt2Img(parameters TextGenPars) (image.Image, error)
• type TextGenPars

Functions

func SavePng

func SavePng(fname string, img image.Image) error

Types

type EnumRNGType

type EnumRNGType int

const (
	STD_DEFAULT_RNG EnumRNGType = 0
	CUDA_RNG        EnumRNGType = 1
)

type EnumSDLogLevel

type EnumSDLogLevel int

const (
	DEBUG EnumSDLogLevel = 0
	INFO  EnumSDLogLevel = 1
	WARN  EnumSDLogLevel = 2
	ERROR EnumSDLogLevel = 3
)

type EnumSampleMethod

type EnumSampleMethod int

const (
	EULER_A          EnumSampleMethod = 0
	EULER            EnumSampleMethod = 1
	HEUN             EnumSampleMethod = 2
	DPM2             EnumSampleMethod = 3
	DPMPP2S_A        EnumSampleMethod = 4
	DPMPP2M          EnumSampleMethod = 5
	DPMPP2Mv2        EnumSampleMethod = 6
	N_SAMPLE_METHODS EnumSampleMethod = 7
)

func ParseSampleMethod

func ParseSampleMethod(s string) (EnumSampleMethod, error)

type EnumSchedule

type EnumSchedule int

const (
	DEFAULT     EnumSchedule = 0
	DISCRETE    EnumSchedule = 1
	KARRAS      EnumSchedule = 2
	N_SCHEDULES EnumSchedule = 3
)

func ParseSchedule

func ParseSchedule(s string) (EnumSchedule, error)

type StableDiffusionModel

type StableDiffusionModel struct {
	// contains filtered or unexported fields
}

func InitStableDiffusion

func InitStableDiffusion(fname string, nThreads int, schedule EnumSchedule) (StableDiffusionModel, error)

func (*StableDiffusionModel) Img2Img

func (p *StableDiffusionModel) Img2Img(startImage image.Image, parameters TextGenPars) (image.Image, error)

Img2Img, not yet ready

func (*StableDiffusionModel) Txt2Img

func (p *StableDiffusionModel) Txt2Img(parameters TextGenPars) (image.Image, error)

type TextGenPars

type TextGenPars struct {
	Prompt         string
	NegativePrompt string
	CfgScale       float32 //7 default
	Width          int
	Height         int
	SampleMethod   EnumSampleMethod
	SampleSteps    int
	Strength       float32 //needed for img2img
	Seed           int64
}

Lets have parameters as struct.. so it is easier to store to exif etc...