Documentation
¶
Index ¶
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
This section is empty.
Types ¶
type LLM ¶
type LLM struct { Model string // Path to the model.bin Llamacpp string // Path to the llama.cpp folder CudaDevices []int // Array of indices of the Cuda devices that will be used CtxSize int // Size of the prompt context Temp float32 // Temperature TopK int // Top-k sampling RepeatPenalty float32 // Penalize repeat sequence of tokens Ngl int // Number of layers to store in VRAM CpuCores int // Number of physical cpu cores MaxTokens int // Max number of tokens for model response Stop []string // Array of generation-stopping strings InstructionBlock string // Instructions to format the model response }
func (*LLM) BufferPromptModel ¶
BufferPromptModel prompts the model expecting the real time output, allowing you to use its response as it's being generated. It sends the LLM response tokens as strings to the provided channel.
func (*LLM) GetLLMProps ¶
func (llm *LLM) GetLLMProps()
GetLLMProps reads the properties currently set to the LLM struct.
func (*LLM) PromptModel ¶
PromptModel method orderly prompts the LLM with the provided prompts in the array, engaging in a sort of conversation. It returns an array with the respones of the LLM, each response matching with the index of its prompt.
Click to show internal directories.
Click to hide internal directories.