spliter

package
v0.0.0-...-04ebf03 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Dec 17, 2024 License: Apache-2.0 Imports: 4 Imported by: 0

Documentation

Index

Constants

View Source
const (
	DefaultChunkSize    = 4000
	DefaultChunkOverlap = 200
)

Variables

This section is empty.

Functions

This section is empty.

Types

type Spliter

type Spliter interface {
	Split(text string) []string
	Merge(elements []vector.Element) []vector.Element
}

func NewTextSpliter

func NewTextSpliter(log logger.Logger, chunkSize int, chunkOverlap int, separator string) Spliter

type TextSpliter

type TextSpliter struct {
	// contains filtered or unexported fields
}

func (*TextSpliter) Merge

func (t *TextSpliter) Merge(elements []vector.Element) []vector.Element

func (*TextSpliter) Split

func (t *TextSpliter) Split(text string) []string

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL