eval

Documentation

Index

• type BasicMatch
  • func (e *BasicMatch) MarshalJSON() ([]byte, error)
• type BasicMatches
  • func ReadHCL(path string) (BasicMatches, error)
  • func (bms BasicMatches) WriteFile(path string) error
• type DataTemplate

Types

type BasicMatch

type BasicMatch struct {
	// The system message to provide context.
	//
	// Example: "You are about to be asked a question. Please answer as concisely as possible."
	SystemContent string `hcl:"system,attr"`

	// The user message containing the instruction.
	//
	// Example: "OpenAI was founded in 20"
	UserContent string `hcl:"user,attr"`

	// The ideal response.
	//
	// Example: "15"
	Ideal string `hcl:"ideal,attr"`
}

BasicMatch contains the data for an OpenAI "evals.elsuite.basic.match:Match" evaluation.

func (*BasicMatch) MarshalJSON

func (e *BasicMatch) MarshalJSON() ([]byte, error)

MarshalJSON is a custom JSON marshaler for the BasicMatch struct.

type BasicMatches

type BasicMatches []*BasicMatch

BasicMatches is a slice of EvalBasicMatch that can be marshaled to JSONL.

func ReadHCL

func ReadHCL(path string) (BasicMatches, error)

ReadHCL reads the HCL file at the given path and returns the BasicMatches slice.

func (BasicMatches) WriteFile

func (bms BasicMatches) WriteFile(path string) error

MarshalJSON is a custom JSON marshaler for the EvalBasicMatches slice, printing each evaluation on a new line.

type DataTemplate

type DataTemplate struct {
	Input []*openai.ChatMessage `json:"input"`
	Ideal string                `json:"ideal"`
}

DataTemplate is a template for the JSON data generated by all evaluation types.