tokenizer

Documentation

Overview

Package tokenizer This file contains the remove tokenizer controller which is responsible for removing existing tokenizers in existing models

Index

• type AddController
  • func (ic AddController) Run(args []string, customArgs downloadermodel.Args) error
• type RemoveTokenizerController
  • func (ic RemoveTokenizerController) RunTokenizerRemove(args []string) error
• type SelectModelController
  • func (ic SelectModelController) SelectTransformerModel(models model.Models) model.Model
• type UpdateTokenizerController
  • func (ic UpdateTokenizerController) TokenizerUpdateCmd(args []string) error

Types

type AddController

type AddController struct{}

func (AddController) Run

func (ic AddController) Run(args []string, customArgs downloadermodel.Args) error

Run the tokenizer add command

type RemoveTokenizerController

type RemoveTokenizerController struct{}

func (RemoveTokenizerController) RunTokenizerRemove

func (ic RemoveTokenizerController) RunTokenizerRemove(args []string) error

RunTokenizerRemove runs the tokenizer remove command

type SelectModelController

type SelectModelController struct {
}

func (SelectModelController) SelectTransformerModel

func (ic SelectModelController) SelectTransformerModel(models model.Models) model.Model

SelectTransformerModel displays a selector of models from which the user will choose to add to his project

type UpdateTokenizerController

type UpdateTokenizerController struct{}

func (UpdateTokenizerController) TokenizerUpdateCmd

func (ic UpdateTokenizerController) TokenizerUpdateCmd(args []string) error

TokenizerUpdateCmd TokenizerRemoveCmd runs the model update command