kernel

package
v0.0.0-...-dc643fe Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Mar 21, 2023 License: MIT Imports: 24 Imported by: 0

Documentation

Overview

Package kernel handles the lower level communication with the Jupyter client.

It creates the sockets, encoding, validation, etc.

Reference documentation: https://jupyter-client.readthedocs.io/en/latest/messaging.html

Index

Constants

View Source 
const (
	StatusStarting = "starting"
	StatusBusy     = "busy"
	StatusIdle     = "idle"
)
View Source 
const (
	// StreamStdout defines the stream name for standard out on the front-end. It
	// is used in `PublishWriteStream` to specify the stream to write to.
	StreamStdout = "stdout"

	// StreamStderr defines the stream name for standard error on the front-end. It
	// is used in `PublishWriteStream` to specify the stream to write to.
	StreamStderr = "stderr"
)

Variables

View Source 
var (
	// ProtocolVersion defines the Jupyter protocol version.
	// Version 5.2 encodes cursor pos as one position per unicode character.
	// (https://jupyter-client.readthedocs.io/en/stable/messaging.html#cursor-pos-and-unicode-offsets)
	// But still Juypter lab is using UTF-16 encoding for cursor-pos.
	ProtocolVersion = "5.2"
)

Functions

func Install 

func Install(extraArgs []string, force bool) error

Install gonb in users local Jupyter configuration, making it available. It assumes the kernel is implemented by the same binary that is calling this function (os.Args[0]) and that the flag to pass the `connection_file` is `--kernel`.

func JupyterCursorPosToBytePos 

func JupyterCursorPosToBytePos(cellContent string, cursorPosUTF16 int) int

JupyterCursorPosToBytePos converts a `cursor_pos` in a cell, encoded in UTF-16, to a byte position in the cell text, encoded in UTF-8 by Go.

Documentation is a bit confusing, according to this: https://jupyter-client.readthedocs.io/en/stable/messaging.html#cursor-pos-and-unicode-offsets For version >= 5.2 cursor pos should be given as number of unicode runes (?). But in practice Jupyter is sending UTF-16

func JupyterToLinesAndCursor 

func JupyterToLinesAndCursor(cellContent string, cursorPosUTF16 int) (lines []string, cursorLine, cursorCol int)

JupyterToLinesAndCursor takes as input the cell contents and a cursor (as a position in UTF16 points) and returns a slice with the individual lines and a cursor split into cursorLine and cursorCol, bytes based (meaning cursorCol is the position in the line in number of bytes).

func NewJupyterStreamWriter 

func NewJupyterStreamWriter(msg Message, stream string) io.Writer

NewJupyterStreamWriter returns an io.Writer that forwards what is written to the Jupyter client, under the given stream name.

func PipeExecToJupyter 

func PipeExecToJupyter(msg Message, dir, name string, args ...string) error

PipeExecToJupyter executes the given command (command plus arguments) and pipe the output to Jupyter stdout and stderr streams connected to msg.

If dir is not empty, before running the command the current directory is changed to dir.

It returns an error if it failed to execute or created the pipes -- but not if the executed program returns an error for any reason.

func PipeExecToJupyterWithInput 

func PipeExecToJupyterWithInput(msg Message, dir, name string, args ...string) error

PipeExecToJupyterWithInput executes the given command (command plus arguments) and pipes the output and error to Jupyter stdout and stderr streams. It also plumbs the input from Jupyter input, after 500ms the program started (so if programs don't execute quick, and optional input will be made available).

If dir is not empty, before running the command the current directory is changed to dir.

It returns an error if it failed to execute or created the pipes -- but not if the executed program returns an error for any reason.

func PipeExecToJupyterWithPassword 

func PipeExecToJupyterWithPassword(msg Message, dir, name string, args ...string) error

PipeExecToJupyterWithPassword executes the given command (command plus arguments) and pipes the output and error to Jupyter stdout and stderr streams. It also plumbs one input from Jupyter input set as a password (input hidden).

If dir is not empty, before running the command the current directory is changed to dir.

func PollDisplayRequests 

func PollDisplayRequests(msg Message, pipeReader *os.File)

PollDisplayRequests will continuously read for incoming requests for displaying content on the notebook. It expects pipeIn to be closed when the polling is to stop.

func PublishDisplayData 

func PublishDisplayData(msg Message, data Data) error

PublishDisplayData publishes a single image.

func PublishDisplayDataWithHTML 

func PublishDisplayDataWithHTML(msg Message, html string) error

PublishDisplayDataWithHTML is a shortcut to PublishDisplayData for HTML content.

func PublishExecutionError 

func PublishExecutionError(msg Message, err string, trace []string) error

PublishExecutionError publishes a serialized error that was encountered during execution.

func PublishExecutionInput 

func PublishExecutionInput(msg Message, execCount int, code string) error

PublishExecutionInput publishes a status message notifying front-ends of what code is currently being executed.

func PublishExecutionResult 

func PublishExecutionResult(msg Message, execCount int, data Data) error

PublishExecutionResult publishes the result of the `execCount` execution as a string.

func PublishKernelStatus 

func PublishKernelStatus(msg Message, status string) error

PublishKernelStatus publishes a status message notifying front-ends of the state the kernel is in. It supports the states "starting", "busy", and "idle".

func PublishUpdateDisplayData 

func PublishUpdateDisplayData(msg Message, data Data) error

PublishUpdateDisplayData is like PublishDisplayData, but expects `transient.display_id` to be set, and that a previous `display_data` has already created this id.

func PublishWriteStream 

func PublishWriteStream(msg Message, stream string, data string) error

PublishWriteStream prints the data string to a stream on the front-end. This is either `StreamStdout` or `StreamStderr`.

func SendKernelInfo 

func SendKernelInfo(msg Message, version string) error

SendKernelInfo sends a kernel_info_reply message.

func StartNamedPipe 

func StartNamedPipe(msg Message, dir string, doneChan <-chan struct{}) error

StartNamedPipe creates a named pipe in `dir` and starts a listener (on a separate goroutine) that reads the pipe and displays rich content. It also exports environment variable GONB_FIFO announcing the name of the named pipe.

The doneChan is listened to: when it is closed, it will trigger the listener goroutine to close the pipe, remove it and quit.

TODO: make this more secure, maybe with a secret key also passed by the environment.

Types

type CompleteReply 

type CompleteReply struct {
	Status      string   `json:"status"`
	Matches     []string `json:"matches"`
	CursorStart int      `json:"cursor_start"`
	CursorEnd   int      `json:"cursor_end"`
	Metadata    MIMEMap  `json:"metadata"`
}

CompleteReply message sent in reply to a "complete_request": used for auto-complete of the code under a certain position of the cursor.

type ComposedMsg 

type ComposedMsg struct {
	Header       zmqMsgHeader
	ParentHeader zmqMsgHeader
	Metadata     map[string]interface{}
	Content      interface{}
}

ComposedMsg represents an entire message in a high-level structure.

func NewComposed 

func NewComposed(msgType string, parent ComposedMsg) (*ComposedMsg, error)

NewComposed creates a new ComposedMsg to respond to a parent message. This includes setting up its headers.

type Data 

type Data = struct {
	Data      MIMEMap
	Metadata  MIMEMap
	Transient MIMEMap
}

Data is the exact structure returned to Jupyter. It allows to fully specify how a value should be displayed. 

type HelpLink struct {
	Text string `json:"text"`
	URL  string `json:"url"`
}

HelpLink stores data to be displayed in the help menu of the notebook.

type InspectReply 

type InspectReply struct {
	Status   string  `json:"status"`
	Found    bool    `json:"found"`
	Data     MIMEMap `json:"data"`
	Metadata MIMEMap `json:"metadata"`
}

InspectReply message sent in reply to an "inspect_request": used for introspection on the code under a certain position of the cursor.

type InvalidSignatureError 

type InvalidSignatureError struct{}

InvalidSignatureError is returned when the signature on a received message does not validate.

func (*InvalidSignatureError) Error 

func (e *InvalidSignatureError) Error() string

type Kernel 

type Kernel struct {

	// ExecCounter is incremented each time we run user code in the notebook.
	ExecCounter int

	// Interrupted indicates whether shell currently being executed was Interrupted.
	Interrupted atomic.Bool
	// contains filtered or unexported fields
}

func NewKernel 

func NewKernel(connectionFile string) (*Kernel, error)

NewKernel builds and start a kernel. Various goroutines are started to poll for incoming messages. It automatically handles the heartbeat.

Incoming messages should be listened to using Kernel.Stdin, Kernel.Shell and Kernel.Control.

The Kernel can be stopped by calling Kernel.Stop. And one can wait for the kernel clean up of the various goroutines, after it is stopped, by calling kernel.ExitWait.

func (*Kernel) Control 

func (k *Kernel) Control() <-chan Message

Control returns the reading channel where incoming control messages are received.

One should also select for the Kernel.StoppedChan, to check if connection to kernel was closed.

func (*Kernel) ExitWait 

func (k *Kernel) ExitWait()

ExitWait will wait for the kernel to be stopped and all polling goroutines to finish.

func (*Kernel) FromWireMsg 

func (k *Kernel) FromWireMsg(zmqMsg zmq4.Msg) Message

FromWireMsg translates a multipart ZMQ messages received from a socket into a ComposedMsg struct and a slice of return identities. This includes verifying the message signature.

func (*Kernel) HandleInterrupt 

func (k *Kernel) HandleInterrupt()

HandleInterrupt will configure the kernel to listen to the system SIGINT, sent by default by Jupyter to indicate an interruption of the kernel (similar to a Control+C).

So instead of the kernel dying, it will recover, and where appropriate interrupt other sub-processes it may have spawned.

func (*Kernel) IsStopped 

func (k *Kernel) IsStopped() bool

IsStopped returns whether the Kernel has been stopped.

func (*Kernel) Shell 

func (k *Kernel) Shell() <-chan Message

Shell returns the reading channel where incoming shell messages are received.

One should also select for the Kernel.StoppedChan, to check if connection to kernel was closed.

func (*Kernel) Stdin 

func (k *Kernel) Stdin() <-chan Message

Stdin returns the reading channel where incoming stdin messages are received.

One should also select for the Kernel.StoppedChan, to check if connection to kernel was closed.

These are messages received from a console like input, while executing a command. For MessageImpl.PipeExecToJupyterWithInput to work, you need to call MessageImpl.DeliverInput() on these messages.

func (*Kernel) Stop 

func (k *Kernel) Stop()

Stop the Kernel, indicating to all polling processes to quit.

func (*Kernel) StoppedChan 

func (k *Kernel) StoppedChan() <-chan struct{}

StoppedChan returns a channel that can be listened (`select`) to check when the kernel is stopped. The channel will be closed when the Kernel is stopped.

func (*Kernel) ToWireMsg 

func (k *Kernel) ToWireMsg(c *ComposedMsg) ([][]byte, error)

ToWireMsg translates a ComposedMsg into a multipart ZMQ message ready to send, and signs it. This does not add the return identities or the delimiter.

type KernelInfo 

type KernelInfo struct {
	ProtocolVersion       string             `json:"protocol_version"`
	Implementation        string             `json:"implementation"`
	ImplementationVersion string             `json:"implementation_version"`
	LanguageInfo          KernelLanguageInfo `json:"language_info"`
	Banner                string             `json:"banner"`
	HelpLinks             []HelpLink         `json:"help_links"`
}

KernelInfo holds information about the kernel being served, for kernel_info_reply messages.

type KernelLanguageInfo 

type KernelLanguageInfo struct {
	Name              string `json:"name"`
	Version           string `json:"version"`
	MIMEType          string `json:"mimetype"`
	FileExtension     string `json:"file_extension"`
	PygmentsLexer     string `json:"pygments_lexer"`
	CodeMirrorMode    string `json:"codemirror_mode"`
	NBConvertExporter string `json:"nbconvert_exporter"`
}

KernelLanguageInfo holds information about the language that this kernel executes code in.

type MIMEMap 

type MIMEMap = map[string]interface{}

MIMEMap holds data that can be presented in multiple formats. The keys are MIME types and the values are the data formatted with respect to its MIME type. All maps should contain at least a "text/plain" representation with a string value.

func EnsureMIMEMap 

func EnsureMIMEMap(bundle MIMEMap) MIMEMap

type Message 

type Message interface {
	// Error returns the error receiving the message, or nil if no error.
	Error() error

	// Ok returns whether there were no errors receiving the message.
	Ok() bool

	// ComposedMsg that started the current Message -- it's contained by Message.
	ComposedMsg() ComposedMsg

	// Kernel returns reference to the Kernel connections from where this Message was created.
	Kernel() *Kernel

	// Publish creates a new ComposedMsg and sends it back to the return identities over the
	// IOPub channel.
	Publish(msgType string, content interface{}) error

	// PromptInput sends a request for input from the front-end. The text in prompt is shown
	// to the user, and password indicates whether the input is a password (input shouldn't
	// be echoed in terminal).
	//
	// onInputFn is the callback function. It receives the original shell execute
	// message (m) and the message with the incoming input value.
	PromptInput(prompt string, password bool, onInput OnInputFn) error

	// CancelInput will cancel any `input_request` message sent by PromptInput.
	CancelInput() error

	// DeliverInput should be called if a message is received in Stdin channel. It will
	// check if there is any running process listening to it, in which case it is forwarded
	// (usually to the caller of PromptInput).
	// Still the dispatcher has to handle its delivery by calling this function.
	DeliverInput() error

	// Reply creates a new ComposedMsg and sends it back to the return identities over the
	// Shell channel.
	Reply(msgType string, content interface{}) error
}

type MessageImpl 

type MessageImpl struct {
	Composed   ComposedMsg
	Identities [][]byte
	// contains filtered or unexported fields
}

MessageImpl represents a received message or an Error, with its return identities, and a reference to the kernel for communication.

func (*MessageImpl) CancelInput 

func (m *MessageImpl) CancelInput() error

CancelInput will cancel any `input_request` message sent by PromptInput.

func (*MessageImpl) ComposedMsg 

func (m *MessageImpl) ComposedMsg() ComposedMsg

ComposedMsg that started the current Message -- it's contained by Message.

func (*MessageImpl) DeliverInput 

func (m *MessageImpl) DeliverInput() error

DeliverInput should be called if a message is received in Stdin channel. It will check if there is any running process listening to it, in which case it is delivered. Still the user has to handle its delivery.

func (*MessageImpl) Error 

func (m *MessageImpl) Error() error

Error returns the error receiving the message, or nil if no error.

func (*MessageImpl) Kernel 

func (m *MessageImpl) Kernel() *Kernel

Kernel returns reference to the Kernel connections from where this Message was created.

func (*MessageImpl) Ok 

func (m *MessageImpl) Ok() bool

Ok returns whether there were no errors receiving the message.

func (*MessageImpl) PromptInput 

func (m *MessageImpl) PromptInput(prompt string, password bool, onInput OnInputFn) error

PromptInput sends a request for input from the front-end. The text in prompt is shown to the user, and password indicates whether the input is a password (input shouldn't be echoed in terminal).

onInputFn is the callback function. It receives the original shell execute message (m) and the message with the incoming input value.

func (*MessageImpl) Publish 

func (m *MessageImpl) Publish(msgType string, content interface{}) error

Publish creates a new ComposedMsg and sends it back to the return identities over the IOPub channel.

func (*MessageImpl) Reply 

func (m *MessageImpl) Reply(msgType string, content interface{}) error

Reply creates a new ComposedMsg and sends it back to the return identities over the Shell channel.

type MsgOrError 

type MsgOrError struct {
	Msg zmq4.Msg
	Err error
}

MsgOrError describes an incoming message or a communication error. If an error occur consider closing the Kernel with Kernel.Stop().

type OnInputFn 

type OnInputFn func(original, input *MessageImpl) error

OnInputFn is the callback function. It receives the original shell execute message and the message with the incoming input value.

type SocketGroup 

type SocketGroup struct {
	ShellSocket   SyncSocket
	ControlSocket SyncSocket
	StdinSocket   SyncSocket
	IOPubSocket   SyncSocket
	HBSocket      SyncSocket
	Key           []byte
}

SocketGroup holds the sockets needed to communicate with the kernel, and the key for message signing.

type SyncSocket 

type SyncSocket struct {
	Socket zmq4.Socket
	Lock   sync.Mutex
}

SyncSocket wraps a zmq socket with a lock which should be used to control write access.

func (*SyncSocket) RunLocked 

func (s *SyncSocket) RunLocked(fn func(socket zmq4.Socket) error) error

RunLocked locks socket and runs `fn`.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.