Documentation ¶
Overview ¶
Package kernel handles the lower level communication with the Jupyter client.
It creates the sockets, encoding, validation, etc.
Reference documentation: https://jupyter-client.readthedocs.io/en/latest/messaging.html
Index ¶
- Constants
- Variables
- func Install(extraArgs []string, forceDeps, forceCopy bool) error
- func JupyterCursorPosToBytePos(cellContent string, cursorPosUTF16 int) int
- func JupyterToLinesAndCursor(cellContent string, cursorPosUTF16 int) (lines []string, cursorLine, cursorCol int)
- func LogDisplayData(data MIMEMap)
- func NewJupyterStreamWriter(msg Message, stream string) io.Writer
- func PublishData(msg Message, data Data) error
- func PublishDisplayData(msg Message, data Data) error
- func PublishExecuteInput(msg Message, code string) error
- func PublishExecuteResult(msg Message, data Data) error
- func PublishExecutionError(msg Message, err string, trace []string, name string) error
- func PublishHtml(msg Message, html string) error
- func PublishJavascript(msg Message, js string) error
- func PublishKernelStatus(msg Message, status string) error
- func PublishMarkdown(msg Message, markdown string) error
- func PublishUpdateDisplayData(msg Message, data Data) error
- func PublishWriteStream(msg Message, stream string, data string) error
- func SendKernelInfo(msg Message, version string) error
- type CommClose
- type CommInfoReply
- type CommMsg
- type CommOpen
- type CompleteReply
- type ComposedMsg
- type Data
- type HelpLink
- type InspectReply
- type InterruptFn
- type InvalidSignatureError
- type Kernel
- func (k *Kernel) CallInterruptSubscribers()
- func (k *Kernel) Control() <-chan Message
- func (k *Kernel) ExitWait()
- func (k *Kernel) FromWireMsg(zmqMsg zmq4.Msg) Message
- func (k *Kernel) HandleInterrupt()
- func (k *Kernel) IsStopped() bool
- func (k *Kernel) Shell() <-chan Message
- func (k *Kernel) Stdin() <-chan Message
- func (k *Kernel) Stop()
- func (k *Kernel) StoppedChan() <-chan struct{}
- func (k *Kernel) SubscribeInterrupt(fn InterruptFn) SubscriptionId
- func (k *Kernel) ToWireMsg(c *ComposedMsg) ([][]byte, error)
- func (k *Kernel) UnsubscribeInterrupt(id SubscriptionId)
- type KernelInfo
- type KernelLanguageInfo
- type MIMEMap
- type Message
- type MessageImpl
- func (m *MessageImpl) CancelInput() error
- func (m *MessageImpl) ComposedMsg() ComposedMsg
- func (m *MessageImpl) DeliverInput() error
- func (m *MessageImpl) Error() error
- func (m *MessageImpl) Kernel() *Kernel
- func (m *MessageImpl) Ok() bool
- func (m *MessageImpl) PromptInput(prompt string, password bool, onInput OnInputFn) error
- func (m *MessageImpl) Publish(msgType string, content interface{}) error
- func (m *MessageImpl) Reply(msgType string, content interface{}) error
- type MsgOrError
- type OnInputFn
- type SocketGroup
- type SubscriptionId
- type SyncSocket
Constants ¶
const ( StatusStarting = "starting" StatusBusy = "busy" StatusIdle = "idle" )
const ( // StreamStdout defines the stream name for standard out on the front-end. It // is used in `PublishWriteStream` to specify the stream to write to. StreamStdout = "stdout" // StreamStderr defines the stream name for standard error on the front-end. It // is used in `PublishWriteStream` to specify the stream to write to. StreamStderr = "stderr" )
const JupyterDataDirEnv = "JUPYTER_DATA_DIR"
JupyterDataDirEnv is the name of the environment variable pointing to the data files for Jupyter, including kernel configuration.
Variables ¶
CaptureSignals list all signals to be captured: except of `os.Interrupt` (Control+C), all others trigger a clean exit of GoNB kernel.
Notice `os.Interrupt` is used by Jupyter to signal to interrupt the execution of the current cell.
var ( // ProtocolVersion defines the Jupyter protocol version. See differences here: // https://jupyter-client.readthedocs.io/en/stable/messaging.html#changelog // // Version >= 5.2 encodes cursor pos as one position per unicode character. // (https://jupyter-client.readthedocs.io/en/stable/messaging.html#cursor-pos-and-unicode-offsets) // But still Jupyter-Lab is using UTF-16 encoding for cursor-pos. ProtocolVersion = "5.4" )
Functions ¶
func Install ¶
Install gonb in users local Jupyter configuration, making it available. It assumes the kernel is implemented by the same binary calling this function (os.Args[0]) and that the flag to pass the `connection_file` is `--kernel`.
If the binary is under `/tmp` (or if forceCopy is true), it is copied to the location of the kernel configuration, and that copy is used.
If forceDeps is true, installation will succeed even with missing dependencies.
Documentation: https://jupyter-client.readthedocs.io/en/latest/kernels.html#kernelspecs
func JupyterCursorPosToBytePos ¶
JupyterCursorPosToBytePos converts a `cursor_pos` in a cell, encoded in UTF-16, to a byte position in the cell text, encoded in UTF-8 by Go.
Documentation is a bit confusing, according to this: https://jupyter-client.readthedocs.io/en/stable/messaging.html#cursor-pos-and-unicode-offsets For version >= 5.2 cursor pos should be given as number of unicode runes (?). But in practice Jupyter is sending UTF-16
func JupyterToLinesAndCursor ¶
func JupyterToLinesAndCursor(cellContent string, cursorPosUTF16 int) (lines []string, cursorLine, cursorCol int)
JupyterToLinesAndCursor takes as input the cell contents and a cursor (as a position in UTF16 points) and returns a slice with the individual lines and a cursor split into cursorLine and cursorCol, bytes based (meaning cursorCol is the position in the line in number of bytes).
func LogDisplayData ¶
func LogDisplayData(data MIMEMap)
LogDisplayData prints out the display data using `klog`.
func NewJupyterStreamWriter ¶
NewJupyterStreamWriter returns an io.Writer that forwards what is written to the Jupyter client, under the given stream name.
func PublishData ¶
PublishData is a wrapper to either PublishExecuteResult or PublishDisplayData, depending if the parent message was "execute_result" or something else.
func PublishDisplayData ¶
PublishDisplayData publishes data of arbitrary data-types.
func PublishExecuteInput ¶
PublishExecuteInput publishes a status message notifying front-ends of what code is currently being executed.
func PublishExecuteResult ¶
PublishExecuteResult publishes using "execute_result" method. Very similar to PublishDisplayData, but in response to an "execute_request" message.
func PublishExecutionError ¶
PublishExecutionError publishes a serialized error that was encountered during execution.
func PublishHtml ¶
PublishHtml is a shortcut to PublishData for HTML content.
func PublishJavascript ¶
PublishJavascript is a shortcut to PublishData for javascript content to be executed.
Note: `text/javascript` mime-type (protocol.MIMETextJavascript) is not supported by VSCode, it's displayed as text. So using this won't work in VSCode.
func PublishKernelStatus ¶
PublishKernelStatus publishes a status message notifying front-ends of the state the kernel is in. It supports the states "starting", "busy", and "idle".
func PublishMarkdown ¶
PublishMarkdown is a shortcut to PublishData for markdown content.
func PublishUpdateDisplayData ¶
PublishUpdateDisplayData is like PublishDisplayData, but expects `transient.display_id` to be set. If the "display_id" is new, it will publish the data with the given "display_id" as usual, creating a new block ("<div>"). If it has already been seen, instead it updates the previously create display data on that "display_id".
func PublishWriteStream ¶
PublishWriteStream prints the data string to a stream on the front-end. This is either `StreamStdout` or `StreamStderr`.
func SendKernelInfo ¶
SendKernelInfo sends a kernel_info_reply message.
Types ¶
type CommClose ¶
type CommClose struct { // CommId is a UUID that identify this "comm" channel. CommId string `json:"comm_id"` // Data key is always a dict and can be any extra JSON information used // in initialization of the comm. Data map[string]any `json:"data"` }
CommClose is part of the "custom messages" protocol. The corresponding message type is "comm_close". https://jupyter-client.readthedocs.io/en/latest/messaging.html#custom-messages
type CommInfoReply ¶
type CommInfoReply struct { // Status should be set to 'ok' if the request succeeded or 'error', // or with error information as in all other replies. Status string `json:"status"` // Comms is a dictionary of comm_id (uuids) to a dictionary of fields. // The only field documented is "target_name". Comms map[string]map[string]string `json:"comms"` }
CommInfoReply message sent in reply to a "comm_info_request". https://jupyter-client.readthedocs.io/en/latest/messaging.html#comm-info
type CommMsg ¶
type CommMsg struct { // CommId is a UUID that identify this "comm" channel. CommId string `json:"comm_id"` // Data key is always a dict and can be any extra JSON information used // in initialization of the comm. Data map[string]any `json:"data"` }
CommMsg is part of the "custom messages" protocol. The corresponding message type is "comm_msg". https://jupyter-client.readthedocs.io/en/latest/messaging.html#custom-messages
type CommOpen ¶
type CommOpen struct { // CommId is a UUID that identify this new "comm" channel being opened (?). // Documentation is not clear how these should be used. CommId string `json:"comm_id"` // TargetName: documentation is not clear what it is, but it says: // - If the target_name key is not found on the receiving side, // then it should immediately reply with a comm_close message to avoid an // inconsistent state. TargetName string `json:"target_name"` // Data key is always a dict and can be any extra JSON information used // in initialization of the comm. Data map[string]any `json:"data"` }
CommOpen is part of the "custom messages" protocol. The corresponding message type is "comm_open". https://jupyter-client.readthedocs.io/en/latest/messaging.html#custom-messages
type CompleteReply ¶
type CompleteReply struct { Status string `json:"status"` Matches []string `json:"matches"` CursorStart int `json:"cursor_start"` CursorEnd int `json:"cursor_end"` Metadata MIMEMap `json:"metadata"` }
CompleteReply message sent in reply to a "complete_request": used for auto-complete of the code under a certain position of the cursor.
type ComposedMsg ¶
type ComposedMsg struct { Header zmqMsgHeader ParentHeader zmqMsgHeader Metadata map[string]any Content any }
ComposedMsg represents an entire message in a high-level structure.
func NewComposed ¶
func NewComposed(msgType string, parent ComposedMsg) (*ComposedMsg, error)
NewComposed creates a new ComposedMsg to respond to a parent message. This includes setting up its headers.
type Data ¶
Data is the exact structure returned to Jupyter. It allows to fully specify how a value should be displayed.
type InspectReply ¶
type InspectReply struct { Status string `json:"status"` Found bool `json:"found"` Data MIMEMap `json:"data"` Metadata MIMEMap `json:"metadata"` }
InspectReply message sent in reply to an "inspect_request": used for introspection on the code under a certain position of the cursor.
type InterruptFn ¶ added in v0.10.0
type InterruptFn func(id SubscriptionId)
InterruptFn is called on its own goroutine.
type InvalidSignatureError ¶
type InvalidSignatureError struct{}
InvalidSignatureError is returned when the signature on a received message does not validate.
func (*InvalidSignatureError) Error ¶
func (e *InvalidSignatureError) Error() string
type Kernel ¶
type Kernel struct { // ExecCounter is incremented each time we run user code in the notebook. ExecCounter int // Interrupted indicates whether cell/shell currently being executed was Interrupted. Interrupted atomic.Bool // JupyterKernelId is a unique id associated to the kernel by Jupyter. // It's different from the id created by goexec.State to identify the temporary Go // code compilation. // The use so far, besides to match Jupyter logs, is also to create URL to websocket port // for the session (see `dispatcher/comm.go`). JupyterKernelId string // KnownBlockIds are display data blocks with a "display_id" that have already been created, and // hence should be updated (instead of created anew) in calls to PublishUpdate KnownBlockIds common.Set[string] // contains filtered or unexported fields }
func New ¶
New builds and start a kernel. Various goroutines are started to poll for incoming messages. It automatically handles the heartbeat.
Incoming messages should be listened to using Kernel.Stdin, Kernel.Shell and Kernel.Control.
The Kernel can be stopped by calling Kernel.Stop. And one can wait for the kernel clean up of the various goroutines, after it is stopped, by calling kernel.ExitWait.
The `connectionFile` is created by Jupyter with information on which ports to connect to each socket. The path itself is also used to extract the JupyterKernelId associated with this instance of the kernel.
func (*Kernel) CallInterruptSubscribers ¶ added in v0.10.1
func (k *Kernel) CallInterruptSubscribers()
CallInterruptSubscribers in a separate goroutine each. Meant to be called when JupyterServer sends a kernel interrupt (either a SIGINT, or a `interrupt_request` message to interrupt).
func (*Kernel) Control ¶
Control returns the reading channel where incoming control messages are received.
One should also select for the Kernel.StoppedChan, to check if connection to kernel was closed.
func (*Kernel) ExitWait ¶
func (k *Kernel) ExitWait()
ExitWait will wait for the kernel to be stopped and all polling goroutines to finish.
func (*Kernel) FromWireMsg ¶
FromWireMsg translates a multipart ZMQ messages received from a socket into a ComposedMsg struct and a slice of return identities. This includes verifying the message signature.
This "Wire Protocol" of the messages is described here: https://jupyter-client.readthedocs.io/en/latest/messaging.html#the-wire-protocol
func (*Kernel) HandleInterrupt ¶
func (k *Kernel) HandleInterrupt()
HandleInterrupt will configure the kernel to listen to the system SIGINT, sent by default by Jupyter to indicate an interruption of the kernel (similar to a Control+C).
So instead of the kernel dying, it will recover, and where appropriate interrupt other subprocesses it may have spawned.
func (*Kernel) Shell ¶
Shell returns the reading channel where incoming shell messages are received.
One should also select for the Kernel.StoppedChan, to check if connection to kernel was closed.
func (*Kernel) Stdin ¶
Stdin returns the reading channel where incoming stdin messages are received.
One should also select for the Kernel.StoppedChan, to check if connection to kernel was closed.
These are messages received from a console like input, while executing a command. For MessageImpl.PipeExecToJupyterWithInput to work, you need to call MessageImpl.DeliverInput() on these messages.
func (*Kernel) Stop ¶
func (k *Kernel) Stop()
Stop the Kernel, indicating to all polling processes to quit.
func (*Kernel) StoppedChan ¶
func (k *Kernel) StoppedChan() <-chan struct{}
StoppedChan returns a channel that can be listened (`select`) to check when the kernel is stopped. The channel will be closed when the Kernel is stopped.
func (*Kernel) SubscribeInterrupt ¶ added in v0.10.0
func (k *Kernel) SubscribeInterrupt(fn InterruptFn) SubscriptionId
SubscribeInterrupt registers `fn` to be called if any interruptions occur. It returns a SubscriptionId that needs to be used to unsubscribe to it later.
func (*Kernel) ToWireMsg ¶
func (k *Kernel) ToWireMsg(c *ComposedMsg) ([][]byte, error)
ToWireMsg translates a ComposedMsg into a multipart ZMQ message ready to send, and signs it. This does not add the return identities or the delimiter.
func (*Kernel) UnsubscribeInterrupt ¶ added in v0.10.0
func (k *Kernel) UnsubscribeInterrupt(id SubscriptionId)
UnsubscribeInterrupt stops being called back in case of interruptions. It takes the `id` returned by [SubscribeInterrupt].
type KernelInfo ¶
type KernelInfo struct { ProtocolVersion string `json:"protocol_version"` Implementation string `json:"implementation"` ImplementationVersion string `json:"implementation_version"` LanguageInfo KernelLanguageInfo `json:"language_info"` Banner string `json:"banner"` HelpLinks []HelpLink `json:"help_links"` }
KernelInfo holds information about the kernel being served, for kernel_info_reply messages.
type KernelLanguageInfo ¶
type KernelLanguageInfo struct { Name string `json:"name"` Version string `json:"version"` MIMEType string `json:"mimetype"` FileExtension string `json:"file_extension"` PygmentsLexer string `json:"pygments_lexer"` CodeMirrorMode string `json:"codemirror_mode"` NBConvertExporter string `json:"nbconvert_exporter"` }
KernelLanguageInfo holds information about the language that this kernel executes code in.
type MIMEMap ¶
MIMEMap holds data that can be presented in multiple formats. The keys are MIME types and the values are the data formatted with respect to its MIME type. All maps should contain at least a "text/plain" representation with a string value.
func EnsureMIMEMap ¶
type Message ¶
type Message interface { // Error returns the error receiving the message, or nil if no error. Error() error // Ok returns whether there were no errors receiving the message. Ok() bool // ComposedMsg that started the current Message -- it's contained by Message. ComposedMsg() ComposedMsg // Kernel returns reference to the Kernel connections from where this Message was created. Kernel() *Kernel // Publish creates a new ComposedMsg and sends it back to the return identities over the // IOPub channel. Publish(msgType string, content interface{}) error // PromptInput sends a request for input from the front-end. The text in prompt is shown // to the user, and password indicates whether the input is a password (input shouldn't // be echoed in terminal). // // onInputFn is the callback function. It receives the original shell execute // message (m) and the message with the incoming input value. PromptInput(prompt string, password bool, onInput OnInputFn) error // CancelInput will cancel any `input_request` message sent by PromptInput. CancelInput() error // DeliverInput should be called if a message is received in Stdin channel. It will // check if there is any running process listening to it, in which case it is forwarded // (usually to the caller of PromptInput). // Still the dispatcher has to handle its delivery by calling this function. DeliverInput() error // Reply creates a new ComposedMsg and sends it back to the return identities over the // Shell channel. Reply(msgType string, content interface{}) error }
Message is the interface of a received message. It includes an identifier that allows publishing back results to the identifier.
type MessageImpl ¶
type MessageImpl struct { Composed ComposedMsg Identities [][]byte // contains filtered or unexported fields }
MessageImpl represents a received message or an Error, with its return identities, and a reference to the kernel for communication.
func (*MessageImpl) CancelInput ¶
func (m *MessageImpl) CancelInput() error
CancelInput will cancel any `input_request` message sent by PromptInput.
func (*MessageImpl) ComposedMsg ¶
func (m *MessageImpl) ComposedMsg() ComposedMsg
ComposedMsg that started the current Message -- it's contained by Message.
func (*MessageImpl) DeliverInput ¶
func (m *MessageImpl) DeliverInput() error
DeliverInput should be called if a message is received in Stdin channel. It will check if there is any running process listening to it, in which case it is delivered. Still the user has to handle its delivery.
func (*MessageImpl) Error ¶
func (m *MessageImpl) Error() error
Error returns the error receiving the message, or nil if no error.
func (*MessageImpl) Kernel ¶
func (m *MessageImpl) Kernel() *Kernel
Kernel returns reference to the Kernel connections from where this Message was created.
func (*MessageImpl) Ok ¶
func (m *MessageImpl) Ok() bool
Ok returns whether there were no errors receiving the message.
func (*MessageImpl) PromptInput ¶
func (m *MessageImpl) PromptInput(prompt string, password bool, onInput OnInputFn) error
PromptInput sends a request for input from the front-end. The text in prompt is shown to the user, and password indicates whether the input is a password (input shouldn't be echoed in terminal).
onInputFn is the callback function. It receives the original shell execute message (m) and the message with the incoming input value.
func (*MessageImpl) Publish ¶
func (m *MessageImpl) Publish(msgType string, content interface{}) error
Publish creates a new ComposedMsg and sends it back to the return identities over the IOPub channel.
func (*MessageImpl) Reply ¶
func (m *MessageImpl) Reply(msgType string, content interface{}) error
Reply creates a new ComposedMsg and sends it back to the return identities over the Shell channel.
type MsgOrError ¶
MsgOrError describes an incoming message or a communication error. If an error occur consider closing the Kernel with Kernel.Stop().
type OnInputFn ¶
type OnInputFn func(original, input *MessageImpl) error
OnInputFn is the callback function. It receives the original shell execute message and the message with the incoming input value.
type SocketGroup ¶
type SocketGroup struct { ShellSocket SyncSocket ControlSocket SyncSocket StdinSocket SyncSocket IOPubSocket SyncSocket HBSocket SyncSocket Key []byte }
SocketGroup holds the sockets needed to communicate with the kernel, and the key for message signing.
type SubscriptionId ¶ added in v0.10.0
SubscriptionId is returned by Kernel.SubscribeInterrupt, and can be used by Kernel.UnsubscribeInterrupt.
type SyncSocket ¶
SyncSocket wraps a zmq socket with a lock which should be used to control write access.