ari

package module
v0.1.4 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Oct 14, 2024 License: MIT Imports: 18 Imported by: 0

README

Ari

Ari stands for Array Relational Interactive programming environment.

Ari is a set of extensions to the Goal programming language with an extensible CLI and dedicated SQL mode.

Installation

Clone this repository and then run the following:

go install github.com/semperos/ari/cmd/ari@latest

Then run ari for a REPL or ari --help to see CLI options.

Features

  • Goal is the core language
    • Goal's lib files are loaded by default, with prefix matching their file names (see vendor-goal folder in this repo)
  • Extensible CLI REPL with:
    • Auto-completion with documentation for:
      • Built-in keywords
      • Built-in syntax aliases (e.g., typing "first" and TAB will show * and ¿ in auto-complete results)
      • User-defined globals
    • Runtime configuration:
      • Configure the REPL prompt by setting string values for the ari.prompt and ari.nextprompt (for multiline input) globals
      • Replace default REPL printing by setting a function value for the ari.print global (function receives a single Goal value to print)
      • Configure the output format with --output-format or using one of the )output. system commands at the REPL. Formats include CSV/TSV, JSON, Markdown, and LaTeX.
    • ari.p is bound to the previous result (value from last evaluation at the REPL)
    • Alternatively run ari with --raw for a simpler, raw REPL that lacks line editing, history, and auto-complete, but is better suited for interaction via an editor like (Neo)Vim, or if you prefer rlwrap or another line editor to the one that ships with ari.
    • help based on Goal's, but allows adding help strings when used dyadically (e.g.,"sql.q"help"Run SQL query")
  • New Goal functions:
    • http. functions for HTTP requests using Resty
    • sql. functions for SQL queries and commands
    • tt. test framework
    • csv.tbl and json.tbl to make Goal tables from the output of csv and json respectively
    • time. functions for more extensive date/time handling
    • tui. functions for basic terminal UI styling (colors, padding/margin, borders)
  • Dedicated SQL mode
    • The ari CLI uses DuckDB, but the github.com/semperos/ari Go package doesn't directly depend on a specific SQL database driver, so you can BYODB.
    • Activate with )sql for read-only, )sql! for read/write modes. Execute )goal to return to the default Goal mode.
    • Auto-completion of SQL keywords
    • Help entries for SQL keywords (shown during auto-complete, still WIP)
    • Results of the last-run query/command set to the sql.p Goal global (for "SQL previous" and mirroring ari.p), so you can switch between )sql and )goal at the REPL to run queries via SQL and do data processing via Goal.

To Do

See CHANGES.md for recent changes.

Non-exhaustive list:

  • TODO: Test coverage
  • TODO: Consider which of ari's functions should be pervasive; review Goal's approach and implement
  • TODO: BUG When using -l flag, loaded files aren't completely evaluated. Evaluation stops partway through and it's not immediately clear why.
  • TODO: BUG json.tbl produces improper zero values when keys are missing. Where possible, column lists should be of uniform type and not generic if the data allows for it.
  • TODO: Make version of fmt.tbl that allows setting maximum column width.
  • TODO: Validate correct usage of goal.NewError vs. goal.NewPanicError
  • TODO: Functions to conveniently populate SQL tables with Goal values.
  • TODO: Support plots/charts (consider https://github.com/wcharczuk/go-chart)
  • TODO: User commands (as found in APL), executable from Goal or SQL modes

I plan to support the above items. The following are stretch goals or nice-to-have's:

  • TODO: Use custom table functions via replacement scan to query Goal tables from DuckDB.
  • TODO: Looser auto-complete, not just prefix-based
  • TODO: )help for cross-mode help
  • TODO: System functions to switch between rich and raw REPL )repl.rich and )repl.raw
  • IN PROGRESS: tui. functions in CLI mode using https://github.com/charmbracelet/lipgloss (already a transitive dependency) for colored output, etc.
  • TODO: Implement a subset of q functions to extend what Goal already has.
  • Specific user commands:
    • TODO: Toggle pretty-printing
    • TODO: Toggle paging at the REPL (as found in PicoLisp)
    • TODO: Toggle colored output

Examples

I began building Ari to replicate the experience described in the Background section of this README. That code is not publicly available at this time.

I am also using Ari to build an API client environment for the Shortcut REST API. The under-major-construction code for that can be found in this GitHub Gist.

Development

Ari is implemented in Go and Goal. See the script folder for common development operations.

To publish a new version of Ari:

./script/release vx.y.z

Background

I stumbled into a fairly flexible, powerful setup using Julia and DuckDB to do data analysis.

Details:

  • Julia as primary programming language
  • Julia Pluto Notebooks as primary programming environment
  • Notebook One: Data ETL
    • HTTP: Calling HTTP APIs to fetch JSON data (GitHub, Shortcut)
    • CSV: Transforming fetched JSON data into CSV
    • SQL: (Out of band) Defining SQL tables using SQL schema
    • SQL: (Out of band) Importing CSV into DuckDB using a schema defined in a SQL file
  • Notebook Two: Data Analyses
    • SQL: DuckDB tables as source of truth
    • Julia: Wrote trivial utility fn to transform arbitrary SQL query results into Julia DataFrames
      • DataFrame(DBInterface.execute(conn, sql))
      • Renders as a well-formatted table in the notebook interface
      • Array-language-like story for interacting with the DataFrame
      • Fully dynamic: type information from the database schema used to dynamically populate the DataFrame with data of appropriate types.
    • Julia: Mustache package for templating functions to build a LaTeX report to house all analyses
    • Julia: Plots package for generating plots as PDF to insert into final LaTeX report
    • Julia: Statistics, StatsBase packages both for aggregates and data to plot, pull from DataFrames
    • Julia: Executed external latexmk using Julia's run to build LaTeX report

Why move away from this setup?

  • Concision and expressive power of array languages.
  • A lot of my code was SQL in Julia.

Ari embeds the Goal array programming language. What gaps from my Julia+DuckDB experience need to be filled to use Goal where I used Julia?

  • Notebook programming environment
    • Cell dependencies and automatic re-run
      • Nice-to-have
    • Autocomplete
      • Required
    • Built-in language/library help documentation
      • Required
    • Rich rendering
      • Tables: Required
      • Graphics: Nice-to-have
  • HTTP Client
    • Nice-to-have (can rely on shelling out to curl to start).
  • SQL
    • Query results as Goal tables/dictionaries/arrays
      • Required
  • Plots
    • Declarative plot definition producing one of PNG/PDF or HTML/JS output
      • Required
      • q tutorials simply leverage JS bridge + Highcharts
      • Julia Plots package has pluggable backends:
        • GR (cross-platform, has C bindings)
        • Plotly (JS)
        • gnuplot (command-line utility)
        • HDF5 file format
      • See GNU Octave plotting
  • See Gonum

Goal already has a number of features, so we don't need to fill these gaps to start (more flexible options may be considered in the future):

  • Powerful string API
  • JSON support
  • CSV support

License

Ari

Original software in this repository is licensed as follows:

Copyright 2024 Daniel Gregoire

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the “Software”), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED “AS IS”, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

Goal

Source code copied and/or adapted from the Goal project has the following license:

Copyright (c) 2022 Yon anaseto@bardinflor.perso.aquilenet.fr

Permission to use, copy, modify, and distribute this software for any purpose with or without fee is hereby granted, provided that the above copyright notice and this permission notice appear in all copies.

THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.

Documentation

Overview

DO NOT EDIT THIS FILE. Generated by the ./script/release script.

Index

Constants

View Source 
const AriVersion = "v0.1.4"

Version of Ari. Run rt.get"v" to access this value at runtime.

Variables

This section is empty.

Functions

func GoalKeywordsHelp 

func GoalKeywordsHelp() map[string]string

func GoalSyntax 

func GoalSyntax() map[string]string

func NewGoalContext 

func NewGoalContext(ariContext *Context, help Help, sqlDatabase *SQLDatabase) (*goal.Context, error)

Initialize a Goal language context with Ari's extensions.

func NewHelp 

func NewHelp() map[string]map[string]string

func SQLExec 

func SQLExec(sqlDatabase *SQLDatabase, sqlQuery string, args []any) (goal.V, error)

func SQLKeywords 

func SQLKeywords() []string

From https://en.wikipedia.org/wiki/List_of_SQL_reserved_words

func SQLQueryContext 

func SQLQueryContext(sqlDatabase *SQLDatabase, sqlQuery string, args []any) (goal.V, error)

func VFGoalHelp added in v0.1.2 

func VFGoalHelp(help Help) func(_ *goal.Context, args []goal.V) goal.V

Implements Goal's help monad + Ari's help dyad.

func VFHTTPClientFn 

func VFHTTPClientFn() func(goalContext *goal.Context, args []goal.V) goal.V

func VFHTTPMaker 

func VFHTTPMaker(ariContext *Context, method string) func(goalContext *goal.Context, args []goal.V) goal.V

func VFServe added in v0.1.2 

func VFServe(goalContext *goal.Context, args []goal.V) goal.V

Implements http.serve dyad.

func VFSqlClose 

func VFSqlClose(_ *goal.Context, args []goal.V) goal.V

Implements sql.close to close the SQL database.

func VFSqlExecFn 

func VFSqlExecFn(sqlDatabase *SQLDatabase) func(goalContext *goal.Context, args []goal.V) goal.V

Implements sql.exec for executing SQL statements.

func VFSqlOpen 

func VFSqlOpen(_ *goal.Context, args []goal.V) goal.V

Implements sql.open to open a SQL database.

func VFSqlQFn 

func VFSqlQFn(sqlDatabase *SQLDatabase) func(goalContext *goal.Context, args []goal.V) goal.V

Implements sql.q for SQL querying.

func VFTimeAdd 

func VFTimeAdd(_ *goal.Context, args []goal.V) goal.V

Implements time.add function.

func VFTimeDate added in v0.1.2 

func VFTimeDate(_ *goal.Context, args []goal.V) goal.V

Implements time.date function.

func VFTimeDay added in v0.1.2 

func VFTimeDay(_ *goal.Context, args []goal.V) goal.V

Implements time.day function, for day of month.

func VFTimeFixedZone added in v0.1.2 

func VFTimeFixedZone(_ *goal.Context, args []goal.V) goal.V

Implements time.fixedzone dyad.

func VFTimeFormat added in v0.1.2 

func VFTimeFormat(_ *goal.Context, args []goal.V) goal.V

Implements time.format function.

func VFTimeHour added in v0.1.2 

func VFTimeHour(_ *goal.Context, args []goal.V) goal.V

Implements time.hour function.

func VFTimeLoadLocation added in v0.1.2 

func VFTimeLoadLocation(_ *goal.Context, args []goal.V) goal.V

Implements time.loadlocation monad.

func VFTimeLocation added in v0.1.2 

func VFTimeLocation(_ *goal.Context, args []goal.V) goal.V

Implements time.location function.

func VFTimeLocationString added in v0.1.2 

func VFTimeLocationString(_ *goal.Context, args []goal.V) goal.V

Implements time.locationstring function.

func VFTimeMicrosecond added in v0.1.2 

func VFTimeMicrosecond(_ *goal.Context, args []goal.V) goal.V

Implements time.microsecond function.

func VFTimeMillisecond added in v0.1.2 

func VFTimeMillisecond(_ *goal.Context, args []goal.V) goal.V

Implements time.millisecond function.

func VFTimeMinute added in v0.1.2 

func VFTimeMinute(_ *goal.Context, args []goal.V) goal.V

Implements time.minute function.

func VFTimeMonth added in v0.1.2 

func VFTimeMonth(_ *goal.Context, args []goal.V) goal.V

Implements time.month function (January = 1).

func VFTimeNanosecond added in v0.1.2 

func VFTimeNanosecond(_ *goal.Context, args []goal.V) goal.V

Implements time.nanosecond function.

func VFTimeNow 

func VFTimeNow(_ *goal.Context, _ []goal.V) goal.V

Implements time.now function.

func VFTimeParse 

func VFTimeParse(_ *goal.Context, args []goal.V) goal.V

Implements time.parse function.

func VFTimeSecond added in v0.1.2 

func VFTimeSecond(_ *goal.Context, args []goal.V) goal.V

Implements time.second function.

func VFTimeSub 

func VFTimeSub(_ *goal.Context, args []goal.V) goal.V

Implements time.sub function.

func VFTimeUTC added in v0.1.2 

func VFTimeUTC(_ *goal.Context, args []goal.V) goal.V

Implements time.utc function.

func VFTimeUnix added in v0.1.2 

func VFTimeUnix(_ *goal.Context, args []goal.V) goal.V

Implements time.unix function.

func VFTimeUnixMicro added in v0.1.2 

func VFTimeUnixMicro(_ *goal.Context, args []goal.V) goal.V

Implements time.unixmicro function.

func VFTimeUnixMilli added in v0.1.2 

func VFTimeUnixMilli(_ *goal.Context, args []goal.V) goal.V

Implements time.unixmilli function.

func VFTimeUnixNano added in v0.1.2 

func VFTimeUnixNano(_ *goal.Context, args []goal.V) goal.V

Implements time.unixnano function.

func VFTimeWeekDay added in v0.1.2 

func VFTimeWeekDay(_ *goal.Context, args []goal.V) goal.V

Implements time.weekday function, for day of week (Sunday = 0).

func VFTimeYear added in v0.1.2 

func VFTimeYear(_ *goal.Context, args []goal.V) goal.V

Implements time.year function.

func VFTimeYearDay added in v0.1.2 

func VFTimeYearDay(_ *goal.Context, args []goal.V) goal.V

Implements time.yearday function.

func VFTimeZoneName added in v0.1.2 

func VFTimeZoneName(_ *goal.Context, args []goal.V) goal.V

Implements time.zonename function.

func VFTimeZoneOffset added in v0.1.2 

func VFTimeZoneOffset(_ *goal.Context, args []goal.V) goal.V

Implements time.zoneoffset function.

func VFUrlEncode added in v0.1.2 

func VFUrlEncode(_ *goal.Context, args []goal.V) goal.V

Implements url.encode.

Types

type Context 

type Context struct {
	// GoalContext is needed to evaluate Goal programs and introspect the Goal execution environment.
	GoalContext *goal.Context
	// HTTPClient exposed for testing purposes.
	HTTPClient *HTTPClient
	// SQLDatabase keeps track of open database connections as well as the data source name.
	SQLDatabase *SQLDatabase
	// Help stores documentation information for identifiers.
	// The top-level keys must match a modes Name output;
	// the inner maps are a mapping from mode-specific identifiers
	// to a string that describes them and which is user-facing.
	Help Help
}

func NewContext 

func NewContext(dataSourceName string) (*Context, error)

Initialize a new Context without connecting to the database.

type HTTPClient 

type HTTPClient struct {
	Client *resty.Client
}

func NewHTTPClient 

func NewHTTPClient(optionsD *goal.D) (*HTTPClient, error)

func (*HTTPClient) Append 

func (httpClient *HTTPClient) Append(_ *goal.Context, dst []byte, _ bool) []byte

Append implements goal.BV.

func (*HTTPClient) LessT 

func (httpClient *HTTPClient) LessT(y goal.BV) bool

LessT implements goal.BV.

func (*HTTPClient) Matches 

func (httpClient *HTTPClient) Matches(y goal.BV) bool

Matches implements goal.BV.

func (*HTTPClient) Type 

func (httpClient *HTTPClient) Type() string

Type implements goal.BV.

type Help 

type Help struct {
	Dictionary map[string]map[string]string
	Func       func(string) string
}

type Location added in v0.1.2 

type Location struct {
	Location *time.Location
}

func (*Location) Append added in v0.1.2 

func (location *Location) Append(_ *goal.Context, dst []byte, _ bool) []byte

Append implements goal.BV.

func (*Location) LessT added in v0.1.2 

func (location *Location) LessT(y goal.BV) bool

LessT implements goal.BV.

func (*Location) Matches added in v0.1.2 

func (location *Location) Matches(_ goal.BV) bool

Matches implements goal.BV.

func (*Location) Type added in v0.1.2 

func (location *Location) Type() string

Type implements goal.BV.

type SQLDatabase 

type SQLDatabase struct {
	DataSource string
	DB         *sql.DB
	IsOpen     bool
}

func NewSQLDatabase 

func NewSQLDatabase(dataSourceName string) (*SQLDatabase, error)

Initialize SQL struct, but don't open the DB yet.

Call SQLDatabase.open to open the database.

func (*SQLDatabase) Append 

func (sqlDatabase *SQLDatabase) Append(_ *goal.Context, dst []byte, _ bool) []byte

Append implements goal.BV.

func (*SQLDatabase) Close 

func (sqlDatabase *SQLDatabase) Close() error

Close will close the underlying sql.DB, if one exists.

func (*SQLDatabase) LessT 

func (sqlDatabase *SQLDatabase) LessT(y goal.BV) bool

LessT implements goal.BV.

func (*SQLDatabase) Matches 

func (sqlDatabase *SQLDatabase) Matches(y goal.BV) bool

Matches implements goal.BV.

func (*SQLDatabase) Open 

func (sqlDatabase *SQLDatabase) Open() error

func (*SQLDatabase) Type 

func (sqlDatabase *SQLDatabase) Type() string

Type implements goal.BV.

type Time 

type Time struct {
	Time *time.Time
}

func (*Time) Append 

func (time *Time) Append(_ *goal.Context, dst []byte, _ bool) []byte

Append implements goal.BV.

func (*Time) LessT 

func (time *Time) LessT(y goal.BV) bool

LessT implements goal.BV.

func (*Time) Matches 

func (time *Time) Matches(y goal.BV) bool

Matches implements goal.BV.

func (*Time) Type 

func (time *Time) Type() string

Type implements goal.BV.

Source Files

View all Source files

Directories

Path Synopsis
cmd
ari
vendored
help
Package help exports a function with default help for the core Goal language.
 Package help exports a function with default help for the core Goal language.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.