Documentation ¶
Overview ¶
Package syms defines the symbols and their properties that are accumulated from a parsed file, and are then used for e.g., completion lookup, etc.
We looked at several different standards for formats, types, etc:
LSP: https://microsoft.github.io/language-server-protocol/specification useful to enable GoPi to act as an LSP server at some point. additional symbol kinds: https://github.com/Microsoft/language-server-protocol/issues/344
See also: github.com/sourcegraph/sourcegraph and specifically: /cmd/frontend/graphqlbackend/search_symbols.go it seems to use https://github.com/universal-ctags/ctags for the raw data..
Other relevant guidance comes from the go compiler system which is used extensively in github.com/mdemsky/gocode for example. In particular: go/types/scope.go type.go, and package.go contain the relevant data structures for how information is organized for compiled go packages, which have all this data cached and avail to be imported via the go/importer which returns a go/types/Package which in turn contains Scope's which in turn contain Objects that define the elements of the compiled language.
Index ¶
- Variables
- func CacheFilename(lang filecat.Supported, filename string) (string, error)
- func GoPiCacheDir(lang filecat.Supported) (string, error)
- func GoRelPath(filename string) (string, error)
- func InitCatMap()
- func InitSub2CatMap()
- func InitSubCatMap()
- func SaveSymCache(sy *Symbol, lang filecat.Supported, filename string) error
- func SaveSymDoc(sy *Symbol, lang filecat.Supported, filename string) error
- type Kinds
- func (tk Kinds) Cat() Kinds
- func (i *Kinds) FromString(s string) error
- func (tk Kinds) InCat(other Kinds) bool
- func (tk Kinds) InSub2Cat(other Kinds) bool
- func (tk Kinds) InSubCat(other Kinds) bool
- func (tk Kinds) IsCat() bool
- func (tk Kinds) IsPrimitiveNonPtr() bool
- func (tk Kinds) IsPtr() bool
- func (tk Kinds) IsSub2Cat() bool
- func (tk Kinds) IsSubCat() bool
- func (ev Kinds) MarshalJSON() ([]byte, error)
- func (ev Kinds) MarshalText() ([]byte, error)
- func (i Kinds) String() string
- func (tk Kinds) Sub2Cat() Kinds
- func (tk Kinds) SubCat() Kinds
- func (ev *Kinds) UnmarshalJSON(b []byte) error
- func (ev *Kinds) UnmarshalText(b []byte) error
- type SymMap
- func (sm *SymMap) Add(sy *Symbol)
- func (sm *SymMap) AddNew(name string, kind token.Tokens, fname string, reg lex.Reg) *Symbol
- func (sm *SymMap) Alloc()
- func (sm *SymMap) CopyFrom(src SymMap)
- func (sm *SymMap) FindContainsRegion(fpath string, pos lex.Pos, extraLns int, kind token.Tokens, matches *SymMap)
- func (sm *SymMap) FindKind(kind token.Tokens, matches *SymMap)
- func (sm *SymMap) FindKindScoped(kind token.Tokens, matches *SymMap)
- func (sm *SymMap) FindName(nm string) (*Symbol, bool)
- func (sm *SymMap) FindNamePrefix(seed string, matches *SymMap)
- func (sm *SymMap) FindNamePrefixRecursive(seed string, matches *SymMap)
- func (sm *SymMap) FindNamePrefixScoped(seed string, matches *SymMap)
- func (sm *SymMap) FindNameScoped(nm string) (*Symbol, bool)
- func (sm *SymMap) First() *Symbol
- func (sm *SymMap) KindNames(sorted bool) []string
- func (sm *SymMap) Names(sorted bool) []string
- func (sm *SymMap) OpenJSON(filename string) error
- func (sm *SymMap) Reset()
- func (sm *SymMap) SaveJSON(filename string) error
- func (sm *SymMap) Slice(sorted bool) []*Symbol
- func (sm *SymMap) WriteDoc(out io.Writer, depth int)
- type SymNames
- type SymStack
- type Symbol
- func (sy *Symbol) AddChild(child *Symbol) bool
- func (sy *Symbol) AddScopesMap(sm SymMap, add bool)
- func (sy *Symbol) AddScopesStack(ss SymStack) bool
- func (sy *Symbol) AllocScopes()
- func (sy *Symbol) Clone() *Symbol
- func (sy *Symbol) CopyFromScope(src *Symbol)
- func (sy *Symbol) FindAnyChildren(seed string, scope1, scope2 SymMap, kids *SymMap) bool
- func (sy *Symbol) HasChildren() bool
- func (sy *Symbol) IsTemp() bool
- func (sy *Symbol) NonPtrTypeName() string
- func (sy *Symbol) OpenJSON(filename string) error
- func (sy *Symbol) SaveJSON(filename string) error
- func (sy *Symbol) String() string
- func (sy *Symbol) WriteDoc(out io.Writer, depth int)
- type Type
- type TypeEl
- type TypeEls
- type TypeKindSize
- type TypeMap
- func (tm *TypeMap) Add(ty *Type)
- func (tm *TypeMap) Alloc()
- func (tm *TypeMap) Clone() TypeMap
- func (tm *TypeMap) CopyFrom(src TypeMap)
- func (tm *TypeMap) KindNames(sorted bool) []string
- func (tm *TypeMap) Names(sorted bool) []string
- func (tm *TypeMap) PrintUnknowns()
- func (tm *TypeMap) WriteDoc(out io.Writer, depth int)
Constants ¶
This section is empty.
Variables ¶
var CatMap map[Kinds]Kinds
CatMap is the map into the category level for each kind
Categories
var KiT_Kinds = kit.Enums.AddEnum(KindsN, kit.NotBitFlag, nil)
var KiT_Symbol = kit.Types.AddType(&Symbol{}, nil)
var ReflectKindMap = map[reflect.Kind]Kinds{ reflect.Invalid: Unknown, reflect.Int: Int, reflect.Int8: Int8, reflect.Int16: Int16, reflect.Int32: Int32, reflect.Int64: Int64, reflect.Uint: Uint, reflect.Uint8: Uint8, reflect.Uint16: Uint16, reflect.Uint32: Uint32, reflect.Uint64: Uint64, reflect.Uintptr: Uintptr, reflect.Float32: Float32, reflect.Float64: Float64, reflect.Complex64: Complex64, reflect.Complex128: Complex128, reflect.Bool: Bool, reflect.Array: Array, reflect.Chan: Chan, reflect.Func: Func, reflect.Interface: Interface, reflect.Map: Map, reflect.Ptr: Ptr, reflect.Slice: List, reflect.String: String, reflect.Struct: Struct, reflect.UnsafePointer: Ptr, }
var Sub2CatMap map[Kinds]Kinds
Sub2CatMap is the map into the sub2-category level for each kind
var Sub2Cats = []Kinds{ Unknown, Primitive, Numeric, Integer, Fixed, Float, Bool, Composite, Tuple, Array, List, Matrix, Tensor, Map, Set, Struct, Chan, Function, Func, Method, Interface, KindsN, }
Sub2-Categories
var SubCatMap map[Kinds]Kinds
SubCatMap is the map into the sub-category level for each kind
var SubCats = []Kinds{ Unknown, Primitive, Numeric, Bool, Composite, Tuple, Array, List, Matrix, Tensor, Map, Set, Struct, Chan, Function, Func, Method, Interface, KindsN, }
Sub-Categories
Functions ¶
func CacheFilename ¶
CacheFilename returns the filename to use for cache file for given filename
func GoPiCacheDir ¶
GoPiCacheDir returns the GoPi cache directory for given language, and ensures that it exists
func SaveSymCache ¶
SaveSymCache saves cache of symbols starting with given symbol (typically a package, module, library), which is at given filename
Types ¶
type Kinds ¶
type Kinds int
Kinds is a complete set of basic type categories and sub(sub..) categories -- these describe builtin types -- user-defined types must be some combination / version of these builtin types.
See: https://en.wikipedia.org/wiki/List_of_data_structures
const ( // Unknown is the nil kind -- kinds should be known in general.. Unknown Kinds = iota // Category: Primitive, in the strict sense of low-level, atomic, small, fixed size Primitive // SubCat: Numeric Numeric // Sub2Cat: Integer Integer // Sub3Cat: Signed -- track this using props in types, not using Sub3 level Signed Int Int8 Int16 Int32 Int64 // Sub3Cat: Unsigned Unsigned Uint Uint8 Uint16 Uint32 Uint64 Uintptr // generic raw pointer data value -- see also Ptr, Ref for more semantic cases // Sub3Cat: Ptr, Ref etc -- in Numeric, Integer even though in some languages // pointer arithmetic might not be allowed, for some cases, etc Ptr // pointer -- element is what we point to (kind of a composite type) Ref // reference -- element is what we refer to UnsafePtr // for case where these are distinguished from Ptr (Go) -- similar to Uintptr // Sub2Cat: Fixed point -- could be under integer, but.. Fixed Fixed26_6 Fixed16_6 Fixed0_32 // Sub2Cat: Floating point Float Float16 Float32 Float64 // Sub3Cat: Complex -- under floating point Complex Complex64 Complex128 // SubCat: Bool Bool // Category: Composite -- types composed of above primitive types Composite // SubCat: Tuple -- a fixed length 1d collection of elements that can be of any type // Type.Els required for each element Tuple Range // a special kind of tuple for Python ranges // SubCat: Array -- a fixed length 1d collection of same-type elements // Type.Els has one element for type Array // SubCat: List -- a variable-length 1d collection of same-type elements // This is Slice for Go // Type.Els has one element for type List String // List of some type of char rep -- Type.Els is type, as all Lists // SubCat: Matrix -- a twod collection of same-type elements // has two Size values, one for each dimension Matrix // SubCat: Tensor -- an n-dimensional collection of same-type elements // first element of Size is number of dimensions, rest are dimensions Tensor // SubCat: Map -- an associative array / hash map / dictionary // Type.Els first el is key, second is type Map // SubCat: Set -- typically a degenerate form of hash map with no value Set FrozenSet // python's frozen set of fixed values // SubCat: Struct -- like a tuple but with specific semantics in most languages // Type.Els are the fields, and if there is an inheritance relationship these // are put first with relevant identifiers -- in Go these are unnamed fields Struct Class Object // Chan: a channel (Go Specific) Chan // Category: Function -- types that are functions // Type.Els are the params and return values in order, with Size[0] being number // of params and Size[1] number of returns Function // SubCat: Func -- a standalone function Func // SubCat: Method -- a function with a specific receiver (e.g., on a Class in C++, // or on any type in Go). // First Type.Els is receiver param -- included in Size[0] Method // SubCat: Interface -- an abstract definition of a set of methods (in Go) // Type.Els are the Methods with the receiver type missing or Unknown Interface KindsN )
The list of Kinds
func (*Kinds) FromString ¶
func (Kinds) IsPrimitiveNonPtr ¶ added in v0.9.11
func (Kinds) MarshalJSON ¶
func (Kinds) MarshalText ¶
map keys require text marshaling:
func (Kinds) Sub2Cat ¶
Sub2Cat returns the sub2-category that a given kind lives in, using Sub2CatMap
func (*Kinds) UnmarshalJSON ¶
func (*Kinds) UnmarshalText ¶
type SymMap ¶
SymMap is a map between symbol names and their full information. A given project will have a top-level SymMap and perhaps local maps for individual files, etc. Namespaces / packages can be created and elements added to them to create appropriate scoping structure etc. Note that we have to use pointers for symbols b/c otherwise it is very expensive to re-assign values all the time -- https://github.com/golang/go/issues/3117
func (*SymMap) CopyFrom ¶
CopyFrom copies all the symbols from given source map into this one, including merging everything from common elements. Symbols with Type resolved are retained when there are duplicates.
func (*SymMap) FindContainsRegion ¶
func (sm *SymMap) FindContainsRegion(fpath string, pos lex.Pos, extraLns int, kind token.Tokens, matches *SymMap)
FindContainsRegion looks for given symbol kind that contains the given source file path (must be filepath.Abs file path) and position. Returns all instances found. Uses cat / subcat based token matching -- if you specify a category-level or subcategory level token, it will match everything in that group. if you specify kind = token.None then all tokens that contain region will be returned. extraLns are extra lines added to the symbol region for purposes of matching.
func (*SymMap) FindKind ¶
FindKind looks for given symbol kind within this map and any children on the map Returns all instances found. Uses cat / subcat based token matching -- if you specify a category-level or subcategory level token, it will match everything in that group
func (*SymMap) FindKindScoped ¶
FindKindScoped looks for given symbol kind within this map and any children on the map that are of subcategory token.NameScope (i.e., namespace, module, package, library). Returns all instances found. Uses cat / subcat based token matching -- if you specify a category-level or subcategory level token, it will match everything in that group
func (*SymMap) FindName ¶
FindName looks for given symbol name within this map and any children on the map
func (*SymMap) FindNamePrefix ¶
FindNamePrefix looks for given symbol name prefix within this map adds to given matches map (which can be nil), for more efficient recursive use
func (*SymMap) FindNamePrefixRecursive ¶ added in v0.5.7
FindNamePrefixRecursive looks for given symbol name prefix within this map and any children on the map. adds to given matches map (which can be nil), for more efficient recursive use
func (*SymMap) FindNamePrefixScoped ¶
FindNamePrefixScoped looks for given symbol name prefix within this map and any children on the map that are of subcategory token.NameScope (i.e., namespace, module, package, library) adds to given matches map (which can be nil), for more efficient recursive use
func (*SymMap) FindNameScoped ¶
FindNameScoped looks for given symbol name within this map and any children on the map that are of subcategory token.NameScope (i.e., namespace, module, package, library)
func (*SymMap) First ¶
First returns the first symbol in the map -- only sensible when there is just one such element
func (*SymMap) KindNames ¶
KindNames returns a slice of the kind:names in this map, optionally sorted
type SymNames ¶
SymNames provides a map-list of symbol names, indexed by their token kinds. Used primarily for specifying Scopes
type SymStack ¶
type SymStack []*Symbol
SymStack is a simple stack (slice) of symbols
func (*SymStack) FindNameScoped ¶
FindNameScoped searches top-down in the stack for something with the given name in symbols that are of subcategory token.NameScope (i.e., namespace, module, package, library)
type Symbol ¶
type Symbol struct { Name string `desc:"name of the symbol"` Detail string `desc:"additional detail and specification of the symbol -- e.g. if a function, the signature of the function"` Kind token.Tokens `desc:"lexical kind of symbol, using token.Tokens list"` Type string `` /* 143-byte string literal not displayed */ Index int `desc:"index for ordering children within a given scope, e.g., fields in a struct / class"` Filename string `desc:"full filename / URI of source"` Region lex.Reg `desc:"region in source encompassing this item -- if = RegZero then this is a temp symbol and children are not added to it"` SelectReg lex.Reg `desc:"region that should be selected when activated, etc"` Scopes SymNames `desc:"relevant scoping / parent symbols, e.g., namespace, package, module, class, function, etc.."` Children SymMap `` /* 140-byte string literal not displayed */ Types TypeMap `desc:"types defined within the scope of this symbol"` Ast ki.Ki `json:"-" xml:"-" desc:"Ast node that created this symbol -- only valid during parsing"` }
Symbol contains the information for everything about a given symbol that is created by parsing, and can be looked up. It corresponds to the LSP DocumentSymbol structure, and the Go Object type.
func NewSymbol ¶
NewSymbol returns a new symbol with the basic info filled in -- SelectReg defaults to Region
func OpenSymCache ¶
OpenSymCache opens cache of symbols into given symbol (typically a package, module, library), which is at given filename -- returns time stamp when cache was last saved
func (*Symbol) AddChild ¶
AddChild adds a child symbol, if this parent symbol is not temporary returns true if item name was added and NOT already on the map, and false if it was already or parent is temp. Always adds new symbol in any case. If parent symbol is of the NameType subcategory, then index of child is set to the size of this child map before adding.
func (*Symbol) AddScopesMap ¶
AddScopesMap adds a given scope element(s) from map to this Symbol. if add is true, add this symbol to those scopes if they are not temporary.
func (*Symbol) AddScopesStack ¶
AddScopesStack adds a given scope element(s) from stack to this Symbol. Adds this symbol as a child to the top of the scopes if it is not temporary -- returns true if so added.
func (*Symbol) AllocScopes ¶
func (sy *Symbol) AllocScopes()
AllocScopes allocates scopes map if nil
func (*Symbol) Clone ¶ added in v0.9.11
Clone returns a clone copy of this symbol. Does NOT copy the Children or Types -- caller can decide about that.
func (*Symbol) CopyFromScope ¶ added in v0.9.11
CopyFromScope copies the Children and Types from given other symbol for scopes (e.g., Go package), to merge with existing.
func (*Symbol) FindAnyChildren ¶ added in v0.5.7
FindAnyChildren finds children of this symbol using either direct children if those are present, or the type name if present -- used for completion routines. Adds to kids map. scope1, scope2 are used for looking up type name. If seed is non-empty it is used as a prefix for filtering children names. Returns false if no children were found.
func (*Symbol) HasChildren ¶
HasChildren returns true if this symbol has children
func (*Symbol) IsTemp ¶
IsTemp returns true if this is temporary symbol that is used for scoping but is not otherwise permanently added to list of symbols. Indicated by Zero Region.
func (*Symbol) NonPtrTypeName ¶
NonPtrTypeName returns the name of the type without any leading * or &
type Type ¶
type Type struct { Name string `desc:"name of the type -- can be the name of a field or the role for a type element"` Kind Kinds `desc:"kind of type -- overall nature of the type"` Desc string `desc:"documentation about this type, extracted from code"` Inited bool `inactive:"-" desc:"set to true after type has been initialized during post-parse processing"` Els TypeEls `` /* 309-byte string literal not displayed */ Meths TypeMap `desc:"methods defined for this type"` Size []int `` /* 247-byte string literal not displayed */ Filename string `desc:"full filename / URI of source where type is defined (may be empty for auto types)"` Region lex.Reg `desc:"region in source encompassing this type"` Scopes SymNames `desc:"relevant scoping / parent symbols, e.g., namespace, package, module, class, function, etc.."` Props ki.Props `` /* 229-byte string literal not displayed */ Ast ki.Ki `json:"-" xml:"-" desc:"Ast node that corresponds to this type -- only valid during parsing"` }
Type contains all the information about types. Types can be builtin or composed of builtin types. Each type can have one or more elements, e.g., fields for a struct or class, multiple values for a go function, or the two types for a map (key, value), etc..
func (*Type) AddScopesStack ¶ added in v0.9.11
AddScopesStack adds a given scope element(s) from stack to this Type.
func (*Type) AllocScopes ¶ added in v0.9.11
func (ty *Type) AllocScopes()
AllocScopes allocates scopes map if nil
func (*Type) ArgString ¶ added in v0.9.11
ArgString() returns string of args to function if it is a function type
func (*Type) Clone ¶ added in v0.9.11
Clone returns a deep copy of this type, cloning / copying all sub-elements except the Ast, and Inited
func (*Type) NonPtrType ¶
NonPtrType returns the non-pointer name of this type, if it is a pointer type otherwise just returns Name
func (*Type) ReturnString ¶ added in v0.9.11
ReturnString() returns string of return vals of function if it is a function type
type TypeEl ¶
type TypeEl struct { Name string `desc:"element name -- e.g., field name for struct, or functional name for other types"` Type string `desc:"type name -- looked up on relevant lists -- includes scoping / package / namespace name as appropriate"` }
TypeEl is a type element -- has a name (local to the type, e.g., field name) and a type name that can be looked up in a master list of types
type TypeEls ¶
type TypeEls []TypeEl
TypeEls are the type elements for types
func (*TypeEls) StringRange ¶ added in v0.9.11
StringRange() returns a string rep of range of items
type TypeKindSize ¶
TypeKindSize is used for initialization of builtin typemaps
type TypeMap ¶
TypeMap is a map of types for quick looking up by name
func (*TypeMap) Clone ¶ added in v0.9.11
Clone returns deep copy of this type map -- types are Clone() copies. returns nil if this map is empty
func (*TypeMap) CopyFrom ¶
CopyFrom copies all the types from given source map into this one. Types that have Kind != Unknown are retained over unknown ones.
func (*TypeMap) KindNames ¶
KindNames returns a slice of the kind:names in this map, optionally sorted
func (*TypeMap) PrintUnknowns ¶ added in v0.9.11
func (tm *TypeMap) PrintUnknowns()
PrintUnknowns prints all the types that have a Kind = Unknown indicates an error in type resolution