SkyLine_Backend_Scanner

package
v0.0.0-...-72c8661 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Aug 6, 2023 License: GPL-2.0 Imports: 10 Imported by: 0

Documentation

Overview

///////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////// 

	 ┏━┓
	┃┃ ┃
━━━━┛

Filename | SkyLine_Backend_Module_Scanner_CreateTKConstruct Extension | .go ( golang source code file ) Purpose | Define constant definitions for string values of Tokens Directory | Modules/Backend/SkyScanner Modular Directory | github.com/SkyPenguinLabs/SkyLine/Modules/Backend/SkyScanner Package Name | SkyLine_Backend_Scanner

::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: Module Description / Learners Activity :::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::

This file defines a new method to scan a new token or rather categorize a new token that the Scanner comes across

///////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////// 

	 ┏━┓
	┃┃ ┃
━━━━┛

Filename | SkyLine_Backend_Module_Scanner_InterfacePlug Extension | .go ( golang source code file ) Purpose | Define constant definitions for string values of Tokens Directory | Modules/Backend/SkyScanner Modular Directory | github.com/SkyPenguinLabs/SkyLine/Modules/Backend/SkyScanner Package Name | SkyLine_Backend_Scanner

::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: Module Description / Learners Activity :::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::

This file defines the main scanner, the function that is called to start lexical analysis which will help categorize and read specific byte orders or byte values. The scanner

will take some input code like `INTEGER value := 10;` and seperate it into readable information like the following below 

 {
		{'INTEGER': TYPE}
     {'value': IDENTIFIER}
     {':=': SLTK.TOKEN_ASSIGNMENT}
     {'10': VALUE}
     {';': SLTK.TOKEN_SEMICOLON}
	}

which can then be passed onto the parser for further dissection and 'parsing'. This file will define the function to scan those tokens, split the input and run other functions

which can be used to better help and better parse functions.

///////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////// 

	 ┏━┓
	┃┃ ┃
━━━━┛

Filename | SkyLine_Backend_Module_Scanner_IsVerifiers Extension | .go ( golang source code file ) Purpose | Define verification functions during lexical analysis Directory | Modules/Backend/SkyScanner Modular Directory | github.com/SkyPenguinLabs/SkyLine/Modules/Backend/SkyScanner Package Name | SkyLine_Backend_Scanner

::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: Module Description / Learners Activity :::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::

This file defines functions that can verify the type of a given byte by comparing its ASCII values. For example if the byte number is greater than or equal to b(a) and b(z) then we

can say that it is a character, if it is greater than or equal to b(1) || b(9) then we can say that it is a digit. This can help categorize the type of byte so the scanner can know

exactly what action or step to take based on a given series of conditions that are met when comparing. A table shows ASCII characters these verifiers use

hex | character ---- | --------- 0x61 | a 0x62 | b 0x63 | c 0x64 | d 0x65 | e 0x66 | f 0x67 | g 0x68 | h 0x69 | i 0x6A | j 0x6B | k 0x6C | l 0x6D | m 0x6E | n 0x6F | o 0x70 | p 0x71 | q 0x72 | r 0x73 | s 0x74 | t 0x75 | u 0x76 | v 0x77 | w 0x78 | x 0x79 | y 0x7A | z 0x41 | A 0x42 | B 0x43 | C 0x44 | D 0x45 | E 0x46 | F 0x47 | G 0x48 | H 0x49 | I 0x4A | J 0x4B | K 0x4C | L 0x4D | M 0x4E | N 0x4F | O 0x50 | P 0x51 | Q 0x52 | R 0x53 | S 0x54 | T 0x55 | U 0x56 | V 0x57 | W 0x58 | X 0x59 | Y 0x5A | Z 0x30 | 0 0x31 | 1 0x32 | 2 0x33 | 3 0x34 | 4 0x35 | 5 0x36 | 6 0x37 | 7 0x38 | 8 0x39 | 9

///////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////// 

	 ┏━┓
	┃┃ ┃
━━━━┛

Filename | SkyLine_Backend_Module_Scanner_Models Extension | .go ( golang source code file ) Purpose | Define constant definitions for string values of Tokens Directory | Modules/Backend/SkyScanner Modular Directory | github.com/SkyPenguinLabs/SkyLine/Modules/Backend/SkyScanner Package Name | SkyLine_Backend_Scanner

::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: Module Description / Learners Activity :::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::

This file helps define the proper interfaces and models that the lexer/scanner will rely on to keep track and store information. These structures help the scanner when categorizing

cutting or moving onto other tokens as well as keeping track of the lines currently being parsed or scanned in the file.

///////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////// 

	 ┏━┓
	┃┃ ┃
━━━━┛

Filename | SkyLine_Backend_Module_Scanner_NewMethod Extension | .go ( golang source code file ) Purpose | Defines a new method for the input scanner Directory | Modules/Backend/SkyScanner Modular Directory | github.com/SkyPenguinLabs/SkyLine/Modules/Backend/SkyScanner Package Name | SkyLine_Backend_Scanner

::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: Module Description / Learners Activity :::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::

This file defines a set of functions known as readers. The reader functions main jobs are to read specific values such as characters, read integers, read floats and data types

like string values as well and data within strings. This can improve the performance of the language by not having to use constant conditional expressions under each time we

want to parse a value or read a new character into the scanners stream.

///////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////// 

	 ┏━┓
	┃┃ ┃
━━━━┛

Filename | SkyLine_Backend_Module_Scanner_Readers Extension | .go ( golang source code file ) Purpose | Define readers for the scanner during lexical analysis Directory | Modules/Backend/SkyScanner Modular Directory | github.com/SkyPenguinLabs/SkyLine/Modules/Backend/SkyScanner Package Name | SkyLine_Backend_Scanner

::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: Module Description / Learners Activity :::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::

This file defines a set of functions known as readers. The reader functions main jobs are to read specific values such as characters, read integers, read floats and data types

like string values as well and data within strings. This can improve the performance of the language by not having to use constant conditional expressions under each time we

want to parse a value or read a new character into the scanners stream.

///////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////// 

	 ┏━┓
	┃┃ ┃
━━━━┛

Filename | SkyLine_Backend_Module_Scanner_Skippers Extension | .go ( golang source code file ) Purpose | Define all skipper functions for the scanner / lexical analysis step of the interpreter. Directory | Modules/Backend/SkyScanner Modular Directory | github.com/SkyPenguinLabs/SkyLine/Modules/Backend/SkyScanner Package Name | SkyLine_Backend_Scanner

::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: Module Description / Learners Activity :::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::

This file defines something the developers of SkyLine like to call skipper functions. Skipper functions are functions that have a purpose to consume or skip over specific sets

of data during lexican analysis. This kind of function will help skip over comments or white space. Most other people call these consumer functions or consume but we call them

skippers because they skip and read the characters until the end of the information that needs to be skipped such as white space. We refuse to use consumer because consuming it would

imply that it is "eating" or "erasing" the data and the information from the scanner / file which is not true. Skipper is a much more better term for this.

///////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////// 

	 ┏━┓
	┃┃ ┃
━━━━┛

Filename | SkyLine_Backend_Module_Scanner_SubFunctions Extension | .go ( golang source code file ) Purpose | Defines a function for string characters Directory | Modules/Backend/SkyScanner Modular Directory | github.com/SkyPenguinLabs/SkyLine/Modules/Backend/SkyScanner Package Name | SkyLine_Backend_Scanner

::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: Module Description / Learners Activity :::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::

This file defines a set of functions known as readers. The reader functions main jobs are to read specific values such as characters, read integers, read floats and data types

like string values as well and data within strings. This can improve the performance of the language by not having to use constant conditional expressions under each time we

want to parse a value or read a new character into the scanners stream.

Index

Constants

This section is empty.

Variables

View Source 
var ConstantIdents = make(map[string]bool)
View Source 
var Datatypes = []string{
	"string.",
	"float.",
	"object.",
	"hash.",
	"array.",
	"boolean.",
}

Functions

func CharacterInputIsDigit 

func CharacterInputIsDigit(Character rune) bool

func CharacterInputIsLetter 

func CharacterInputIsLetter(Character rune) bool

func CharacterIsEscapedCharacter 

func CharacterIsEscapedCharacter(Character rune) bool

func CharacterIsIdentifier 

func CharacterIsIdentifier(Character rune) bool

func CharacterIsWhiteSpace 

func CharacterIsWhiteSpace(Character rune) bool

func CheckEscapedCharacter 

func CheckEscapedCharacter(QUOTE, PEEK rune) string

func CheckintRange 

func CheckintRange(input string, min, max int64) bool

func ReadVerifyIdentifier 

func ReadVerifyIdentifier(ident string) SLTK.SL_TokenDataType

func ScanNewToken 

func ScanNewToken(TokenDataType SLTK.SL_TokenDataType, Character rune) SLTK.SL_TokenConstruct

Types

type Controller 

type Controller struct {
	ConstantIdentifiers []struct {
		Name   string `json:"Name"`
		Module string `json:"Module"`
	} `json:"ConstantIdentifiers"`
}

type SL_ScannerInterface 

type SL_ScannerInterface interface {
	NT() SLTK.SL_TokenConstruct
}

Type list

type SL_ScannerStructure 

type SL_ScannerStructure struct {
	CharacterInput            string                 // Scanners current character based input
	Scanner_POS               int                    // Scanners current position
	Scanner_RPOS              int                    // Scanners READ position
	Scanner_Character         rune                   // Scanners current character
	Scanner_Characters        []rune                 // Scanners list of current characters
	Scanner_PreviousToken     SLTK.SL_TokenConstruct // Scanners previous token construct
	Scanner_PreviousCharacter byte                   // Scanners previous character
	Scanner_CurrentLine       int                    // Scanners current line
}

Type list

func New 

func New(InputStream string) *SL_ScannerStructure

func (*SL_ScannerStructure) CharacterPeek 

func (SL_Scanner *SL_ScannerStructure) CharacterPeek() rune

func (*SL_ScannerStructure) NT 

func (SL_Scanner *SL_ScannerStructure) NT() SLTK.SL_TokenConstruct

func (*SL_ScannerStructure) RCHAR 

func (l *SL_ScannerStructure) RCHAR()

func (*SL_ScannerStructure) R_DECIMAL 

func (SL_Scanner *SL_ScannerStructure) R_DECIMAL() SLTK.SL_TokenConstruct

func (*SL_ScannerStructure) R_IDENTIFIER 

func (SL_Scanner *SL_ScannerStructure) R_IDENTIFIER() string

func (*SL_ScannerStructure) R_NUMBER 

func (SL_Scanner *SL_ScannerStructure) R_NUMBER() (string, string)

func (*SL_ScannerStructure) R_STRING 

func (SL_Scanner *SL_ScannerStructure) R_STRING(Character rune) string

func (*SL_ScannerStructure) ReadBacktick 

func (SL_Scanner *SL_ScannerStructure) ReadBacktick() string

func (*SL_ScannerStructure) ReadLineNum 

func (SL_Scanner *SL_ScannerStructure) ReadLineNum() int

func (*SL_ScannerStructure) SkipMultiLineComment 

func (SL_Scanner *SL_ScannerStructure) SkipMultiLineComment()

func (*SL_ScannerStructure) SkipSingleLineComment 

func (SL_Scanner *SL_ScannerStructure) SkipSingleLineComment()

func (*SL_ScannerStructure) SkipWhitespace 

func (SL_Scanner *SL_ScannerStructure) SkipWhitespace()

type TokenizerFunction 

type TokenizerFunction func(*SL_ScannerStructure) SLTK.SL_TokenConstruct

Type list

type TokenizerPeeked 

type TokenizerPeeked func(*SL_ScannerStructure) SLTK.SL_TokenConstruct

Type list

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.