fs533lib

package
v0.0.0-...-b744d6b Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Apr 24, 2021 License: MIT Imports: 24 Imported by: 0

Documentation

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func ConvertCurrentTimeToString 

func ConvertCurrentTimeToString() string

ConvertCurrentTimeToString ... to convert time.Now to a specific datetime format

func ConvertTimeToLongString 

func ConvertTimeToLongString(time time.Time) string

ConvertTimeToLongString ... to convert time to a specific datetime format

Types

type Configuration 

type Configuration struct {
	InstanceID                              string //instanceId of EC2 instance (a node)
	IPAddress                               string //instance's private address
	TCPPort                                 int    //TCP port which is used for listening to leave/join requests
	UDPPort                                 int    //UDP port which is used for listening to heartbeat messages
	LogPath                                 string //log files directory
	Fs533FilePath                           string //fs533 files directory
	LocalFilePath                           string //local files directory
	HostAddress                             string //host address which is IPAddress:TCPPort
	HeartbeatAddress                        string //heartbeat address which is IPAddress:UDPPort
	MaxDatagramSize                         int
	Timeout                                 int      //timeout for Join/Leave request
	HeartbeatIntervalTimeInMilliseconds     int      //interval time between 2 heartbeat messages
	HeartbeatTimeoutInMilliseconds          int      //timeout indicating that the node has failed
	GatewayNodes                            []string //3 leading nodes which are supposed that they will not fail at the same
	NumberOfPrecessorsAndSuccessors         int      //number of the predecessors and successors on the healthcheck topology ring
	NumberOfReplicas                        int      //number of preference nodes where storing replicate of a file
	WriteQuorum                             int      //write quorum
	ReadQuorum                              int      //read quorum
	FileChunkSize                           int      //chunk size for treaming file
	PendingWriteTransactionTimeoutInSeconds int      //a pending write transaction will be timeout if it reaches this total time
	NumberOfHeartbeatsShown                 int      //number of heartbeats which are printed out
	HearbeatLog                             bool     //turn on/off heartbeat log
	Membershiplog                           bool     //turn on/off membership log
}

Configuration ...Server settings

func LoadConfiguration 

func LoadConfiguration(configFile string) Configuration

LoadConfiguration ... do

type FetchedFileMetadata 

type FetchedFileMetadata struct {
	Fs533FileName    string //file name on the fs533 file system
	Fs533FileVersion int32  //file version
	Fs533FileSize    int32  //file size
	LocalFileName    string //file name which is fetched on local storag
}

FetchedFileMetadata ... this struct is to describe all information related to a fs533file which is already fetched to local storage

type FileClient 

type FileClient struct{}

FileClient ... this struct is to handle all the file operations from client side

func (*FileClient) ClearLocalDbTestFiles 

func (a *FileClient) ClearLocalDbTestFiles()

ClearLocalDbTestFiles ... This function is to delete all local test files

func (*FileClient) DeleteAllFiles 

func (a *FileClient) DeleteAllFiles()

DeleteAllFiles ... This function is to delete all files

func (*FileClient) Fetch 

func (a *FileClient) Fetch(fs533filename string, localfilename string)

Fetch ... this function is to list all files in fs533

func (*FileClient) Initialize 

func (a *FileClient) Initialize(mn string, config Configuration)

Initialize ... setup fileClient with a given configuation file

func (*FileClient) InternalEndWriteFile 

func (a *FileClient) InternalEndWriteFile(requestID string, fs533filename string, fs533filesize int32, targetNodeAddress string, cancelWriteRegistration bool, commit bool) bool

InternalEndWriteFile ... this function is to finish a write transaction, it will be commit or aborted arguments -fs533filename: filename on fs533 system which is being processed -targetNode: the node receives this commit request -cancelWriteRegistration: cancel pending write registration on master node -commit: the transation will be commited if it is true. Otherwise, the transaction will be aborted

func (*FileClient) InternalReadAcquireFile 

func (a *FileClient) InternalReadAcquireFile(fs533filename string, replicaNodeAddress string) (bool, int32, []string)

InternalReadAcquireFile ... this function is to send read acquire request to the target node to get the file's latest version It return file's latest version on the target replica node and its preference nodes

func (*FileClient) InternalRegisterWriteFile 

func (a *FileClient) InternalRegisterWriteFile(requestID string, fs533filename string, targetNodeAddress string) (bool, bool, int32, []string)

InternalRegisterWriteFile ... this function is to register write operation to master it should return register status, needUserInteraction, fileVersion, preferenceNodes

func (*FileClient) InternalRemoveFile 

func (a *FileClient) InternalRemoveFile(fs533filename string, targetNode string) bool

InternalRemoveFile ... this function is to send remove request directly to the target node

func (*FileClient) ListAllFiles 

func (a *FileClient) ListAllFiles() []string

ListAllFiles ... this function is to list all files in fs533

func (*FileClient) ListHere 

func (a *FileClient) ListHere()

ListHere ... this function is to list all files in fs533

func (*FileClient) LocateFile 

func (a *FileClient) LocateFile(fs533filename string) Fs533FileMetadata

LocateFile ... this function is to locate replica node for fs533 file

func (*FileClient) PrintAllFiles 

func (a *FileClient) PrintAllFiles(files []string)

PrintAllFiles ... this function is to print all files queried from ListAllFile API

func (*FileClient) PrintLocateFile 

func (a *FileClient) PrintLocateFile(fs533filename string)

PrintLocateFile ... this function is to list all machines (name / id / IP address) of the servers that contain a copy of the file.

func (*FileClient) Put 

func (a *FileClient) Put(localfilename string, fs533filename string)

Put ... this function is to put all files in fs533

func (*FileClient) RemoveFile 

func (a *FileClient) RemoveFile(fs533filename string)

RemoveFile ... this public function is to delete file fs533filename from fs533

func (*FileClient) Report 

func (a *FileClient) Report()

Report ... this function is to list all the local fetched files, local replicated files and global files metadata

type FileOperation 

type FileOperation struct{}

FileOperation ... this struct is to handle all the local file operations. All local files are stored on ../fs533db

func (*FileOperation) ClearFs533Db 

func (a *FileOperation) ClearFs533Db()

ClearFs533Db ... delete all existing files on the fs533 db folder

func (*FileOperation) ClearLocalDb 

func (a *FileOperation) ClearLocalDb(prefix string)

ClearLocalDb ... delete all existing files on the fs533 db folder

func (*FileOperation) Copy 

func (*FileOperation) Copy(src string, des string) (int64, error)

Copy ... copy a file to another file

func (*FileOperation) CreateOrOpenFile 

func (*FileOperation) CreateOrOpenFile(localfile string) *os.File

CreateOrOpenFile ... open or create a file for writing

func (*FileOperation) Delete 

func (*FileOperation) Delete(fs533FileName string)

Delete ... delete file from folder fs533db

func (*FileOperation) Fs533FilePath 

func (*FileOperation) Fs533FilePath(filename string) string

Fs533FilePath ... the path and the file name to the full path file

func (*FileOperation) ListHere 

func (*FileOperation) ListHere(path string) []string

ListHere ... list all files on folder fs533db

func (*FileOperation) LocalFileExist 

func (a *FileOperation) LocalFileExist(fileName string) bool

LocalFileExist ... check if the file exists on local folder or not

func (*FileOperation) LocalFilePath 

func (*FileOperation) LocalFilePath(filename string) string

LocalFilePath ... the path and the file name to the full path file

func (*FileOperation) OpenFile 

func (*FileOperation) OpenFile(localfile string) *os.File

OpenFile ... open file for reading

type FileServer 

type FileServer struct{}

FileServer ... this struct is to handle all the file operations from server side

func (*FileServer) Backup 

func (a *FileServer) Backup()

Backup ... backup all infomration about file operations that a master node should know

func (*FileServer) FireNodeLeave 

func (a *FileServer) FireNodeLeave(nodeleaveAddress string)

FireNodeLeave ... farewell a node leave by handover its replicated files to the other nodes

func (*FileServer) Start 

func (a *FileServer) Start(master string, eTerm int32, initFs533Files []Fs533FileMetadata, initPendingWriteTransactions []WriteTransaction)

Start ... start file server by initialize the global variables

func (*FileServer) StartLeaderElectionProcess 

func (a *FileServer) StartLeaderElectionProcess()

StartLeaderElectionProcess ... start self voting process to be the new master node

func (*FileServer) StartPromotingAsMaster 

func (a *FileServer) StartPromotingAsMaster() bool

StartPromotingAsMaster ... start sending requests to other nodes to self-promote as a new master node

func (*FileServer) Stop 

func (a *FileServer) Stop()

Stop ... stop the current service by resetting all global variable

func (*FileServer) UpdateFs533ReplicaNodes 

func (a *FileServer) UpdateFs533ReplicaNodes(newMember string)

UpdateFs533ReplicaNodes ... update all the information of file operations that a master node should know

func (*FileServer) UpdateMetada 

func (a *FileServer) UpdateMetada(initFs533Files []Fs533FileMetadata, initPendingWriteTransactions []WriteTransaction)

UpdateMetada ... update all the information of file operations that a master node should know

type Fs533FileMetadata 

type Fs533FileMetadata struct {
	Fs533FileName    string    //file name on the fs533 file system
	Fs533FileVersion int32     //file version
	Fs533FileSize    int32     //file size
	RegisteredTime   time.Time //latest registered time
	PreferenceNodes  []string  //list of its replica nodes
}

Fs533FileMetadata ... this truct is to describe all the information of a file located on fs533 system

type LogManager 

type LogManager struct{}

LogManager ... this struct is to manage all log events

func (*LogManager) DisableLog 

func (a *LogManager) DisableLog()

DisableLog ... this function is to disable logs

func (*LogManager) EnableLog 

func (a *LogManager) EnableLog()

EnableLog ... this function is to enable logs related to file operation, it will log on both terminal and file

type MembershipService 

type MembershipService struct {
	// contains filtered or unexported fields
}

MembershipService ... is a struct to expose public functions

func (*MembershipService) GetFollowerAddresses 

func (a *MembershipService) GetFollowerAddresses() []string

GetFollowerAddresses ... This function is to return active members on the group, except the master node

func (*MembershipService) GetNodeAddresses 

func (a *MembershipService) GetNodeAddresses() []string

GetNodeAddresses ... This function is to return active members on the group, except the master node

func (*MembershipService) Initialize 

func (a *MembershipService) Initialize(byAddress []string, byName []string)

Initialize ... This function is to initialize the active members list arguments -byAddress: list of node ip addresses of all members -byName: list of node name including ip address and a timestamp of all members

func (*MembershipService) Len 

func (a *MembershipService) Len() int

Len ... This function is to return total active members on the group

func (*MembershipService) Report 

func (a *MembershipService) Report()

Report ... report current active nodes on the membership

func (*MembershipService) SetConfiguration 

func (a *MembershipService) SetConfiguration(config Configuration)

SetConfiguration ... This function is to initialize the configuration

func (*MembershipService) Start 

func (a *MembershipService) Start(init bool)

Start ... start membership service Arguments - init: boolean. It will start a new membership group if init is true. Otherwise, it will send joining request to the gateway nodes

func (*MembershipService) Stop 

func (a *MembershipService) Stop()

Stop ... stop membership service

type MonitorPacket 

type MonitorPacket struct {
	NodeAddress      string
	LastTimeReceived time.Time
	Count            int32
}

MonitorPacket ...Heartbeat package

type TemporaryFs533File 

type TemporaryFs533File struct {
	Fs533FileName     string //the file being written
	Fs533FileVersion  int32  //the new version of this file
	RequestID         string //id of the write request which is to differentiate write transaction
	TemporaryFileName string //the file will be saved as a temporary file while it is waiting for the commit request
	Fs533FileSize     int32  //file size
}

TemporaryFs533File ... this struct is to describe the temporary file which is created for write preparation

type WriteTransaction 

type WriteTransaction struct {
	Fs533FileName    string   //the file being written
	Fs533FileVersion int32    //the new version of this file
	RequestID        string   //id of the write request which is to differentiate write transaction
	PreferenceNodes  []string //list of its replica nodes
	RegisteredTime   time.Time
}

WriteTransaction ... this struct is to describe a pendding write transaction

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.