sftp

package
v6.25.1 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jan 9, 2025 License: MIT Imports: 21 Imported by: 1

Documentation

Overview

Package sftp - SFTP VFS implementation.

Usage

Rely on github.com/c2fo/vfs/v6/backend 

  import(
	  "github.com/c2fo/vfs/v6/backend"
	  "github.com/c2fo/vfs/v6/backend/sftp"
  )

  func UseFs() error {
	  fs := backend.Backend(sftp.Scheme)
	  ...
  }

Or call directly: 

  import "github.com/c2fo/vfs/v6/backend/sftp"

  func DoSomething() {
	  fs := sftp.NewFileSystem()

	  location, err := fs.NewLocation("myuser@server.com:22", "/some/path/")
	  if err != nil {
		 #handle error
	  }
	  ...
  }

sftp can be augmented with some implementation-specific methods. Backend returns vfs.FileSystem interface so it would have to be cast as sftp.FileSystem to use them.

These methods are chainable: (*FileSystem) WithClient(client interface{}) *FileSystem (*FileSystem) WithOptions(opts vfs.Options) *FileSystem 

  func DoSomething() {
	  // cast if fs was created using backend.Backend().  Not necessary if created directly from sftp.NewFileSystem().
	  fs := backend.Backend(sftp.Scheme)
	  fs = fs.(*sftp.FileSystem)

	  // to pass specific client
	  sshClient, err := ssh.Dial("tcp", "myuser@server.com:22", &ssh.ClientConfig{
		  User:            "someuser",
		  Auth:            []ssh.AuthMethod{ssh.Password("mypassword")},
		  HostKeyCallback: ssh.InsecureIgnoreHostKey,
	  })
	  #handle error
	  client, err := _sftp.NewClient(sshClient)
	  #handle error

	  fs = fs.WithClient(client)

	  // to pass in client options. See Options for more info.  Note that changes to Options will make nil any client.
	  // This behavior ensures that changes to settings will get applied to a newly created client.
	  fs = fs.WithOptions(
		  sftp.Options{
			  KeyFilePath:   "/home/Bob/.ssh/id_rsa",
			  KeyPassphrase: "s3cr3t",
			  KnownHostsCallback: ssh.InsecureIgnoreHostKey,
		  },
	  )

	  location, err := fs.NewLocation("myuser@server.com:22", "/some/path/")
	  #handle error

	  file := location.NewFile("myfile.txt")
	  #handle error

	  _, err := file.Write([]bytes("some text")
	  #handle error

	  err := file.Close()
	  #handle error

  }

Authentication

Authentication, by default, occurs automatically when Client() is called. Since user is part of the URI authority section (Volume), auth is handled slightly differently than other vfs backends.

A client is initialized lazily, meaning we only make a connection to the server at the last moment so we are free to modify options until then. The authenticated session is closed any time WithOption(), WithClient(), or Close() occurs. Currently, that means that closing a file belonging to an fs will break the connection of any other open file on the same fs.

USERNAME

User may only be set in the URI authority section (Volume in vfs parlance). 

 scheme             host
 __/             ___/____  port
/  \            /        \ /\
sftp://someuser@server.com:22/path/to/file.txt
       \____________________/ \______________/
       \______/       \               \
           /     authority section    path
     username       (Volume)

sftp vfs backend accepts either a password or an ssh key, with or without a passphrase.

PASSWORD/PASSPHRASE

Passwords may be passed via Options.Password or via the environmental variable VFS_SFTP_PASSWORD.

SSH keys may be passed via Options.KeyFilePath and (optionally) Options.KeyPassphrase. They can also be passed via environmental variables VFS_SFTP_KEYFILE and VFS_SFTP_KEYFILE_PASSPHRASE, respectively.

KNOWN HOSTS

Known hosts ensures that the server you're connecting to hasn't been somehow redirected to another server, collecting your info (man-in-the-middle attack). Handling for this can be accomplished via:

  1. Options.KnownHostsString which accepts a string.
  2. Options.KnownHostsFile or environmental variable VFS_SFTP_KNOWN_HOSTS_FILE which accepts a path to a known_hosts file.
  3. Options.KnownHostsCallback which allows you to specify any of the ssh.AuthMethod functions. Environmental variable VFS_SFTP_INSECURE_KNOWN_HOSTS will set this callback function to ssh.InsecureIgnoreHostKey which may be helpful for testing but should not be used in production.
  4. Defaults to trying to find and use <homedir>/.ssh/known_hosts. For unix, system-wide location /etc/ssh/.ssh/known hosts is also checked. SSH doesn't exist natively on Windows and each third-party implementation has a different location for known_hosts. Because of this, no attempt is made to find a system-wide file for Windows. It's better to specify in KnownHostsFile in that case.

OTHER OPTIONS

Passing in multiple host key algorithms, key exchange algorithms is supported - these are specified as string slices. Example: 

fs = fs.WithOptions(
	sftp.Options{
		KeyExchanges: []string{ "diffie-hellman-group-a256", "ecdh-sha2-nistp256" },
		Ciphers: []string{ "aes256-ctr", "aes192-ctr", "aes128-ctr" },
		MACs: []string{ "hmac-sha2-256", "hmac-sha2-512" },
		HostKeyAlgorithms: []string{ "ssh-rsa", "ssh-ed25519" },
		// other settings
	},
  )

FilePermissions

The `FilePermissions` option allows you to specify the file permissions for files created or modified using the SFTP backend. These permissions will override the sftp server or underlying filesystem's umask (default permissions). Permissions should be specified using an octal literal (e.g., `0777` for full read, write, and execute permissions for all users).

Example: 

	fs = fs.WithOptions(
 		sftp.Options{
 			FilePermissions: "0777", // Correctly specify permissions as octal (in string form)
			// other settings
		},
	)

When a file is opened for Write() or Touch()'d, the specified `FilePermissions` will be applied to the file.

AutoDisconnect

When dialing a TCP connection, Go doesn't disconnect for you. This is true even when the connection falls out of scope, and even when garbage collection is forced. The connection must be explicitly closed. Unfortunately, VFS.FileSystem has no explicit close mechanism.

Instead, the SFTP backend will automatically disconnect 10 seconds (default) after connection. This disconnect timer is canceled anytime a server-side request (like list, read, etc) is made. Once the request has completed, a new timer will begin. If the timer expires (because it is not interrupted by any request), the server connection will be closed. Any subsequent server request will first reconnect, perform the request, and start a new disconnect timer.

Options.AutoDisconnect accepts an integer representing the number seconds before disconnecting after being idle. Default value is 10 seconds.

Any server request action using the same underlying FileSystem (and therefore sftp client), will reset the timer. This should be the most desirable behavior. 

func doSFTPStuff() {
	fs := sftp.NewFileSystem()
	loc, err := fs.NewLocation("myuser@server.com:22", "/some/path/")
	file1, _ := loc.NewFile("file1.txt")
	file2, _ := loc.NewFile("file2.txt")
	file1.Touch()                               // "touches" file and starts disconnect timer (default: 10sec)
	_, _ := loc.List()                          // stops timer, does location listing, resets timer to 10 seconds
	file2.Touch()                               // stops timer, "touches" file2, resets timer to 10 seconds
	time.Sleep(time.Duration(15) * time.Second) // pause for 15 seconds, disconnects for server after 10 seconds
	_, _ := loc.List()                          // reconnects, does location listing, starts new disconnect timer
	return
}

func main {
	// call our sftp function
	doSFTPStuff()
	// even though the vfs sftp objects have fallen out of scope, our connection remains UNTIL the timer counts down

	// do more work (that take longer than 10 seconds)
	doOtherTimeConsumingStuff()

	// at some point during the above, the sftp connection will have closed
}

NOTE: AutoDisconnect has nothing to do with "keep alive". Here we're only concerned with releasing resources, not keeping the server from disconnecting us. If that is something you want, you'd have to implement yourself, injecting your own client using WithClient().

Index

Constants

View Source 
const Scheme = "sftp"

Scheme defines the filesystem type.

Variables

This section is empty.

Functions

This section is empty.

Types

type Client 

type Client interface {
	Chmod(path string, mode os.FileMode) error
	Chtimes(path string, atime, mtime time.Time) error
	Create(path string) (*_sftp.File, error)
	MkdirAll(path string) error
	OpenFile(path string, f int) (*_sftp.File, error)
	ReadDir(p string) ([]os.FileInfo, error)
	Remove(path string) error
	Rename(oldname, newname string) error
	Stat(p string) (os.FileInfo, error)
	Close() error
}

Client is an interface to make it easier to test

type File 

type File struct {
	Authority utils.Authority
	// contains filtered or unexported fields
}

File implements vfs.File interface for SFTP fs.

func (*File) Close 

func (f *File) Close() error

Close calls the underlying sftp.File Close, if opened, and clears the internal pointer

func (*File) CopyToFile 

func (f *File) CopyToFile(file vfs.File) (err error)

CopyToFile puts the contents of File into the targetFile passed.

func (*File) CopyToLocation 

func (f *File) CopyToLocation(location vfs.Location) (vfs.File, error)

CopyToLocation creates a copy of *File, using the file's current path as the new file's path at the given location.

func (*File) Delete 

func (f *File) Delete(_ ...options.DeleteOption) error

Delete removes the remote file. Error is returned, if any.

func (*File) Exists 

func (f *File) Exists() (bool, error)

Exists returns a boolean of whether the file exists on the sftp server

func (*File) LastModified 

func (f *File) LastModified() (*time.Time, error)

LastModified returns the LastModified property of sftp file.

func (*File) Location 

func (f *File) Location() vfs.Location

Location returns a vfs.Location at the location of the file. IE: if file is at sftp://someuser@host.com/here/is/the/file.txt the location points to sftp://someuser@host.com/here/is/the/

func (*File) MoveToFile 

func (f *File) MoveToFile(t vfs.File) error

MoveToFile puts the contents of File into the targetFile passed using File.CopyToFile. If the copy succeeds, the source file is deleted. Any errors from the copy or delete are returned. If the given location is also sftp AND for the same user and host, the sftp Rename method is used, otherwise we'll do an io.Copy to the destination file then delete source file.

func (*File) MoveToLocation 

func (f *File) MoveToLocation(location vfs.Location) (vfs.File, error)

MoveToLocation works by creating a new file on the target location then calling MoveToFile() on it.

func (*File) Name 

func (f *File) Name() string

Name returns the path portion of the file's path property. IE: "file.txt" of "sftp://someuser@host.com/some/path/to/file.txt

func (*File) Path 

func (f *File) Path() string

Path returns the directory portion of the file's path. IE: "path/to" of "sftp://someuser@host.com/some/path/to/file.txt

func (*File) Read 

func (f *File) Read(p []byte) (n int, err error)

Read calls the underlying sftp.File Read.

func (*File) Seek 

func (f *File) Seek(offset int64, whence int) (int64, error)

Seek calls the underlying sftp.File Seek.

func (*File) Size 

func (f *File) Size() (uint64, error)

Size returns the size of the remote file.

func (*File) String 

func (f *File) String() string

String implement fmt.Stringer, returning the file's URI as the default string.

func (*File) Touch 

func (f *File) Touch() error

Touch creates a zero-length file on the vfs.File if no File exists. Update File's last modified timestamp. Returns error if unable to touch File.

func (*File) URI 

func (f *File) URI() string

URI returns the File's URI as a string.

func (*File) Write 

func (f *File) Write(data []byte) (res int, err error)

Write calls the underlying sftp.File Write.

type FileSystem 

type FileSystem struct {
	// contains filtered or unexported fields
}

FileSystem implements vfs.FileSystem for the SFTP filesystem.

func NewFileSystem 

func NewFileSystem() *FileSystem

NewFileSystem initializer for fileSystem struct.

func (*FileSystem) Client 

func (fs *FileSystem) Client(authority utils.Authority) (Client, error)

Client returns the underlying sftp client, creating it, if necessary See Overview for authentication resolution

func (*FileSystem) Name 

func (fs *FileSystem) Name() string

Name returns "Secure File Transfer Protocol"

func (*FileSystem) NewFile 

func (fs *FileSystem) NewFile(authority, filePath string, opts ...options.NewFileOption) (vfs.File, error)

NewFile function returns the SFTP implementation of vfs.File.

func (*FileSystem) NewLocation 

func (fs *FileSystem) NewLocation(authority, locPath string) (vfs.Location, error)

NewLocation function returns the SFTP implementation of vfs.Location.

func (*FileSystem) Retry 

func (fs *FileSystem) Retry() vfs.Retry

Retry will return the default no-op retrier. The SFTP client provides its own retryer interface, and is available to override via the sftp.FileSystem Options type.

func (*FileSystem) Scheme 

func (fs *FileSystem) Scheme() string

Scheme return "sftp" as the initial part of a file URI ie: sftp://

func (*FileSystem) WithClient 

func (fs *FileSystem) WithClient(client interface{}) *FileSystem

WithClient passes in an sftp client and returns the filesystem (chainable)

func (*FileSystem) WithOptions 

func (fs *FileSystem) WithOptions(opts vfs.Options) *FileSystem

WithOptions sets options for client and returns the filesystem (chainable)

type Location 

type Location struct {
	Authority utils.Authority
	// contains filtered or unexported fields
}

Location implements the vfs.Location interface specific to sftp fs.

func (*Location) ChangeDir 

func (l *Location) ChangeDir(relativePath string) error

ChangeDir takes a relative path, and modifies the underlying Location's path. The caller is modified by this so the only return is any error. For this implementation there are no errors.

func (*Location) DeleteFile 

func (l *Location) DeleteFile(fileName string, opts ...options.DeleteOption) error

DeleteFile removes the file at fileName path.

func (*Location) Exists 

func (l *Location) Exists() (bool, error)

Exists returns true if the remote SFTP file exists.

func (*Location) FileSystem 

func (l *Location) FileSystem() vfs.FileSystem

FileSystem returns a vfs.fileSystem interface of the location's underlying fileSystem.

func (*Location) List 

func (l *Location) List() ([]string, error)

List calls SFTP ReadDir to list all files in the location's path. If you have many thousands of files at the given location, this could become quite expensive.

func (*Location) ListByPrefix 

func (l *Location) ListByPrefix(prefix string) ([]string, error)

ListByPrefix calls SFTP ReadDir with the location's path modified relatively by the prefix arg passed to the function.

func (*Location) ListByRegex 

func (l *Location) ListByRegex(regex *regexp.Regexp) ([]string, error)

ListByRegex retrieves the filenames of all the files at the location's current path, then filters out all those that don't match the given regex. The resource considerations of List() apply here as well.

func (*Location) NewFile 

func (l *Location) NewFile(filePath string, opts ...options.NewFileOption) (vfs.File, error)

NewFile uses the properties of the calling location to generate a vfs.File (backed by an sftp.File). The filePath argument is expected to be a relative path to the location's current path.

func (*Location) NewLocation 

func (l *Location) NewLocation(relativePath string) (vfs.Location, error)

NewLocation makes a copy of the underlying Location, then modifies its path by calling ChangeDir with the relativePath argument, returning the resulting location. The only possible errors come from the call to ChangeDir, which, for the SFTP implementation doesn't ever result in an error.

func (*Location) Path 

func (l *Location) Path() string

Path returns the path the location references in most SFTP calls.

func (*Location) String 

func (l *Location) String() string

String implement fmt.Stringer, returning the location's URI as the default string.

func (*Location) URI 

func (l *Location) URI() string

URI returns the Location's URI as a string.

func (*Location) Volume 

func (l *Location) Volume() string

Volume returns the Authority the location is contained in.

type Options 

type Options struct {
	Password           string              `json:"password,omitempty"`       // env var VFS_SFTP_PASSWORD
	KeyFilePath        string              `json:"keyFilePath,omitempty"`    // env var VFS_SFTP_KEYFILE
	KeyPassphrase      string              `json:"keyPassphrase,omitempty"`  // env var VFS_SFTP_KEYFILE_PASSPHRASE
	KnownHostsFile     string              `json:"knownHostsFile,omitempty"` // env var VFS_SFTP_KNOWN_HOSTS_FILE
	KnownHostsString   string              `json:"knownHostsString,omitempty"`
	KeyExchanges       []string            `json:"keyExchanges,omitempty"`
	Ciphers            []string            `json:"ciphers,omitempty"`
	MACs               []string            `json:"macs,omitempty"`
	HostKeyAlgorithms  []string            `json:"hostKeyAlgorithms,omitempty"`
	AutoDisconnect     int                 `json:"autoDisconnect,omitempty"` // seconds before disconnecting. default: 10
	KnownHostsCallback ssh.HostKeyCallback // env var VFS_SFTP_INSECURE_KNOWN_HOSTS
	FileBufferSize     int                 `json:"fileBufferSize,omitempty"`  // Buffer Size In Bytes Used with utils.TouchCopyBuffered
	FilePermissions    *string             `json:"filePermissions,omitempty"` // Default File Permissions for new files
}

Options holds sftp-specific options. Currently only client options are used.

func (*Options) GetFileMode added in v6.19.0 

func (o *Options) GetFileMode() (*os.FileMode, error)

GetFileMode converts the FilePermissions string to os.FileMode.

type ReadWriteSeekCloser 

type ReadWriteSeekCloser interface {
	io.ReadWriteSeeker
	io.Closer
}

ReadWriteSeekCloser is a read write seek closer interface representing capabilities needed from std libs sftp File struct.

Source Files

View all Source files

Directories

Path Synopsis

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.