Documentation
¶
Overview ¶
Package cloudio is a wrapper for godoc.org/gocloud.dev/blob emulating os.WriteFile and os.ReadFile.
It is zero config; either saving to local ./app-bucket/ or to appenginge bucket <appID>, depending on environment variables.
The zero configuration is important, cause we load the *actual* configuration file with this package, and want to avoid circular trouble or bootstrap hell.
MarshalWriteFile and ReadFileUnmarshal incorporate JSON serialization and deserialization.
Open() is similar to file.Open
r, err := file.Open("name")
but deviates in that is also returns a bucket closer func.
OpenAny() is just a wrapper arond Open() searching in various subdirectories.
ServeFileBulk() serves files to via http ServeFileStream() serves large files as stream
Index ¶
- func Attrs(fpth string) (attrs *blob.Attributes, err error)
- func CreateNotExist(err error) error
- func Delete(fileName string) error
- func IsNotExist(err error) bool
- func MarshalWriteFile(intf interface{}, fileName string) error
- func Md5Str(s string) string
- func Open(fileName string) (r io.ReadCloser, bucketClose func() error, err error)
- func OpenAny(fileName string, optSubdirs ...string) (r io.ReadCloser, bucketClose func() error, err error)
- func ReadDir(prefix string) (*[]*blob.ListObject, error)
- func ReadFile(fileName string) (bts []byte, err error)
- func ReadFileUnmarshal(fileName string, intf interface{}) (err error)
- func RenderStaticContent(w io.Writer, subPth, site, lang, packageDocPrefix string) error
- func ServeFileBulk(w http.ResponseWriter, req *http.Request)
- func ServeFileStream(w http.ResponseWriter, req *http.Request)
- func WriteFile(fileName string, r io.Reader, perm os.FileMode) (err error)
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
func Attrs ¶
func Attrs(fpth string) (attrs *blob.Attributes, err error)
Attrs retrieves the attributes from a path; though file size seems the only use case.
func CreateNotExist ¶
CreateNotExist returns a "not exists" - similar to io package behaviour.
func IsNotExist ¶
IsNotExist function for cloudio
we could check err := errors.As(..., &combiErr{}) and then peform
errors.Is(err, cloudStoreErr1) errors.Is(err, cloudStoreErr2)
func MarshalWriteFile ¶
MarshalWriteFile is like WriteFile - but marshals to JSON first; allocates contents of intf into []byte slice
func Open ¶
func Open(fileName string) (r io.ReadCloser, bucketClose func() error, err error)
Open a blob/file fileName in default bucket. Returns ReadCloser for the blob/file and bucketCloser() for the underlying bucket
No memory allocation ¶
Example
fileName := "config.json"
r, bucketClose, err := cloudio.Open(fileName)
if err != nil {
log.Fatalf("Error opening writer to %v: %v", fileName, err)
}
defer func() {
err := r.Close()
if err != nil {
log.Printf("Error closing writer to bucket to %v: %v", fileName, err)
}
}()
defer func() {
err := bucketClose()
if err != nil {
log.Printf("Error closing bucket of writer to %v: %v", fileName, err)
}
}()
func OpenAny ¶
func OpenAny(fileName string, optSubdirs ...string) (r io.ReadCloser, bucketClose func() error, err error)
OpenAny tries Open(fileName) for any subdirs and then the appdir "."
func ReadDir ¶
func ReadDir(prefix string) (*[]*blob.ListObject, error)
ReadDir is similar to os.ReadDir but returns ListObjects instead of FileInfos; prefix is the path to search into; returned keys will nevertheless consist of path . path.Separator . fileName; under windows we might have to
o.Key = strings.ReplaceAll(o.Key, "\\", "/")
On windows, `prefix` directory itself is not returned On appengine, `prefix` directory itself is returned as well
func ReadFile ¶
ReadFile is the cousin of os.ReadFile Memory allocation for the file contents.
Open() is another version returning a reader but also an io.ReadCloser and an io.Closer - for bucket and reader.
func ReadFileUnmarshal ¶
ReadFileUnmarshal is like ReadFile - but unmarshals from JSON afterwards; allocates contents of file into []byte slice
func RenderStaticContent ¶
RenderStaticContent writes the content of subPth into w; *.md files are rendered to HTML; *.html files only get URLs rewriting; static files reside in ./app-bucket/content; files may be differentiated by /[site]/[lang]/subPth subPth is a partial path plus filename
func ServeFileBulk ¶
func ServeFileBulk(w http.ResponseWriter, req *http.Request)
ServeFileBulk writes a binary file into the response; it's inner body is similar to ReadFile; files > 128 kB are offloaded to ServeFileStream() at hugely reduced memory consumption. As ServeFileStream() can not read session data, since the session middleware brings a buffered writer, we need to authorize the hand off with a hash
func ServeFileStream ¶
func ServeFileStream(w http.ResponseWriter, req *http.Request)
ServeFileStream writes a binary file into the response; it's inner body is similar to ReadFile; the URL for this handler must be exempted from session middleware and logAndRecover middleware - anything buffering the response. URL GET parameter hijack [true,false] governs the way of flushing; ordinary flushing requires middleware free unbuffered http.ResponseWriter and after each write a
flusher, ok := w.(http.Flusher) ...flusher.Flush()
instead, we use
w ... := stream.NewFlushable(w)
with auto flushing; however, stream.NewFlushable() *hijacks* the connection; preempting middleware but also requiring explicit headers and
fmt.Fprintf(w, "HTTP/1.1 200 OK\n")
URL GET parameter file-size must contain the correct file size; URL GET parameter h (hask) must contain a checksum;
Types ¶
This section is empty.
Source Files