README
¶
= caddyv2-upload
:toc:
:source-highlighter: rouge
This repo holds a simple caddyserver v2 upload handler
WARNING: You have to secure the upload URL as the nature of
this handler/plugin is to save uploaded artifacts on the disc.
In this handler are the following modules in use.
* templates
* file_server
* github.com/caddyserver/jsonc-adapter
* upload :-)
== Configuration
=== Parameters
.Parameters
[cols="2,6",options=header]
|===
|Name
|Description
|**dest_dir**
|The directory in which the files will be uploaded
|**file_field_name**
|The field name for the multi-part form upload
|**response_template**
|The response template after a upload was successfully
|**max_form_buffer**
|The maximum buffer size for https://pkg.go.dev/net/http#Request.ParseMultipartForm[ParseMultipartForm]. It accepts all size values supported by https://pkg.go.dev/github.com/dustin/go-humanize#pkg-constants[go-humanize]. Here can also be used a https://caddyserver.com/docs/conventions#placeholders[Placeholder] +
JSON: `"{env.MAXFILESIZE}"` +
Caddyfile: `{$MAXBUFFSIZE}` +
The default size is **1G**.
|**max_form_buffer_int**
|The maximum buffer size for https://pkg.go.dev/net/http#Request.ParseMultipartForm[ParseMultipartForm]. +
The default size is **1G**.
|**max_filesize**
|is the maximum size in bytes allowed for the upload.
It accepts all size values supported by https://pkg.go.dev/github.com/dustin/go-humanize#pkg-constants[go-humanize]. Reads of more bytes will return an error with HTTP status 413. Here can also be used a https://caddyserver.com/docs/conventions#placeholders[Placeholder] +
JSON: `"{env.MAXFILESIZE}"` +
Caddyfile: `{$MAXBUFFSIZE}` +
The default size is **1G**.
|**max_filesize_int**
|is the maximum size in bytes allowed for the upload. Reads of more bytes will return an error with HTTP status 413. +
The default size is **1G**.
|**notify_url**
|After a successful upload will this URL be called. The only supported schema is **https**.
|**notify_method**
|The default method is `GET`. If you need to make a `POST` request please open a feature issue
as for now is a request body handling not implemented
|**insecure**
|This boolean flag configure the `InsecureSkipVerify` in the https://pkg.go.dev/crypto/tls#Config[tls-config] .
Default is false which implies that a valid CA must be used
|**capath**
|This is the Parameter where you can define the CA filename for the **notify_url**.
|===
=== JSON
Because I prefer the https://caddyserver.com/docs/json/[JSON Config ]
will I write here the configuration snipplet in JSON Syntax.
[source,json]
----
"handle": [
{
"MyTlsSetting": {},
"dest_dir": "upload", <1>
"handler": "upload",
"max_filesize": "5GB", <2>
"max_form_buffer_int": 1000000, <3>
"response_template": "upload-resp.txt" <4>
}
]
----
<1> Destination Directory on the Server site
<2> Maximal possible upload size
<3> Maximal buffer for uploading
<4> the response template which will be used for response after upload
A full working example is in
`docker-files/opt/webroot/config/Caddyfile-upload.json`
=== Caddyfile
Here a example Caddyfile which expects that the environment variable
`APPPORT` is set.
[source]
----
{
order upload before file_server
log {
level DEBUG
}
}
{$APPPORT} {
root .
file_server browse
templates
@mypost method POST
upload @mypost {
dest_dir tmp-upl
max_form_buffer 1G
max_filesize 4MB
response_template templates/upload-resp-template.txt
}
log {
output file access.log
}
}
----
== build
=== local
[source,shell]
---
xcaddy build --with github.com/kirsch33/realip \
--with github.com/git001/caddyv2-upload
---
=== docker
[source,shell]
---
buildah bud --tag caddyv2-upload .
# or
docker build --tag caddyv2-upload .
---
== run
=== cli
[source,shell]
---
APPPORT=:2011 ./caddy run \
-config Caddyfile-upload.json
---
=== docker
You can get this image from docker hub
The default listen port must be defined with this variable
`APPPORT=:2011`
https://hub.docker.com/r/me2digital/caddyv2-upload
[source,shell]
---
podman run --rm --network host --name caddy-test \
--env APPPORT=:8888 -it \
docker.io/me2digital/caddyv2-upload:latest
# or
docker run --name caddy-test --rm \
docker.io/me2digital/caddyv2-upload:latest
---
== example cli
When you run the Image with port 8888 can you use curl or any other
tool to post (upload) files
It's not necessary to use `-X POST` as written in this Blog post
https://daniel.haxx.se/blog/2015/09/11/unnecessary-use-of-curl-x/[UNNECESSARY USE OF CURL -X]
Here a example call with curl
[source,shell]
----
curl -v --form myFile=@README.adoc http://localhost:8888/templates/upload-template.html
* Trying 127.0.0.1:8888...
* TCP_NODELAY set
* Connected to localhost (127.0.0.1) port 8888 (#0)
> POST /templates/upload-template.html HTTP/1.1
> Host: localhost:8888
> User-Agent: curl/7.68.0
> Accept: */*
> Content-Length: 2492
> Content-Type: multipart/form-data; boundary=------------------------58b770bc61c0e691
> Expect: 100-continue
>
* Mark bundle as not supporting multiuse
< HTTP/1.1 100 Continue
* We are completely uploaded and fine
* Mark bundle as not supporting multiuse
< HTTP/1.1 200 OK
< Accept-Ranges: bytes
< Content-Length: 299
< Etag: "rbb1gx8b"
< Last-Modified: Tue, 03 May 2022 11:34:09 GMT
< Server: Caddy
< Date: Thu, 19 May 2022 21:45:07 GMT
<
http.request.uri.path: {{placeholder "http.request.uri.path"}}
http.request.uuid {{placeholder "http.request.uuid" }}
http.request.host {{placeholder "http.request.host" }}
http.upload.filename: {{placeholder "http.upload.filename"}}
http.upload.filesize: {{placeholder "http.upload.filesize"}}
----
== Background informations
The **max_form_buffer** paramater will be directly passed to https://cs.opensource.google/go/go/+/refs/tags/go1.18.2:src/mime/multipart/formdata.go;l=34;drc=7791e934c882fd103357448aee0fd577b20013ce[readForm] function and is used to check if the uploaded file should be saved temporarly on disk or keep it in the memory. This have dicret impact into the performance and disk usage of that module. Keep in mind when this paramter is low and the upload is a big file then will be there a lot of disk io. +
INFO: The observation from https://github.com/etherwvlf in issue https://github.com/git001/caddyv2-upload/issues/2[Memory issues on large uploads] was that the initial memory usage is 7-8 times higher then the configured **max_form_buffer** size.
Documentation
¶
Index ¶
- Constants
- type Upload
- func (Upload) CaddyModule() caddy.ModuleInfo
- func (u *Upload) Provision(ctx caddy.Context) error
- func (u Upload) SendNotify(requuid string) error
- func (u Upload) ServeHTTP(w http.ResponseWriter, r *http.Request, next caddyhttp.Handler) error
- func (u *Upload) UnmarshalCaddyfile(d *caddyfile.Dispenser) error
- func (u *Upload) Validate() error
Constants ¶
View Source
const (
Version = "0.9"
)
Variables ¶
This section is empty.
Functions ¶
This section is empty.
Types ¶
type Upload ¶
type Upload struct {
DestDir string `json:"dest_dir,omitempty"`
FileFieldName string `json:"file_field_name,omitempty"`
MaxFilesize int64 `json:"max_filesize_int,omitempty"`
MaxFilesizeH string `json:"max_filesize,omitempty"`
MaxFormBuffer int64 `json:"max_form_buffer_int,omitempty"`
MaxFormBufferH string `json:"max_form_buffer,omitempty"`
ResponseTemplate string `json:"response_template,omitempty"`
NotifyURL string `json:"notify_url,omitempty"`
NotifyMethod string `json:"notify_method,omitempty"`
MyTlsSetting struct {
InsecureSkipVerify bool `json:"insecure,omitempty"`
CAPath string `json:"capath,omitempty"`
}
// contains filtered or unexported fields
}
Middleware implements an HTTP handler that writes the uploaded file to a file on the disk.
func (Upload) CaddyModule ¶
func (Upload) CaddyModule() caddy.ModuleInfo
CaddyModule returns the Caddy module information.
func (Upload) SendNotify ¶
func (*Upload) UnmarshalCaddyfile ¶
UnmarshalCaddyfile implements caddyfile.Unmarshaler.
Source Files
Click to show internal directories.
Click to hide internal directories.