- About
- How it works
- Differences from Camo
- Installing pre-built binaries
- Building
- Running
- Running on Heroku
- Securing an installation
- Configuring
- Additional tools
- Changelog
- License
Go version of Camo server.
Camo is a special type of image proxy that proxies non-secure images over SSL/TLS. This prevents mixed content warnings on secure pages.
It works in conjunction with back-end code to rewrite image URLs and sign them with an HMAC.
First you parse the original URL, generate an HMAC signature of it, then encode it, and then place the pieces into the expected format replacing the original image URL.
The client requests the generated URL from Go-Camo. Go-Camo validates the HMAC, decodes the URL, then requests the content and streams it to the client.
+----------+ request +-------------+
| |----------------------------->| |
| | | |
| | | web-app |
| | img src=https://go-camo/url | |
| |<-----------------------------| |
| | +-------------+
| client |
| | https://go-camo/url +-------------+ http://some/img
| |----------------------------->| |--------------->
| | | |
| | | go-camo |
| | img data | | img data
| |<-----------------------------| |<---------------
| | +-------------+
+----------+
Go-Camo supports both hex and base64 encoded urls at the same time.
| encoding | tradeoffs |
|---|---|
| hex | longer, case insensitive, slightly faster (pre go1.6) |
| base64 | shorter, case sensitive, slightly slower (pre go1.6) |
NOTE: As of go1.6 base64 is very close to hex in terms of performance.
BenchmarkHexEncoder-2 500000 3027 ns/op
BenchmarkB64Encoder-2 500000 3234 ns/op
BenchmarkHexDecoder-2 500000 3609 ns/op
BenchmarkB64Decoder-2 500000 3495 ns/op
For examples of url generation, see the examples directory.
While Go-Camo will support proxying HTTPS images as well, for performance reasons you may choose to filter HTTPS requests out from proxying, and let the client simply fetch those as they are. The linked code examples do this.
Note that it is recommended to front Go-Camo with a CDN when possible.
- Go-Camo supports 'Path Format' url format only. Camo's "Query String Format" is not supported.
- Go-Camo supports "allow regex host filters".
- Go-Camo supports client http keep-alives.
- Go-Camo provides native SSL support.
- Go-Camo provides native HTTP/2 support (if built using >=go1.6).
- Go-Camo supports using more than one os thread (via GOMAXPROCS) without the need of multiple instances or additional proxying.
- Go-Camo builds to a static binary. This makes deploying to large numbers of servers a snap.
- Go-Camo supports both Hex and Base64 urls. Base64 urls are smaller, but case sensitive.
- Go-Camo supports HTTP HEAD requests.
- Go-Camo allows custom default headers to be added -- useful for things like adding HSTS headers.
Download the tarball appropriate for your OS/ARCH from releases. Extract, and copy files to desired locations.
Building requires:
* git
* make
* go (version >= 1.5 required, 1.7 recommended)
Building:
# show make targets
$ make
Available targets:
help this help
clean clean up
all build binaries and man pages
test run tests
cover run tests with cover output
build-setup fetch dependencies
build build all
man build all man pages
tar build release tarball
cross-tar cross compile and build release tarballs
# fetch vendor dependencies
$ make build-setup
# build all binaries and man pages
$ make all
# as an alternative to the previous command, build and strip debug symbols.
# this is useful for production, and reduces the resulting file size.
$ make all GOBUILD_LDFLAGS="-s -w"
By default, Go-Camo builds with -tags netgo. However, depending on your
Go version, this will not actually result in Go-Camo using the netgo resolver
unless your Go stdlib is similarly compiled. There are known issues with
using the libc resolver with significant traffic amounts over time. The use of
netgo is recommended. Prior to Go 1.5, to recompile your Go net libraries to
use netgo, do the following as root (or the owner of your GOROOT install)
before building Go-Camo:
$ go clean -i net
$ go install -a -tags netgo std
To confirm that you are using the netgo resolver:
$ make build
$ ldd bin/go-camo
not a dynamic executable
If you are using the libc resolver, you will see something like this instead:
$ make build
$ ldd bin/go-camo
linux-vdso.so.1 => (0x00007fff98fff000)
libpthread.so.0 => /lib64/libpthread.so.0 (0x0000003fb2a00000)
libc.so.6 => /lib64/libc.so.6 (0x0000003fb2600000)
/lib64/ld-linux-x86-64.so.2 (0x0000003fb2200000)
$ go-camo -k "somekey"
Go-Camo does not daemonize on its own. For production usage, it is recommended to launch in a process supervisor, and drop privileges as appropriate.
Examples of supervisors include: daemontools, runit, upstart, launchd, and many more.
For the reasoning behind lack of daemonization, see daemontools/why. In addition, the code is much simpler because of it.
In order to use this on Heroku with the provided Procfile, you need to:
- Create an app specifying the https://github.com/kr/heroku-buildpack-go buildpack
- Set
HMAC_KEYto the key you are using
go-camo will generally do what you tell it to with regard to fetching signed urls. There is some limited support for trying to preventing dns rebinding attacks.
go-camo will attempt to reject any address matching an rfc1918 network block, or a private scope ipv6 address, be it in the url or via resulting hostname resolution. Do note, however, that this does not provide protecton for a network that uses public address space (ipv4 or ipv6), or some of the more exotic ipv6 addresses.
The list of networks currently rejected include...
| Network | Description |
|---|---|
127.0.0.0/8 |
loopback |
169.254.0.0/16 |
ipv4 link local |
10.0.0.0/8 |
rfc1918 |
172.16.0.0/12 |
rfc1918 |
192.168.0.0/16 |
rfc1918 |
::1/128 |
ipv6 loopback |
fe80::/10 |
ipv6 link local |
fec0::/10 |
deprecated ipv6 site-local |
fc00::/7 |
ipv6 ULA |
::ffff:0:0/96 |
IPv4-mapped IPv6 address |
More generally, it is recommended to either:
- Run go-camo on an isolated instance (physical, vlans, firewall rules, etc).
- Run a local resolver for go-camo that returns NXDOMAIN responses for
addresses in blacklisted ranges (for example unbound's
private-addressfunctionality). This is also useful to help prevent dns rebinding in general.
GOCAMO_HMAC- HMAC key to use.
$ go-camo -h
Usage:
go-camo [OPTIONS]
Application Options:
-k, --key= HMAC key
-H, --header= Extra header to return for each response. This option
can be used multiple times to add multiple headers
--stats Enable Stats
--no-log-ts Do not add a timestamp to logging
--allow-list= Text file of hostname allow regexes (one per line)
--max-size= Max response image size (KB) (5120)
--timeout= Upstream request timeout (4s)
--max-redirects= Maximum number of redirects to follow (3)
--no-fk Disable frontend http keep-alive support
--no-bk Disable backend http keep-alive support
--listen= Address:Port to bind to for HTTP (0.0.0.0:8080)
--ssl-listen= Address:Port to bind to for HTTPS/SSL/TLS
--ssl-key= ssl private key (key.pem) path
--ssl-cert= ssl cert (cert.pem) path
-v, --verbose Show verbose (debug) log level output
-V, --version print version and exit
Help Options:
-h, --help Show this help message
If an allow-list file is defined, that file is read and each line converted into a hostname regex. If a request does not match one of the listed host regex, then the request is denied.
If stats flag is provided, then the service will track bytes and clients
served, and offer them up at an http endpoint /status via HTTP GET request.
If the HMAC key is provided on the command line, it will override (if present), an HMAC key set in the environment var.
Additional default headers (headers sent on every reply) can also be set. The
-H, --header argument may be specified many times.
The list of default headers sent are:
X-Content-Type-Options: nosniff
X-XSS-Protection: 1; mode=block
Content-Security-Policy: default-src 'none'; img-src data:; style-src 'unsafe-inline'
As an example, if you wanted to return a Strict-Transport-Security header
by default, you could add this to the command line:
-H "Strict-Transport-Security: max-age=16070400"
Go-Camo includes a couple of additional tools.
The url-tool utility provides a simple way to generate signed URLs from the command line.
$ url-tool -h
Usage:
url-tool [OPTIONS] <decode | encode>
Application Options:
-k, --key= HMAC key
-p, --prefix= Optional url prefix used by encode output
Help Options:
-h, --help Show this help message
Available commands:
decode Decode a url and print result
encode Encode a url and print result
Example usage:
# hex
$ url-tool -k "test" encode -p "https://img.example.org" "http://golang.org/doc/gopher/frontpage.png"
https://img.example.org/0f6def1cb147b0e84f39cbddc5ea10c80253a6f3/687474703a2f2f676f6c616e672e6f72672f646f632f676f706865722f66726f6e74706167652e706e67
$ url-tool -k "test" decode "https://img.example.org/0f6def1cb147b0e84f39cbddc5ea10c80253a6f3/687474703a2f2f676f6c616e672e6f72672f646f632f676f706865722f66726f6e74706167652e706e67"
http://golang.org/doc/gopher/frontpage.png
# base64
$ url-tool -k "test" encode -b base64 -p "https://img.example.org" "http://golang.org/doc/gopher/frontpage.png"
https://img.example.org/D23vHLFHsOhPOcvdxeoQyAJTpvM/aHR0cDovL2dvbGFuZy5vcmcvZG9jL2dvcGhlci9mcm9udHBhZ2UucG5n
$ url-tool -k "test" decode "https://img.example.org/D23vHLFHsOhPOcvdxeoQyAJTpvM/aHR0cDovL2dvbGFuZy5vcmcvZG9jL2dvcGhlci9mcm9udHBhZ2UucG5n"
http://golang.org/doc/gopher/frontpage.png
The simple-server utility has moved to its own repo.
See CHANGELOG.md
Released under the MIT
license. See LICENSE.md
file for details.