How to embed files into Go binaries

check packr, its quite friendly to use

package main

import (
  "net/http"

  "github.com/gobuffalo/packr"
)

func main() {
  box := packr.NewBox("./templates")

  http.Handle("/", http.FileServer(box))
  http.ListenAndServe(":3000", nil)
}

Was looking for the same thing and came across esc: Embedding Static Assets in Go (by 19 Nov 2014) where author, Matt Jibson, is evaluating 3 other popular packages that claims to do file embedding:

  1. rakyll/statik
  2. jteeuwen/go-bindata (and the new official go-bindata/go-bindata and another improved one kevinburke/go-bindata)
  3. GeertJohan/go.rice

and explain why he eventually come up with his own package:

  1. mjibson/esc

So after briefly trying them all (in that order) I've naturally settled on Matt's esc as it was the only one that was working out of the box with necessary for me functionality (HTTPS service in a single executable), namely:

  1. Can take some directories and recursively embed all files in them in a way that was compatible with http.FileSystem
  2. Can optionally be disabled for use with the local file system for local development without changing the client's code
  3. Will not change the output file on subsequent runs has reasonable-sized diffs when files changed
  4. Capable of doing the work via //go:generate instead of forcing you to manually write additional Go code

The point #2 was important for me and the rest of the packages for one reason or another didn't work out that well.

From esc's README:

esc embeds files into go programs and provides http.FileSystem interfaces to them.

It adds all named files or files recursively under named directories at the path specified. The output file provides an http.FileSystem interface with zero dependencies on packages outside the standard library.


Use go-bindata. From the README:

This tool converts any file into managable Go source code. Useful for embedding binary data into a go program. The file data is optionally gzip compressed before being converted to a raw byte slice.


===== Edit Jan 2021 =====

Starting with Go 1.16, released in Feb 2021, you can use the go:embed directive:

import "embed"

//go:embed hello.txt
var s string
print(s)

//go:embed hello.txt
var b []byte
print(string(b))

//go:embed hello.txt
var f embed.FS
data, _ := f.ReadFile("hello.txt")
print(string(data))

===== Original answer ======

Since Go 1.4, you can use go generate if you need more flexibility.

If you have more than one text file or the text file may change you might not want to hardcode the text file but include it at compile time.

If you have the following files:

main.go
scripts/includetxt.go
a.txt
b.txt

And want to have access to the contents of all .txt files in main.go, you can include a special comment containing a go generate command.

main.go

package main

import "fmt"

//go:generate go run scripts/includetxt.go

func main() {
    fmt.Println(a)
    fmt.Println(b)
}

The go generate command will run the script after go:generate. In this case it runs a go script which reads all text files and outputs them as string literals into a new file. I skipped the error handling for shorter code.

script/includetxt.go

package main

import (
    "io"
    "io/ioutil"
    "os"
    "strings"
)

// Reads all .txt files in the current folder
// and encodes them as strings literals in textfiles.go
func main() {
    fs, _ := ioutil.ReadDir(".")
    out, _ := os.Create("textfiles.go")
    out.Write([]byte("package main \n\nconst (\n"))
    for _, f := range fs {
        if strings.HasSuffix(f.Name(), ".txt") {
            out.Write([]byte(strings.TrimSuffix(f.Name(), ".txt") + " = `"))
            f, _ := os.Open(f.Name())
            io.Copy(out, f)
            out.Write([]byte("`\n"))
        }
    }
    out.Write([]byte(")\n"))
}

To compile all .txt files into your exectutable:

$ go generate
$ go build -o main

Now your directory structure will look like:

main.go
main
scripts/includetxt.go
textfiles.go
a.txt
b.txt

Where textfiles.go was generated by go generate and script/includetxt.go

textfiles.go

package main 

const (
a = `hello`
b = `world`
)

And running main gives

$ ./main
hello
world

This will work fine as long as you're encoding UTF8 encoded files. If you want to encode other files you have the full power of the go language (or any other tool) to do so. I used this technique to hex encode png:s into a single executable. That requires a minor change to includetxt.go.

Tags:

Binaries

Go

Recent Posts