Read template in init or in handler function?

Question

I'm writing a basic server for a website. Now I face a (for me) difficult performance question. Is it better to read the template file in the init() function?

// Initialize all pages of website
func init(){
 indexPageData, err := ioutil.ReadFile("./tpl/index.tpl")
 check(err)
}

Or in the http.HandlerFunc?

func index(w http.ResponseWriter, req *http.Request){
  indexPageData, err := ioutil.ReadFile("./tpl/index.tpl")
  check(err)
  indexPageTpl := template.Must(template.New("index").Parse(string(indexPageData)))
  indexPageTpl.Execute(w, "test")
}

I think in the first example, after the server is started you have no need to access the disk and increase the performance of the request.
But during development I want to refresh the browser and see the new content. That can be done with the second example.

Does someone have a state-of-the-art solution? Or what is the right from the performance point of view?

Answer 1

Let's analyze the performance:

We name your first solution (with slight changes, see below) a and your second solution b.

One request:
a: One disk access
b: One disk access

Ten requests:
a: One disk access
b: Ten disk accesses

10 000 000 requests:
a: One disk access
b: 10 000 000 disk accesses (this is slow)

So, performance is better with your first solution. But what about your concern regarding up-to-date data? From the documentation of func (t *Template) Execute(wr io.Writer, data interface{}) error:

Execute applies a parsed template to the specified data object, writing the output to wr. If an error occurs executing the template or writing its output, execution stops, but partial results may already have been written to the output writer. A template may be executed safely in parallel.

So, what happens is this:

1. You read a template from disk
2. You parse the file into a template
3. You choose the data to fill in the blanks with
4. You Execute the template with that data, the result is written out into an io.Writer

Your data is as up-to-date as you choose it. This has nothing to do with re-reading the template from disk, or even re-parsing it. This is the whole idea behind templates: One disk access, one parse, multiple dynamic end results.

The documentation quoted above tells us another thing:

A template may be executed safely in parallel.

This is very useful, because your http.HandlerFuncs are ran in parallel, if you have multiple requests in parallel.

So, what to do now?
Read the template file once,
Parse the template once,
Execute the template for every request.

I'm not sure if you should read and parse in the init() function, because at least the Must can panic (and don't use some relative, hard coded path in there!) - I would try to do that in a more controlled environment, e.g. provide a function (like New()) to create a new instance of your server and do that stuff in there.

EDIT: I re-read your question and I might have misunderstood you:

If the template itself is still in development then yes, you would have to read it on every request to have an up-to-date result. This is more convenient than to restart the server every time you change the template. For production, the template should be fixed and only the data should change.

Sorry if I got you wrong there.

Answer 2

Never read and parse template files in the request handler in production, that is as bad as it can get (you should like always avoid this). During development it is ok of course.

Read this question for more details:

It takes too much time when using "template" package to generate a dynamic web page to client in golang

You could approach this in multiple ways. Here I list 4 with example implementation.

1. With a "dev mode" setting

You could have a constant or variable telling if you're running in development mode which means templates are not to be cached.

Here's an example to that:

const dev = true

var indexTmpl *template.Template

func init() {
    if !dev { // Prod mode, read and cache template
        indexTmpl = template.Must(template.New("index").ParseFiles(".tpl/index.tpl"))
    }
}

func getIndexTmpl() *template.Template {
    if dev { // Dev mode, always read fresh template
        return template.Must(template.New("index").ParseFiles(".tpl/index.tpl"))
    } else { // Prod mode, return cached template
        return indexTmpl
    }
}

func indexHandler(w http.ResponseWriter, r *http.Request) {
    getIndexTmpl().Execute(w, "test")
}

2. Specify in the request (as a param) if you want a fresh template

When you develop, you may specify an extra URL parameter indicating to read a fresh template and not use the cached one, e.g. http://localhost:8080/index?dev=true

Example implementation:

var indexTmpl *template.Template

func init() {
    indexTmpl = getIndexTmpl()
}

func getIndexTmpl() *template.Template {
    return template.Must(template.New("index").ParseFiles(".tpl/index.tpl"))
}

func indexHandler(w http.ResponseWriter, r *http.Request) {
    t := indexTmpl
    if r.FormValue("dev") != nil {
        t = getIndexTmpl()
    }
    t.Execute(w, "test")
}

3. Decide based on host

You can also check the host name of the request URL, and if it is "localhost", you can omit the cache and use a fresh template. This requires the smallest extra code and effort. Note that you may want to accept other hosts as well e.g. "127.0.0.1" (up to you what you want to include).

Example implementation:

var indexTmpl *template.Template

func init() {
    indexTmpl = getIndexTmpl()
}

func getIndexTmpl() *template.Template {
    return template.Must(template.New("index").ParseFiles(".tpl/index.tpl"))
}

func indexHandler(w http.ResponseWriter, r *http.Request) {
    t := indexTmpl
    if r.URL.Host == "localhost" || strings.HasPrefix(r.URL.Host, "localhost:") {
        t = getIndexTmpl()
    }
    t.Execute(w, "test")
}

4. Check template file last modified

You could also store the last modified time of the template file when it is loaded. Whenever the template is requested, you can check the last modified time of the source template file. If it has changed, you can reload it before executing it.

Example implementation:

type mytempl struct {
    t       *template.Template
    lastmod time.Time
    mutex   sync.Mutex
}

var indexTmpl mytempl

func init() {
    // You may want to call this in init so first request won't be slow
    checkIndexTempl()
}

func checkIndexTempl() {
    nm := ".tpl/index.tpl"
    fi, err := os.Stat(nm)
    if err != nil {
        panic(err)
    }
    if indexTmpl.lastmod != fi.ModTime() {
        // Changed, reload. Don't forget the locking!
        indexTmpl.mutex.Lock()
        defer indexTmpl.mutex.Unlock()
        indexTmpl.t = template.Must(template.New("index").ParseFiles(nm))
        indexTmpl.lastmod = fi.ModTime()
    }
}

func indexHandler(w http.ResponseWriter, r *http.Request) {
    checkIndexTempl()
    indexTmpl.t.Execute(w, "test")
}