How to write log to file

Question

I'm trying to write to a log file with Go.

I have tried several approaches, all of which have failed. This is what I have tried:

func TestLogging(t *testing.T) {
    if !FileExists("logfile") {
        CreateFile("logfile")
    }
    f, err := os.Open("logfile")
    if err != nil {
        t.Fatalf("error: %v", err)
    }

    // attempt #1
    log.SetOutput(io.MultiWriter(os.Stderr, f))
    log.Println("hello, logfile")

    // attempt #2
    log.SetOutput(io.Writer(f))
    log.Println("hello, logfile")

    // attempt #3
    log.SetOutput(f)
    log.Println("hello, logfile")
}

func FileExists(name string) bool {
    if _, err := os.Stat(name); err != nil {
       if os.IsNotExist(err) {
            return false
        }
    }
    return true
}

func CreateFile(name string) error {
    fo, err := os.Create(name)
    if err != nil {
        return err
    }
    defer func() {
        fo.Close()
    }()
    return nil
}

The log file gets created, but nothing ever gets printed or appended to it. Why?

If you deploy your program in Linux you can just write your log to std output then pipe the output to a file like: *./program 2>&1 | tee logs.txt*. There must be some other way in other system. — nvcnvn, Nov 14 '13 at 02:15

score 215 · Accepted Answer · edited Mar 29 '18 at 04:03

215

os.Open() must have worked differently in the past, but this works for me:

f, err := os.OpenFile("testlogfile", os.O_RDWR | os.O_CREATE | os.O_APPEND, 0666)
if err != nil {
    log.Fatalf("error opening file: %v", err)
}
defer f.Close()

log.SetOutput(f)
log.Println("This is a test log entry")

Based on the Go docs, os.Open() can't work for log.SetOutput, because it opens the file "for reading:"

func Open

func Open(name string) (file *File, err error) Open opens the named file for reading. If successful, methods on the returned file can be used for reading; the associated file descriptor has mode O_RDONLY. If there is an error, it will be of type *PathError.

EDIT

Moved defer f.Close() to after if err != nil check

edited Mar 29 '18 at 04:03

derFunk

1,587
2
20
31

answered Nov 13 '13 at 22:56

Allison A

5,575
6
28
32

9

Do not defer Close before checking err for nil! – Volker Nov 14 '13 at 08:56
It's not activity actually harmful to close in all cases iirc. That isn't true for all types, though. – Dustin Nov 14 '13 at 09:45
2

@Dustin `f` might be `nil`, which would result in a panic. So checking for `err` before deferring the call is advisable. – nemo Nov 14 '13 at 15:00
@AllisonA care to explain why `Open` won't work with `log.SetOutput`? – nemo Nov 14 '13 at 15:04
@nemo because os.Open() is O_RDONLY (readonly cursor), and for the log to write it needs to be writeable. RE the defer, I was assuming that if err was not nil, that f might still have a value, is that not true? – Allison A Nov 14 '13 at 20:16
@AllisonA ah, sorry, missed the read-only part, now it makes sense :). Re. defer, it is common to return a nil-value when an error occurs as it is often impossible to return a valid object (due to that error). So you can't rely on calling anything on the returned value when err is non-nil. – nemo Nov 14 '13 at 20:25
2

The safer permissions are 0644 or even 0664 to allow user read/write, user and group read/write, and in both cases disallow everyone write. – Jonathan Jun 10 '16 at 16:22
`f.Close` returns an error value that isn't being checked. Would it be better to wrap `f.Close` in some function that checks the error? – Parm Aug 12 '20 at 21:06
I'm using a very similar method to this. I don't want to keep logs indefinitely though. Do you have a way to age off older logs? I'm aware that you can't just delete lines from the file – Gamora Mar 15 '22 at 13:51

score 54 · Answer 2 · answered Jul 30 '16 at 15:21

54

I prefer the simplicity and flexibility of the 12 factor app recommendation for logging. To append to a log file you can use shell redirection. The default logger in Go writes to stderr (2).

./app 2>> logfile

See also: http://12factor.net/logs

answered Jul 30 '16 at 15:21

Philip Nelson

849
7
7

1

wont be a good practice when you want to daemonize things, esp with start-tsop-daemon – Shrey Sep 14 '16 at 09:03
3

@Shrey Systemd could easily take care of logging, as well as about start-stop functions. – WarGasm May 16 '17 at 00:13
Despite this is a good practice or not, this is the type of logging that I have been looking for in Golang. Thanks for sharing this! – addicted Jun 17 '19 at 13:54
Is there something similar under windows? – surfmuggle Sep 02 '19 at 21:17
Was like `$ cd /etc/systemd/system` `$ sudo vi app.service ExecStart=/bin/bash -c 'sudo go run main.go >> /home/ubuntu/go/src/html_menu_1/logfile' `Me NOT work `Ubuntu 18.04.3` – Ryosuke Hujisawa Feb 20 '20 at 23:07

score 43 · Answer 3 · answered Aug 01 '18 at 07:56

I usually print the logs on screen and write into a file as well. Hope this helps someone.

f, err := os.OpenFile("/tmp/orders.log", os.O_RDWR|os.O_CREATE|os.O_APPEND, 0666)
if err != nil {
    log.Fatalf("error opening file: %v", err)
}
defer f.Close()
wrt := io.MultiWriter(os.Stdout, f)
log.SetOutput(wrt)
log.Println(" Orders API Called")

score 12 · Answer 4 · answered Nov 06 '17 at 14:01

This works for me

created a package called logger.go

package logger

import (
  "flag"
  "os"
  "log"
  "go/build"
)

var (
  Log      *log.Logger
)


func init() {
    // set location of log file
    var logpath = build.Default.GOPATH + "/src/chat/logger/info.log"

   flag.Parse()
   var file, err1 = os.Create(logpath)

   if err1 != nil {
      panic(err1)
   }
      Log = log.New(file, "", log.LstdFlags|log.Lshortfile)
      Log.Println("LogFile : " + logpath)
}

import the package wherever you want to log e.g main.go

package main

import (
   "logger"
)

const (
   VERSION = "0.13"
 )

func main() {

    // time to use our logger, print version, processID and number of running process
    logger.Log.Printf("Server v%s pid=%d started with processes: %d", VERSION, os.Getpid(),runtime.GOMAXPROCS(runtime.NumCPU()))

}

Afaik this leaves an open file descriptor open for `file`. (Missing `defer file.Close()` after err check) Is that a concern? — Dean Coakley, Mar 30 '21 at 17:14

score 10 · Answer 5 · answered Nov 24 '18 at 13:52

10

If you run binary on linux machine you could use shell script.

overwrite into a file

./binaryapp > binaryapp.log

append into a file

./binaryapp >> binaryapp.log

overwrite stderr into a file

./binaryapp &> binaryapp.error.log

append stderr into a file

./binaryapp &>> binalyapp.error.log

it can be more dynamic using shell script file.

answered Nov 24 '18 at 13:52

Adzimzf

181
2
11

Nice to know, how do we override stderr to log. – impossible Jan 25 '19 at 02:26

openwonk · Answer 6 · 2018-07-25T16:19:06.797

6

Declare up top in your global var so all your processes can access if needed.

package main

import (
    "log"
    "os"
)
var (
    outfile, _ = os.Create("path/to/my.log") // update path for your needs
    l      = log.New(outfile, "", 0)
)

func main() {
    l.Println("hello, log!!!")
}

edited Jul 25 '18 at 16:19

answered May 02 '18 at 03:36

openwonk

14,023
7
43
39

Hey @CostaHuang, please leave detailed feedback. Thanks – openwonk Jul 20 '18 at 23:16
@CostaHuang, I just ran my code snippet and it works. – openwonk Jul 20 '18 at 23:25
Hi @openwonk, I have tested again and it did not work on my computer. My version is `go version go1.10.2 windows/amd64`, what's yours? – Costa Huang Jul 24 '18 at 15:28
@CostaHuang, I just ran example with same set up as you. The example assumes that you already have a folder structure set up. There are easy ways to check for this, however my goal with example is to show how relatively simple writing to a log file is. Change your code to `outfile, _ = os.Create("my.log")` and it will work as expected. – openwonk Jul 25 '18 at 16:23
Your code works. I was using `outfile, _ = os.Create("./path/to/my.log")`. Somehow I had the expectation that the code will create the `path/to` folders and the `my.log` file, but apparently it didn't work. I would suggest that you modify your answer to be `outfile, _ = os.Create("./my.log")`. That way we clearly know that it's creating a log in the current folder. – Costa Huang Jul 25 '18 at 21:46

score 5 · Answer 7 · answered Mar 13 '17 at 13:07

The default logger in Go writes to stderr (2). redirect to file

import ( 
    "syscall"
    "os" 
 )
func main(){
  fErr, err = os.OpenFile("Errfile", os.O_APPEND|os.O_WRONLY|os.O_CREATE, 0600)
  syscall.Dup2(int(fErr.Fd()), 1) /* -- stdout */
  syscall.Dup2(int(fErr.Fd()), 2) /* -- stderr */

}

score 5 · Answer 8 · answered Aug 30 '19 at 00:02

Building on Allison and Deepak's answer, I started using logrus and really like it:

var log = logrus.New()

func init() {

    // log to console and file
    f, err := os.OpenFile("crawler.log", os.O_RDWR|os.O_CREATE|os.O_APPEND, 0666)
    if err != nil {
        log.Fatalf("error opening file: %v", err)
    }
    wrt := io.MultiWriter(os.Stdout, f)

    log.SetOutput(wrt)
}

I have a defer f.Close() in the main function

score 0 · Answer 9 · answered Mar 21 '19 at 09:48

I'm writing logs to the files, which are generate on daily basis (per day one log file is getting generated). This approach is working fine for me :

var (
    serverLogger *log.Logger
)

func init() {
    // set location of log file
    date := time.Now().Format("2006-01-02")
    var logpath = os.Getenv(constant.XDirectoryPath) + constant.LogFilePath + date + constant.LogFileExtension
    os.MkdirAll(os.Getenv(constant.XDirectoryPath)+constant.LogFilePath, os.ModePerm)
    flag.Parse()
    var file, err1 = os.OpenFile(logpath, os.O_RDWR|os.O_CREATE|os.O_APPEND, 0666)

    if err1 != nil {
        panic(err1)
    }
    mw := io.MultiWriter(os.Stdout, file)
    serverLogger = log.New(mw, constant.Empty, log.LstdFlags)
    serverLogger.Println("LogFile : " + logpath)
}

// LogServer logs to server's log file
func LogServer(logLevel enum.LogLevel, message string) {
    _, file, no, ok := runtime.Caller(1)
    logLineData := "logger_server.go"
    if ok {
        file = shortenFilePath(file)
        logLineData = fmt.Sprintf(file + constant.ColonWithSpace + strconv.Itoa(no) + constant.HyphenWithSpace)
    }
    serverLogger.Println(logLineData + logLevel.String() + constant.HyphenWithSpace + message)
}

// ShortenFilePath Shortens file path to a/b/c/d.go tp d.go
func shortenFilePath(file string) string {
    short := file
    for i := len(file) - 1; i > 0; i-- {
        if file[i] == constant.ForwardSlash {
            short = file[i+1:]
            break
        }
    }
    file = short
    return file
}

"shortenFilePath()" method used to get the name of the file from full path of file. and "LogServer()" method is used to create a formatted log statement (contains : filename, line number, log level, error statement etc...)

score 0 · Answer 10 · answered Nov 06 '19 at 03:56

To help others, I create a basic log function to handle the logging in both cases, if you want the output to stdout, then turn debug on, its straight forward to do a switch flag so you can choose your output.

func myLog(msg ...interface{}) {
    defer func() { r := recover(); if r != nil { fmt.Print("Error detected logging:", r) } }()
    if conf.DEBUG {
        fmt.Println(msg)
    } else {
        logfile, err := os.OpenFile(conf.LOGDIR+"/"+conf.AppName+".log", os.O_RDWR | os.O_CREATE | os.O_APPEND,0666)
        if !checkErr(err) {
            log.SetOutput(logfile)
            log.Println(msg)
        }
        defer logfile.Close()
    }
}

score 0 · Answer 11 · answered Aug 12 '20 at 13:27

maybe this will help you (if the log file exists use it, if it does not exist create it):

package main

import (
    "flag"
    "log"
    "os"
)
//Se declara la variable Log. Esta será usada para registrar los eventos.
var (
    Log *log.Logger = Loggerx()
)

func Loggerx() *log.Logger {
    LOG_FILE_LOCATION := os.Getenv("LOG_FILE_LOCATION")
        //En el caso que la variable de entorno exista, el sistema usa la configuración del docker.
    if LOG_FILE_LOCATION == "" {
        LOG_FILE_LOCATION = "../logs/" + APP_NAME + ".log"
    } else {
        LOG_FILE_LOCATION = LOG_FILE_LOCATION + APP_NAME + ".log"
    }
    flag.Parse()
        //Si el archivo existe se rehusa, es decir, no elimina el archivo log y crea uno nuevo.
    if _, err := os.Stat(LOG_FILE_LOCATION); os.IsNotExist(err) {
        file, err1 := os.Create(LOG_FILE_LOCATION)
        if err1 != nil {
            panic(err1)
        }
                //si no existe,se crea uno nuevo.
        return log.New(file, "", log.Ldate|log.Ltime|log.Lshortfile)
    } else {
                //si existe se rehusa.
        file, err := os.OpenFile(LOG_FILE_LOCATION, os.O_CREATE|os.O_APPEND|os.O_WRONLY, 0666)
        if err != nil {
            panic(err)
        }
        return log.New(file, "", log.Ldate|log.Ltime|log.Lshortfile)
    }
}

For more detail: https://su9.co/9BAE74B

How to write log to file

11 Answers11

Linked