Docker Compose logging to Papertrail

Problems:

1. There's a limit on how much you can scroll back in your console to see previous outputs generated by Docker compose.
2. If you deploy your services on many different servers, you need to chase logs on all those servers to debug your services.

Solution: Centralized logging

1. the remote logging service: papertrail.com
2. the logging driver: syslog

Here's how it works

I use Docker version 18.09.7, which means I can use the docker-compose version 3.7 syntax.

In my docker-compose.yml file, under the service I'm logging over to papertrail, I have a logging block, like this:

  logging:
    driver: "syslog"
    options:
      syslog-address: "udp://XXXX.papertrailapp.com:XXXX"
      tag: "{{.Name}}/{{.ID}}"

...Which looks like this, in the context of the service block:

  web:
    build: .
    command: bash -c "rm -f tmp/pids/server.pid && foreman start -f Procfile"
    volumes:
      - ./:/myapp
    ports:
      - "3001:3000"
    depends_on:
      - db
    env_file:
      - .env
    networks:
      - SM
    logging:
      driver: "syslog"
      options:
        syslog-address: "udp://XXXX.papertrailapp.com:XXXX"
        tag: "{{.Name}}/{{.ID}}"

BTW, you see that address (XXXX.papertrailapp.com:XXXX)?

When subscribing to Papertrail, it should give you a specific address just for you, like this (censored):

Just copy that address and put it after the udp:// part, in your docker-compose.yml config.

Run it!

Now, you should be good to go:

user@hostname:~/project-x⟫ docker-compose up
Starting project-x_db_1 ... done
Starting project-x_doc_1 ... done
Starting project-x_web_1 ... done
Starting project-x_ngrok_web_1 ... done
Attaching to project-x_db_1, project-x_doc_1, project-x_web_1, project-x_ngrok_web_1
db_1         |
db_1         | PostgreSQL Database directory appears to contain a database; Skipping initialization
db_1         |
db_1         | 2020-12-31 21:30:48.711 UTC [1] LOG:  starting PostgreSQL [REDACTED] (Debian [REDACTED]) on x86_64-pc-linux-gnu, compiled by gcc (Debian [REDACTED]) [REDACTED], 64-bit
db_1         | 2020-12-31 21:30:48.711 UTC [1] LOG:  listening on IPv4 address "0.0.0.0", port 5432
db_1         | 2020-12-31 21:30:48.711 UTC [1] LOG:  listening on IPv6 address "::", port 5432
db_1         | 2020-12-31 21:30:48.712 UTC [1] LOG:  listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
db_1         | 2020-12-31 21:30:48.727 UTC [25] LOG:  database system was shut down at 2020-12-31 21:22:08 UTC
db_1         | 2020-12-31 21:30:48.733 UTC [1] LOG:  database system is ready to accept connections
doc_1        | [Thu Dec 31 21:30:49.327421 2020] [mpm_prefork:notice] [pid 1] AH00163: Apache/[REDACTED] (Debian) PHP/[REDACTED] configured -- resuming normal operations
doc_1        | [Thu Dec 31 21:30:49.327605 2020] [core:notice] [pid 1] AH00094: Command line: 'apache2 -D FOREGROUND'
web_1        | WARNING: no logs are available with the 'syslog' log driver
ngrok_web_1  | + exec ngrok http '-subdomain=[REDACTED]' '-region=us' -log stdout web:3000
ngrok_web_1  | t=2020-12-31T21:30:49+0000 lvl=info msg="no configuration paths supplied"
[...]

See that WARNING after web_1?

This is "normal". Meaning, it should work anyway. (i.e. it's just a warning)

Now, take a look at your Events¹ page to make sure papertrail is receiving your logs.

You should see something like this:

Great success!