Usage¶

Invocation¶

On a single host¶

Usually you would run deck-chores in a container:

$ docker run --rm -v /var/run/docker.sock:/var/run/docker.sock funkyfuture/deck-chores:1

Note

There’s a manifest on the Docker Hub that maps images to builds targeting amd64, amd64 and armv7l architectures. Thus you don’t need to specify any platform indicator, the Docker client will figure out which one is the proper image to pull.

Likewise, docker-compose can be used with such configuration:

version: "3.7"

services:
  officer:
    image: funkyfuture/deck-chores:1
    restart: unless-stopped
    environment:
      TIMEZONE: Asia/Tel Aviv
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock

You could also install deck-chores from the Python Package Index with pip or pipx (recommended):

$ pipx install deck-chores

and then run it:

$ deck-chores

Now one instance of deck-chores is running and will handle all job definitions that it discovers on containers that run on the Docker host.

In a Docker Swarm¶

deck-chores can be run in a Docker Swarm cluster, but it must be deployed on all nodes and it cannot restrict jobs to be run in only one of the containers that manifest a service. This would be a suitable stack definition:

version: "3.7"

services:
  officer:
    image: funkyfuture/deck-chores:1
    deploy:
      mode: global
    environment:
      TIMEZONE: Europe/Berlin
      # it isn't guaranteed that service or job options don't override this:
      DEFAULT_FLAGS: noservice
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock

It can be deployed with:

$ docker stack deploy --compose-file docker-compose.yml deck-chores

Now one instance of deck-chores is running on each Swarm node and each will handle all job definitions that it discovers on containers that run on the same Swarm node. No instance is aware of the events and containers on other nodes.

Caveats & Tips¶

Caution

There’s yet no way to distinguish container events that happen during an image build from others (#6 and #15211). Thus when an image is built, deck-chores will register and remove jobs on all intermediate containers following labels that define jobs. It would possibly trigger these jobs, which might lead to a corrupted build. You can avoid this risk by building images on a host that is not observed by deck-chores or by pausing it during image builds. Another alternative could be using Podman to build images.

Containers without an enduring main process¶

If the container is supposed to only run the scheduled commands and not a main process, use a non-stopping no-op command as main process like in this snippet of a docker-compose.yml file:

services:
  neverending:
    # …
    command: tail -f /dev/null
    labels:
      deck-chores.daily_job.command: daily_command …
      deck-chores.daily_job.interval: daily

Making jobs’ output available to `docker logs` of the executing container¶

Docker captures the output of the first process in a container as logged data. In order to capture the output of a job’s command as well, its output needs to be redirected to the main process’ stdout or stderr, e.g. with by redirecting a command’s output with a shell:

deck-chores.a_job.command: sh -c "/usr/local/bin/job_script.sh &> /proc/1/fd/1"

Listing all registered jobs¶

Information, including the next scheduled execution, about the registered jobs of a deck-chores instance can be logged at once by sending SIGUSR1 signal to the process, e.g. to one that runs in a container:

docker kill --signal USR1 <CONTAINER>

Job definitions¶

Job definitions are parsed from a container’s metadata aka labels. A label’s key must be in the namespace defined by LABEL_NAMESPACE (default: deck-chores) to be considered. A job has its own namespace that holds all its attributes. Thus an attribute’s key has usually this schema:

$LABEL_NAMESPACE.<job name>.<job attribute>

An exception is a job’s env namespace that is structured like this:

$LABEL_NAMESPACE.<job name>.env.<variable name>

The job name options cannot be used as it is reserved for setting Container-scoped configuration.

A job name can consist of lower-case letters, digits and dashes.

The following attributes are available:

Attribute	Description
command	the command to run
cron	a cron definition
date	a date definition
env	this namespace holds environment variables that are set on the command’s execution context
interval	an interval definition
jitter	the maximum length of a random delay before each job’s execution (in conjunction with a cron or interval trigger); can be either a number that define seconds or a number with a subsequent time unit indicator like the interval trigger
max	the maximum of simultaneously running command instances, defaults to `DEFAULT_MAX`
timezone	the timezone that the trigger relates to, defaults to `TIMEZONE`
user	the user to run the command; see the user option for details regarding the defaults
workdir	the working directory when the command is executed

The attribute command and one of cron, date or interval are required for each job.

Example snippet from a docker-compose.yml file:

services:
  web:
    # ...
    labels:
      deck-chores.clear-caches.command: drush cc all
      deck-chores.clear-caches.interval: daily
      deck-chores.clear-caches.user: www-data
      deck-chores.clear-caches.env.ENVIRONMENT: production

Or baked into an image:

LABEL deck-chores.clear-caches.command="drush cc all" \
      deck-chores.clear-caches.interval="daily" \
      deck-chores.clear-caches.user="www-data" \
      deck-chores.clear-caches.env.ENVIRONMENT="production"

Job triggers¶

cron¶

cron triggers allow definitions for repeated run times like for the well-known cron daemon. In contrast to the classic, the sequence of fields is flipped, starting with the greatest unit on the left. The fields are separated by spaces, missing fields are filled up with * on the left.

The fields from left to right define:

year

month

day (of month)

week (of year)

day_of_week

hour

minute

second

See APScheduler’s documentation for details on its versatile expressions.

Examples¶

* * * * * */3 0 0  # run on all hours dividable by 3
*/3 0 0            # as shortened expression
* * * * 6 1 0 0    # run every Sunday at 1:00
6 1 0 0            # as shortened expression
sun 1 0 0          # as 'speaking' variant
* * * * * 1-4 0 0  # run daily at 1:00, 2:00, 3:00 and 4:00
1-4 0 0            # as shortened expression

date¶

A one-time trigger that is formatted as YYYY-MM-DD [HH:MM:SS].

An omitted time is interpreted as 0:00:00. Note that times must include a seconds field.

interval¶

This trigger defines a repetition by a fixed interval. It can either be a string where time units follow numbers or a sequence of numbers that qualify time units by order.

In the first form the numbers can be decimal fractions and the time units are determined by the first letter of a token as week, day, hour, minute or second.

In the anonymous form the interval is added up by the fields weeks, days, hours, minutes and seconds in that order. Possible field separators are ., :, / and spaces. Missing fields are filled up with 0 on the left.

Examples¶

28 Days       # run every 4 weeks
4 wookies     # run every 4 weeks
42s 0.5d      # run every twelve hours and 42 seconds
42:00:00      # run every fourty-two hours
100/00:00:00  # run every one hundred days

There are also the convenience shortcuts weekly, daily, hourly, every minute and every second.

Note

Though it uses the same units of measurement, an interval is different from a recurring point in time of a specific calendar system, it describes the time between two events. Hence you should expect a job that is defined with this type of trigger to run the defined time after the job has been registered. To define a recurring point in time, see the cron trigger.

Caution

Mind that deck-chores doesn’t track jobs’ status when they are removed from the scheduler and doesn’t persist any data between its invocations. Any such event would therefore reset the next scheduled run time of a job. Depending on a system’s usage this is more or less likely to happen. For longer intervals, a cron trigger would therefore be preferable.

Container-scoped configuration¶

user¶

A user that shall run all jobs for a container can be set with a label name of this form:

$LABEL_NAMESPACE.options.user

The option can also be defined for an image and is considered when the image flag is set. If this option is not set, Docker uses the user that was specified with the --user option on container creation or falls back to the one defined in the underlying image.

flags¶

Option flags control deck-chores’s behaviour with regard to the labeled container and override the setting of DEFAULT_FLAGS. The schema for a flags label name is:

$LABEL_NAMESPACE.options.flags

Options are set as comma-separated list of flags. An option set by DEFAULT_FLAGS can be unset by prefixing with no.

These options are available:

image¶: Job definitions in the container’s basing image labels are also parsed while container label keys override these.

service¶

Restricts jobs to one container of those that are identified with the same service.

See SERVICE_ID_LABELS regarding service identity.

Environment variables¶

deck-chore’s behaviour is defined by these environment variables:

CLIENT_TIMEOUT¶: The timeout for responses from the Docker daemon in seconds without unit indicator. The default is imported from docker-py.

CONTAINER_CACHE_SIZE¶

default: 128

The size of caches that save immutable container properties, like the parsed and possibly absent job definitions. Since memory is cheap and so are the stored objects, increase this when you have a lot of containers floating around to reduce latency.

DOCKER_HOST¶

default: unix://var/run/docker.sock

The URL of the Docker daemon to connect to.

DEBUG¶

default: no

Log debugging messages, enabled by on, true or yes.

DEFAULT_FLAGS¶

default: image,service

The default for a job option’s flags attribute.

DEFAULT_MAX¶

default: 1

The default for a job’s max attribute.

JOB_POOL_SIZE¶

default: 10

The pool size of job executors defines the maximum number of jobs that can run at the same time.

LABEL_NAMESPACE¶

default: deck-chores

The label namespace to look for job definitions and container options.

LOG_FORMAT¶

default: {asctime}|{levelname:8}|{message}

Pattern that formats log record attributes.

SERVICE_ID_LABELS¶

default: com.docker.compose.project,com.docker.compose.service

A comma-separated list of container labels that identify a unique service with possibly multiple container instances. This has an impact on how the service option behaves.

TIMEZONE¶

default: UTC

The job scheduler’s timezone and the default for a job’s timezone attribute.

TLS options¶

ASSERT_HOSTNAME¶

default: no

Enabled by on, true or yes.

SSL_VERSION¶

default: TLS (selects the highest version supported by the client and the daemon)

For other options see the names provided by Python’s ssl library prefixed with PROTOCOL_.

Authentication related files are expected to be available at /config/ca.pem, /config/cert.pem respectively /config/key.pem.

Usage¶

Invocation¶

On a single host¶

In a Docker Swarm¶

Caveats & Tips¶

Containers without an enduring main process¶

Making jobs’ output available to docker logs of the executing container¶

Listing all registered jobs¶

Job definitions¶

Job triggers¶

cron¶

Examples¶

date¶

interval¶

Examples¶

Container-scoped configuration¶

user¶

flags¶

Environment variables¶

TLS options¶

Making jobs’ output available to `docker logs` of the executing container¶