Skip to main content

Installing Rucio Daemons

Prerequisites​

The Rucio daemons run on Python 2.7, 3.6 and 3.7 on any Unix-like platform.

Install via pip​

Heads up: We recommend to use the docker-based install (see next section) as it will configure many things for you automatically. Only use the pip-based install if you have a good reason and know how to configure your web service manually:

pip install rucio

This will pull the latest release from PyPi. The Rucio server also needs several Python dependencies. These are all listed in the file requirements.txt and will be pulled in as necessary.

Install via Docker​

This image provides the Rucio daemons. Each daemon has to be run in a separate container. It supports MySQL, PostgreSQL, Oracle, and SQLite as database backends.

This image expects that there is an already initialised Rucio DB. To start a simple judge-cleaner daemon using a database on mysql.db without any additional parameters just run this:

docker run --name=rucio-judge-cleaner \
  -e RUCIO_CFG_DATABASE_DEFAULT="mysql+pymysql://rucio:rucio@mysql.db/rucio" \
  -e RUCIO_DAEMON=judge-cleaner \
  rucio/rucio-daemons

The RUCIO_DAEMON environment variable gives the name of the rucio daemon.

Rucio can be configured fully using environment variables like RUCIO_CFG_DATABASE_DEFAULT. If you want to instead use a complete rucio.cfg it can also be mounted. This will then ignore the RUCIO_CFG environment variables:

The rucio.cfg is used to configure the database backend and the daemons:

docker run --name=rucio-judge-cleaner \
  -v /tmp/rucio.cfg:/opt/rucio/etc/rucio.cfg \
  -e RUCIO_DAEMON=judge-cleaner \
  rucio/rucio-daemons

By default the daemon logs are written to stdout and stderr if you want to write to a file you can use RUCIO_ENABLE_LOGS like this:

docker run --name=rucio-judge-cleaner \
  -v /tmp/rucio.cfg:/opt/rucio/etc/rucio.cfg \
  -v /tmp/logs:/var/log/rucio -e RUCIO_DAEMON=judge-cleaner \
  -e RUCIO_ENABLE_LOGS=True \
  rucio/rucio-daemons

Environment Variables​

As shown in the examples above the rucio-daemon image can be configured using environment variables that are passed with docker run. Below is a list of all available variables and their behaviour:

RUCIO_DAEMON​

This variable is mandatory and it specifies the name of the daemon, e.g., hermes, kronos, judge-evaluator, etc.

RUCIO_DAEMON_ARGS​

Any additional command line parameter can be specified here, e.g., \--run-once This field is optional.

RUCIO_ENABLE_LOGS​

By default, the log output of the daemon is written to stdout and stderr. If you set this variable to True the output will be written to access_log and error_log under /var/log/rucio

RUCIO_CFG configuration parameters​

Environment variables can be used to set values for the auto-generated rucio.cfg. The names are derived from the actual names in the configuration file prefixed by RUCIO_CFG, e.g., the default value in the database section becomes RUCIO_CFG_DATABASE_DEFAULT. All available environment variables are:

  • RUCIO_CFG_ACCOUNTS_SPECIAL_ACCOUNTS
  • RUCIO_CFG_COMMON_LOGDIR
  • RUCIO_CFG_COMMON_LOGLEVEL
  • RUCIO_CFG_COMMON_MAILTEMPLATEDIR
  • RUCIO_CFG_DATABASE_DEFAULT
  • RUCIO_CFG_DATABASE_SCHEMA
  • RUCIO_CFG_DATABASE_SCHEMA
  • RUCIO_CFG_DATABASE_POOL_RESET_ON_RETURN
  • RUCIO_CFG_DATABASE_ECHO
  • RUCIO_CFG_DATABASE_POOL_RECYCLE
  • RUCIO_CFG_DATABASE_POOL_SIZE
  • RUCIO_CFG_DATABASE_POOL_TIMEOUT
  • RUCIO_CFG_DATABASE_MAX_OVERFLOW
  • RUCIO_CFG_DATABASE_POWUSERACCOUNT
  • RUCIO_CFG_DATABASE_POWUSERPASSWORD
  • RUCIO_CFG_MONITOR_CARBON_SERVER
  • RUCIO_CFG_MONITOR_CARBON_PORT
  • RUCIO_CFG_MONITOR_USER_SCOPE
  • RUCIO_CFG_PERMISSION_POLICY
  • RUCIO_CFG_PERMISSION_SCHEMA
  • RUCIO_CFG_PERMISSION_LFN2PFN_ALGORITHM_DEFAULT
  • RUCIO_CFG_PERMISSION_SUPPORT
  • RUCIO_CFG_PERMISSION_SUPPORT_RUCIO
  • RUCIO_CFG_AUTOMATIX_SITES
  • RUCIO_CFG_AUTOMATIX_SLEEP_TIME
  • RUCIO_CFG_AUTOMATIX_DATASET_LIFETIME
  • RUCIO_CFG_AUTOMATIX_SET_METADATA
  • RUCIO_CFG_AUDITOR_RESULTS
  • RUCIO_CFG_AUDITOR_CACHE
  • RUCIO_CFG_CONVEYOR_SCHEME
  • RUCIO_CFG_CONVEYOR_TRANSFERTOOL
  • RUCIO_CFG_CONVEYOR_FTSHOSTS
  • RUCIO_CFG_CONVEYOR_CACERT
  • RUCIO_CFG_CONVEYOR_USERCERT
  • RUCIO_CFG_CONVEYOR_CACHE_TIME
  • RUCIO_CFG_CONVEYOR_USE_DETERMINISTIC_ID
  • RUCIO_CFG_CONVEYOR_POLL_TIMEOUT
  • RUCIO_CFG_CONVEYOR_SUBMIT_TIMEOUT
  • RUCIO_CFG_CONVEYOR_BRING_ONLINE
  • RUCIO_CFG_CONVEYOR_QUEUE_MODE
  • RUCIO_CFG_CONVEYOR_USING_MEMCACHE
  • RUCIO_CFG_CONVEYOR_FTSMONHOSTS
  • RUCIO_CFG_MESSAGING_FTS3_PORT
  • RUCIO_CFG_MESSAGING_FTS3_SSL_KEY_FILE
  • RUCIO_CFG_MESSAGING_FTS3_SSL_CERT_FILE
  • RUCIO_CFG_MESSAGING_FTS3_DESTINATION
  • RUCIO_CFG_MESSAGING_FTS3_BROKERS
  • RUCIO_CFG_MESSAGING_FTS3_VONAME
  • RUCIO_CFG_MESSAGING_HERMES_USERNAME
  • RUCIO_CFG_MESSAGING_HERMES_PASSWORD
  • RUCIO_CFG_MESSAGING_HERMES_PORT
  • RUCIO_CFG_MESSAGING_HERMES_NONSSL_PORT
  • RUCIO_CFG_MESSAGING_HERMES_USE_SSL
  • RUCIO_CFG_MESSAGING_HERMES_SSL_KEY_FILE
  • RUCIO_CFG_MESSAGING_HERMES_SSL_CERT_FILE
  • RUCIO_CFG_MESSAGING_HERMES_DESTINATION
  • RUCIO_CFG_MESSAGING_HERMES_BROKERS
  • RUCIO_CFG_MESSAGING_HERMES_VONAME
  • RUCIO_CFG_MESSAGING_HERMES_EMAIL_FROM
  • RUCIO_CFG_MESSAGING_HERMES_EMAIL_TEST
  • RUCIO_CFG_TRACER_KRONOS_BROKERS
  • RUCIO_CFG_TRACER_KRONOS_PORT
  • RUCIO_CFG_TRACER_SSL_KEY_FILE
  • RUCIO_CFG_TRACER_SSL_CERT_FILE
  • RUCIO_CFG_TRACER_QUEUE
  • RUCIO_CFG_TRACER_PREFETCH_SIZE
  • RUCIO_CFG_TRACER_CHUNKSIZE
  • RUCIO_CFG_TRACER_SUBSCRIPTION_ID
  • RUCIO_CFG_TRACER_USE_SSL
  • RUCIO_CFG_TRACER_RECONNECT_ATTEMPTS
  • RUCIO_CFG_TRACER_EXCLUDED_USRDNS
  • RUCIO_CFG_TRACER_KRONOS_USERNAME
  • RUCIO_CFG_TRACER_KRONOS_PASSWORD
  • RUCIO_CFG_TRACER_DATASET_WAIT
  • RUCIO_CFG_MESSAGING_CACHE_PORT
  • RUCIO_CFG_MESSAGING_CACHE_SSL_KEY_FILE
  • RUCIO_CFG_MESSAGING_CACHE_SSL_CERT_FILE
  • RUCIO_CFG_MESSAGING_CACHE_DESTINATION
  • RUCIO_CFG_MESSAGING_CACHE_BROKERS
  • RUCIO_CFG_MESSAGING_CACHE_VONAME
  • RUCIO_CFG_MESSAGING_CACHE_ACCOUNT
  • RUCIO_CFG_CREDENTIALS_GCS
  • RUCIO_CFG_CREDENTIALS_SIGNATURE_LIFETIME