Installing Rucio Server

Prerequisites

The Rucio server runs on Python 2.7, 3.6 and 3.7 on any Unix-like platform.

Install via pip

Heads up: We recommend to use the docker-based install (see next section) as it will configure many things for you automatically. Only use the pip-based install if you have a good reason and know how to configure your webservices manually:

pip install rucio

This will pull the latest release from PyPi. The Rucio server also needs several Python dependencies. These are all listed in the file requirements.server.txt and will be pulled in as necessary.

Install via Docker

A simple server without SSL can be started like this:

docker run --name rucio-server --publish 80:80 --detach rucio/rucio-server

This will start up a simple server using sqlite based on an automatically generated configuration. You can check if the server is running with

curl http://localhost/ping

This should return the Rucio version used in the container. Any other curl requests will not work as the database backend is not initialized as this image is meant to be used with an already bootstrapped database backend. I.e., that the container has to be configured to point to the correct database. There are two ways to manage the Rucio configuration: using environment variables or by mounting a full rucio.cfg.

If you want to set the connection string for the database it can be done using the RUCIO_CFG_DATABASE_DEFAULT environment variable, e.g., to start a container connecting to a MySQL DB running at mysql.db you could use something like this:

docker run --name rucio-server \
  --env RUCIO_CFG_DATABASE_DEFAULT="mysql+pymysql://rucio:rucio@mysql.db/rucio" \
  --publish 80:80 \
  --detach \
  rucio/rucio-server

The are much more configuration parameters available that will be listed at the end of this readme.

Another way to configure Rucio is to directly mount a complete rucio.cfg into the container. This will then be used instead of the auto-generated one, e.g., if you have a rucio.cfg ready on your host system under /tmp/rucio.cfg you could start a container like this:

docker run --name rucio-server \
  --volume /tmp/rucio.cfg:/opt/rucio/etc/rucio.cfg \
  --publish 80:80 \
  --detach \
  rucio/rucio-server

The rucio.cfg is used to configure the database backend.

If you want to enable SSL you would need to set the RUCIO_ENABLE_SSL variable and also need to include the host certificate, key and the the CA certificate as volumes. E.g.,:

docker run --name rucio-server \
  --volume /tmp/ca.pem:/etc/grid-security/ca.pem \
  --volume /tmp/hostcert.pem:/etc/grid-security/hostcert.pem \
  --volume /tmp/hostkey.pem:/etc/grid-security/hostkey.pem \
  --volume /tmp/rucio.cfg:/opt/rucio/etc/rucio.cfg \
  --publish 443:443 \
  --env RUCIO_ENABLE_SSL=True \
  --detach \
  rucio/rucio-server

By default the output of the Apache web server is written directly to stdout and stderr. If you would rather direct them into separate files it can be done using the RUCIO_ENABLE_LOGS variable. The storage folder of the logs can be used as a volume:

docker run --name rucio-server \
  --volume /tmp/rucio.cfg:/opt/rucio/etc/rucio.cfg \
  --volume /tmp/logs:/var/log/httpd \
  --publish 80:80 \
  --env RUCIO_ENABLE_LOGS=True \
  --detach \
  rucio/rucio-server

Environment Variables

As shown in the examples above the rucio-server image can be configured using environment variables that are passed with docker run. Below is a list of all available variables and their behavior:

RUCIO_ENABLE_SSL

By default, the Rucio server runs without SSL on port 80. If you want to enable SSL set this variable to True. If you enable SSL you will also have to provide the host certificate and key and the certificate authority file. The server will look for hostcert.pem, hostkey.pem and ca.pem under /etc/grid-security so you will have to mount them as volumes. Furthermore you will also have to expose port 443.

RUCIO_SSL_PROTOCOL

By default, the server will only accept TLSv1.2 connections. Defining this environment variable will allow specification of a custom Apache SSLProtocol.

RUCIO_CA_PATH

If you are using SSL and want use SSLCACertificatePath and SSLCARevocationPath you can do so by specifying the path in this variable.

RUCIO_DEFINE_ALIASES

By default, the web server is configured with all common rest endpoints except the authentication endpoint. If you want to specify your own set of aliases you can set this variable to True. The web server then expects an alias file under /opt/rucio/etc/aliases.conf

RUCIO_ENABLE_LOGS

By default, the log output of the web server is written to stdout and stderr. If you set this variable to True the output will be written to access_log and error_log under /var/log/httpd.

RUCIO_LOG_LEVEL

The default log level is info. You can change it using this variable.

RUCIO_LOG_FORMAT

The default Rucio log format is %ht%tt%{X-Rucio-Forwarded-For}it%Tt%Dt\"%{X-Rucio-Auth-Token}i\"t%{X-Rucio-RequestId}it%{X-Rucio-Client-Ref}it\"%r\"t%\>st%b You can set your own format using this variable.

RUCIO_HOSTNAME

This variable sets the server name in the apache config.

RUCIO_SERVER_ADMIN

This variable sets the server admin in the apache config.

RUCIO_CFG configuration parameters

Environment variables can be used to set values for the auto-generated rucio.cfg. The names are derived from the actual names in the configuration file prefixed by RUCIO_CFG, e.g., the default value in the database section becomes RUCIO_CFG_DATABASE_DEFAULT. All available environment variables are:

RUCIO_CFG_COMMON_LOGDIR
RUCIO_CFG_COMMON_LOGLEVEL
RUCIO_CFG_COMMON_MAILTEMPLATEDIR
RUCIO_CFG_DATABASE_DEFAULT
RUCIO_CFG_DATABASE_SCHEMA
RUCIO_CFG_DATABASE_POOL_RESET_ON_RETURN
RUCIO_CFG_DATABASE_ECHO
RUCIO_CFG_DATABASE_POLL_RECYCLE
RUCIO_CFG_DATABASE_POOL_SIZE
RUCIO_CFG_DATABASE_POOL_TIMEOUT
RUCIO_CFG_DATABASE_MAX_OVERFLOW
RUCIO_CFG_DATABASE_POWUSERACCOUNT
RUCIO_CFG_DATABASE_USERPASSWORD
RUCIO_CFG_MONITOR_CARBON_SERVER
RUCIO_CFG_MONITOR_CARBON_PORT
RUCIO_CFG_MONITOR_USER_SCOPE
RUCIO_CFG_TRACE_TRACEDIR
RUCIO_CFG_TRACE_BROKERS
RUCIO_CFG_TRACE_PORT
RUCIO_CFG_TRACE_USERNAME
RUCIO_CFG_TRACE_PASSWORD
RUCIO_CFG_TRACE_TOPIC
RUCIO_CFG_PERMISSION_POLICY
RUCIO_CFG_PERMISSION_SCHEMA
RUCIO_CFG_PERMISSION_LFN2PFN_ALGORITHM_DEFAULT
RUCIO_CFG_PERMISSION_SUPPORT
RUCIO_CFG_PERMISSION_SUPPORT_RUCIO
RUCIO_CFG_WEBUI_USERCERT

Enable OIDC

This section describes how to enable authentication and authorization in Rucio using OpenID Connect (OIDC), OAuth2, and JSON Web Tokens (JWTs).

Rucio currently provides full support for INDIGO IAM, and has been tested with CILogon. A OIDC client must be registered in the Identity Provider (IdP) with required info.

Registering OIDC Clients to Identity provider (IdP)

Rucio requires a client at the IdP with following info.

Grant Type: authorization_code and client_credentials
Scopes:
- Auth scopes:
  - At minimum openid .
  - Default in rucio openid and profile.
- Transfer and Deletion related scopes:
  - fts
  - storage.modify:<path>
  - storage.create:<path>
  - storage.read:<path>
  - offline_access
redirect_uris :
- https://<RUCIO_AUTH_SERVER_HOSTNAME>/auth/oidc_token
- https://<RUCIO_AUTH_SERVER_HOSTNAME>/auth/oidc_code

tip

Ensure your server uses HTTPS, as most Identity Providers will reject non-secure redirect URIs

info

The audience value (default rucio) is set by the Rucio auth server during the authorization request.
The audience set in the client_credentials grant's token request corresponds to the storage hostname(s) and FTS hostname.
Some IdPs require these audience to be pre-registered with the client, while others don't (like IndigoIAM).

Use case of different grant types:

authorization_code: This grant type is used for interactive user authentication to rucio. User must use OIDC authorization type.
client_credentials: This grant is machine-machine interaction (daemons-IdP interactions). It is used for getting tokens for FTS and Storage(s). This is used by transfer and deletion daemons.

The <path> to register depends upon your IdP's scope matching algorithm in combination with configuration for RSEs as described here

Please save the value of client_id and client_secret after registration.

Preparing idpsecrets.json

Create an idpsecrets.json file containing the configuration of the two IdP clients. Then mount this file to Rucio server and daemons. The path to the mounted file have to be set in configuration If using Helm Chart then use mounting as described here.

Security

Never commit idpsecrets.json to version control. Store the file securely (Kubernetes Secret, encrypted backup or password manager). Mount secrets as read-only in production.

warning

The SCIM section of json file is not used but required to be present due to some legacy code still present. Thus please set some random value for it.

Example:

{
    "<IdP_nickname>": {

        "issuer": "<issuer_url_of_idp>",

        "redirect_uris": [
            "https://<auth_server_name>/auth/oidc_token",
            "https://<auth_server_name>/auth/oidc_code"
        ],
        "client_id": "<client_id>",
        "client_secret": "<client_secret>",

        "SCIM": {
            "client_id": "random",
            "client_secret": "random"
        }
    },

    "wlcg": {

        "issuer": "https://wlcg.cloud.cnaf.infn.it/",

        "redirect_uris": [
            "https://rucio-auth.cern.ch/auth/oidc_token",
            "https://rucio-auth.cern.ch/auth/oidc_code"
        ],

        "client_id": "fdc297fc-09 ...",
        "client_secret": "APFVcga_X ...",

        "SCIM": {
            "client_id": "random",
            "client_secret": "random"
        }
    }
}

Configuring Rucio Server for OIDC based authentication

Enable OIDC in rucio.cfg:

[oidc]
# Required: Path to the idpsecrets JSON file.
idpsecrets = /path/to/your/idpsecrets.json

# Required: Matches the <IdP_nickname> key in idpsecrets.json.
admin_issuer = <IdP_nickname>

# Optional: Expected 'aud' value in the user JWT. Defaults to 'rucio'.
# if different from default then put what you have
expected_audience = 'rucio'

# Optional: Expected scopes in the JWT. Defaults to 'openid profile'.
# if different from default then put what you have
expected_scope = 'openid profile'

Each user must have an OIDC identity linked to their Rucio account. The OIDC identity consists of:

sub claim which is subject claim of user.
iss claim which is issuer URL of the IdP.

Example:

rucio account identity add --account rucio_user_account \
  --type OIDC \
  --id "SUB=5b5e5d37-926b-4b42-8a98-a0b4b28baf18, ISS=https://wlcg.cloud.cnaf.infn.it/" \
  --email "wlcg-doma-rucio@cern.ch"

info

5b5e5d37-926b-4b42-8a98-a0b4b28baf18 is subject claim of user and https://wlcg.cloud.cnaf.infn.it/ is issuer url of IdP.
The format of id has single space after the comma, so be mindful of that.

Enabling OIDC for Transfers & Deletions

Rucio uses WLCG profile with Capability based authorization for token-based interactions with storage. Authorization is applied at the RSE level: This means tokens are scoped to the RSE's storage path prefix described here, not to individual files or datasets. A single token grants access to perform operations on any file under the RSE's path based on its capabilities (read, create, modify).

Token-based operations require:

The RSE must have the davs protocol enabled.

The RSE must have the oidc_support attribute set to True.

rucio rse attribute add --key oidc_support --value True RSE_NAME

The RSE must be able to accept token with WLCG profile and audience as its hostname
The RSE must allow permission for <path>
FTS must be configured to accept token fts scope and <fts_hostname> audience.
- FTS audience config described here.
- FTS scope config described here

Configuring `<path>` for Storage Capabilities in Rucio:

Each storage enforces a specific prefix for storage.<capability>:<path> scopes. There are two scenarios:

Scenario 1: Storage Accepts Full RSE Protocol Prefix

If your storage accepts the full RSE protocol prefix as-is, no additional configuration is needed.

Example:

RSE protocol prefix: /path/to/vo
Storage expects: /path/to/vo

Scenario 2: Storage Expects Partial RSE Protocol Prefix

If your storage expects only part of the RSE protocol prefix, use the oidc_base_path RSE attribute to strip the unwanted leading prefix.

Example:

RSE protocol prefix: /path/to/vo
Storage expects: /vo
Action: Set oidc_base_path to remove /path/to as shown below.

rucio rse attribute add --key oidc_base_path --value '/path/to' RSE_NAME

Transfer daemon token flow.

Transfer job submission to FTS

For TPC transfer rucio sends 3 tokens to FTS. fts token [F], src storage token [S] and destination storage token [D].

info

All tokens sent to FTS ([F], [S], [D]) are managed tokens.
This requires configuring FTS to perform token exchange and Just in Time token refresh, so long-running transfers continue even after the original tokens sent from Rucio expires. More info here and here.

Poller Daemon Token Support

For Poller to use tokens to communicate with FTS, add following to the config:
```
[conveyor]
poller_oidc_support = True
```
Reaper (deletion) Daemon Token support.

If the RSE has token enabled and meet the criteria then its enough for deletion via token.

Prerequisites​

Install via pip​

Install via Docker​

Environment Variables​

RUCIO_ENABLE_SSL​

RUCIO_SSL_PROTOCOL​

RUCIO_CA_PATH​

RUCIO_DEFINE_ALIASES​

RUCIO_ENABLE_LOGS​

RUCIO_LOG_LEVEL​

RUCIO_LOG_FORMAT​

RUCIO_HOSTNAME​

RUCIO_SERVER_ADMIN​

RUCIO_CFG configuration parameters​

Enable OIDC​

Registering OIDC Clients to Identity provider (IdP)​

Preparing idpsecrets.json​

Configuring Rucio Server for OIDC based authentication​

Enabling OIDC for Transfers & Deletions​

Configuring <path> for Storage Capabilities in Rucio:​

Transfer daemon token flow.​

Prerequisites

Install via pip

Install via Docker

Environment Variables

RUCIO_ENABLE_SSL

RUCIO_SSL_PROTOCOL

RUCIO_CA_PATH

RUCIO_DEFINE_ALIASES

RUCIO_ENABLE_LOGS

RUCIO_LOG_LEVEL

RUCIO_LOG_FORMAT

RUCIO_HOSTNAME

RUCIO_SERVER_ADMIN

RUCIO_CFG configuration parameters

Enable OIDC

Registering OIDC Clients to Identity provider (IdP)

Preparing idpsecrets.json

Configuring Rucio Server for OIDC based authentication

Enabling OIDC for Transfers & Deletions

Configuring `<path>` for Storage Capabilities in Rucio:

Transfer daemon token flow.