Installing Rucio Server

Prerequisites

The Rucio server runs on Python 2.7, 3.6 and 3.7 on any Unix-like platform.

Install via pip

Heads up: We recommend to use the docker-based install (see next section) as it will configure many things for you automatically. Only use the pip-based install if you have a good reason and know how to configure your webservices manually:

pip install rucio

This will pull the latest release from PyPi. The Rucio server also needs several Python dependencies. These are all listed in the file requirements.server.txt and will be pulled in as necessary.

Install via Docker

A simple server without SSL can be started like this:

docker run --name=rucio-server -p 80:80 -d rucio/rucio-server

This will start up a simple server using sqlite based on an automatically generated configuration. You can check if the server is running with

curl http://localhost/ping

This should return the Rucio version used in the container. Any other curl requests will not work as the database backend is not initialized as this image is meant to be used with an already bootstrapped database backend. I.e., that the container has to be configured to point to the correct database. There are two ways to manage the Rucio configuration: using environment variables or by mounting a full rucio.cfg.

If you want to set the connection string for the database it can be done using the RUCIO_CFG_DATABASE_DEFAULT environment variable, e.g., to start a container connecting to a MySQL DB running at mysql.db you could use something like this:

docker run --name=rucio-server \
  -e RUCIO_CFG_DATABASE_DEFAULT="mysql+pymysql://rucio:rucio@mysql.db/rucio" \
  -p 80:80 \
  -d \
  rucio/rucio-server

The are much more configuration parameters available that will be listed at the end of this readme.

Another way to configure Rucio is to directly mount a complete rucio.cfg into the container. This will then be used instead of the auto-generated one, e.g., if you have a rucio.cfg ready on your host system under /tmp/rucio.cfg you could start a container like this:

docker run --name=rucio-server \
  -v /tmp/rucio.cfg:/opt/rucio/etc/rucio.cfg \
  -p 80:80 \
  -d \
  rucio/rucio-server

The rucio.cfg is used to configure the database backend.

If you want to enable SSL you would need to set the RUCIO_ENABLE_SSL variable and also need to include the host certificate, key and the the CA certificate as volumes. E.g.,:

docker run --name=rucio-server \
  -v /tmp/ca.pem:/etc/grid-security/ca.pem \
  -v /tmp/hostcert.pem:/etc/grid-security/hostcert.pem \
  -v /tmp/hostkey.pem:/etc/grid-security/hostkey.pem \
  -v /tmp/rucio.cfg:/opt/rucio/etc/rucio.cfg \
  -p 443:443 \
  -e RUCIO_ENABLE_SSL=True \
  -d \
  rucio/rucio-server

By default the output of the Apache web server is written directly to stdout and stderr. If you would rather direct them into separate files it can be done using the RUCIO_ENABLE_LOGS variable. The storage folder of the logs can be used as a volume:

docker run --name=rucio-server \
  -v /tmp/rucio.cfg:/opt/rucio/etc/rucio.cfg \
  -v /tmp/logs:/var/log/httpd \
  -p 80:80 \
  -e RUCIO_ENABLE_LOGS=True \
  -d \
  rucio/rucio-server

Environment Variables

As shown in the examples above the rucio-server image can be configured using environment variables that are passed with docker run. Below is a list of all available variables and their behavior:

RUCIO_ENABLE_SSL

By default, the Rucio server runs without SSL on port 80. If you want to enable SSL set this variable to True. If you enable SSL you will also have to provide the host certificate and key and the certificate authority file. The server will look for hostcert.pem, hostkey.pem and ca.pem under /etc/grid-security so you will have to mount them as volumes. Furthermore you will also have to expose port 443.

RUCIO_SSL_PROTOCOL

By default, the server will only accept TLSv1.2 connections. Defining this environment variable will allow specification of a custom Apache SSLProtocol.

RUCIO_CA_PATH

If you are using SSL and want use SSLCACertificatePath and SSLCARevocationPath you can do so by specifying the path in this variable.

RUCIO_DEFINE_ALIASES

By default, the web server is configured with all common rest endpoints except the authentication endpoint. If you want to specify your own set of aliases you can set this variable to True. The web server then expects an alias file under /opt/rucio/etc/aliases.conf

RUCIO_ENABLE_LOGS

By default, the log output of the web server is written to stdout and stderr. If you set this variable to True the output will be written to access_log and error_log under /var/log/httpd.

RUCIO_LOG_LEVEL

The default log level is info. You can change it using this variable.

RUCIO_LOG_FORMAT

The default Rucio log format is %ht%tt%{X-Rucio-Forwarded-For}it%Tt%Dt\"%{X-Rucio-Auth-Token}i\"t%{X-Rucio-RequestId}it%{X-Rucio-Client-Ref}it\"%r\"t%\>st%b You can set your own format using this variable.

RUCIO_HOSTNAME

This variable sets the server name in the apache config.

RUCIO_SERVER_ADMIN

This variable sets the server admin in the apache config.

RUCIO_CFG configuration parameters

Environment variables can be used to set values for the auto-generated rucio.cfg. The names are derived from the actual names in the configuration file prefixed by RUCIO_CFG, e.g., the default value in the database section becomes RUCIO_CFG_DATABASE_DEFAULT. All available environment variables are:

RUCIO_CFG_COMMON_LOGDIR
RUCIO_CFG_COMMON_LOGLEVEL
RUCIO_CFG_COMMON_MAILTEMPLATEDIR
RUCIO_CFG_DATABASE_DEFAULT
RUCIO_CFG_DATABASE_SCHEMA
RUCIO_CFG_DATABASE_POOL_RESET_ON_RETURN
RUCIO_CFG_DATABASE_ECHO
RUCIO_CFG_DATABASE_POLL_RECYCLE
RUCIO_CFG_DATABASE_POOL_SIZE
RUCIO_CFG_DATABASE_POOL_TIMEOUT
RUCIO_CFG_DATABASE_MAX_OVERFLOW
RUCIO_CFG_DATABASE_POWUSERACCOUNT
RUCIO_CFG_DATABASE_USERPASSWORD
RUCIO_CFG_MONITOR_CARBON_SERVER
RUCIO_CFG_MONITOR_CARBON_PORT
RUCIO_CFG_MONITOR_USER_SCOPE
RUCIO_CFG_TRACE_TRACEDIR
RUCIO_CFG_TRACE_BROKERS
RUCIO_CFG_TRACE_PORT
RUCIO_CFG_TRACE_USERNAME
RUCIO_CFG_TRACE_PASSWORD
RUCIO_CFG_TRACE_TOPIC
RUCIO_CFG_PERMISSION_POLICY
RUCIO_CFG_PERMISSION_SCHEMA
RUCIO_CFG_PERMISSION_LFN2PFN_ALGORITHM_DEFAULT
RUCIO_CFG_PERMISSION_SUPPORT
RUCIO_CFG_PERMISSION_SUPPORT_RUCIO
RUCIO_CFG_WEBUI_USERCERT

Enable OIDC

This section describes how to enable authentication and authorization in Rucio using OpenID Connect (OIDC), OAuth2, and JSON Web Tokens (JWTs).

Rucio currently provides full support for INDIGO IAM, and has been tested with CILogon. Two separate OIDC clients must be registered in the Identity Provider (IdP): one for user authentication and one for daemon/service authorization.

Registering OIDC Clients to Identity provider (IdP)

Rucio requires two clients at the IdP:

[C1] Rucio Auth Client This client is used for interactive user authentication (CLI, WebUI) using the Authorization Code Flow.
- Grant Type: authorization_code
- Audience: rucio .
- Scopes:
  - At minimum openid .
  - Default in rucio openid profile and email .
- redirect_uris :
  - https://<auth_server_name>/auth/oidc_token
  - https://<auth_server_name>/auth/oidc_code
Note: use https for redirect_uris .
[C2] Rucio Admin Client This client is responsible interaction to storages and FTS. Used by transfer and deletion daemons.
- Grant Type: client_credentials
- Audience:
  - <storage_hostname>
  - fts
- Scopes:
  - fts
  - storage.modify:<path>
  - storage.create:<path>
  - storage.read:<path>

<path> typically corresponds to the scope accepted by the storage, as described later.

Please save the client_id and client_secret from both of [C1] and [C2].

Preparing idpsecrets.json

Create an idpsecrets.json file containing the configuration of the two IdP clients. Then mount this file to Rucio server and daemons. If using Helm Chart then use mounting as described here.

Security: Never commit idpsecrets.json to version control. Store the file securely (Kubernetes Secret, encrypted backup or password manager). Mount secrets as read-only in production.

Example:

{
    "<IdP_nickname>": {

        "issuer": "https://<issuer_server_name>",

        "redirect_uris": [
            "https://<auth_server_name>/auth/oidc_token",
            "https://<auth_server_name>/auth/oidc_code"
        ],
        "client_id": "<C1_client_id>",
        "client_secret": "<C1_client_secret>",

        "SCIM": {
            "client_id": "<C2_client_id>",
            "client_secret": "<C2_client_secret>",
        }
    },

    "wlcg": {

        "issuer": "https://wlcg.cloud.cnaf.infn.it/",

        "redirect_uris": [
            "https://rucio-auth.cern.ch/auth/oidc_token",
            "https://rucio-auth.cern.ch/auth/oidc_code"
        ],

        "client_id": "fdc297fc-09 ...",
        "client_secret": "APFVcga_X ...",

        "SCIM": {
            "client_id": "5b5e5d3 ...",
            "client_secret": "IQqAcMOa ...",
        }
    }
}

Configuring Rucio Server for OIDC based authentication

The Rucio Auth Client (C1) is used for user login. Enable OIDC in rucio.cfg:

[oidc]
# Required: Path to the idpsecrets JSON file.
idpsecrets = /path/to/your/idpsecrets.json 

# Required: Matches the <IdP_nickname> key in idpsecrets.json. 
admin_issuer = <IdP_nickname>

# Optional: Expected 'aud' value in the user JWT. Defaults to 'rucio'.
# if different from default then put what you have for [C1]
expected_audience = 'rucio' 

# Optional: Expected scopes in the JWT. Defaults to 'openid profile email'.
# if different from default then put what you have for [c1]
expected_scope = 'openid profile email'

Each user must have an OIDC identity linked to their Rucio account. The OIDC identity consists of:

sub claim which is subject claim of user.
iss claim which is issuer URL of the IdP.

Example:

rucio account identity add --account rucio_user_account \
  --type OIDC \
  --id "SUB=5b5e5d37-926b-4b42-8a98-a0b4b28baf18, \
    ISS=https://wlcg.cloud.cnaf.infn.it/" \
  --email "wlcg-doma-rucio@cern.ch"

Note: 5b5e5d37-926b-4b42-8a98-a0b4b28baf18 is subject claim of user and https://wlcg.cloud.cnaf.infn.it/ is issuer url of IdP.

Enabling OIDC for Transfers & Deletions

Rucio uses WLCG profile with Capability based authorization for token-based interactions with storage and FTS. Authorization is applied at the RSE level: This means tokens are scoped to the RSE's storage path prefix described here, not to individual files or datasets. A single token grants access to perform operations on any file under the RSE's path based on its capabilities (read, create, modify).

Token-based operations require:

The RSE must have the davs protocol enabled.

The RSE must have the oidc_support attribute set to True.

rucio rse attribute add --key oidc_support --value True RSE_NAME

The RSE must be able to accept token with WLCG profile and audience as its hostname
The RSE must allow permission for <path>
FTS must be configured to accept token fts scope and <fts_hostname> audience.
- FTS audience config described here.
- FTS scope config described here

Defining `<path>` for Storage Capabilities:

Each storage enforces a specific prefix for storage.<capability>:<path> scopes. There are two cases:

Storage accepts the full RSE protocol prefix
Storage expects part of the RSE protocol prefix Set the oidc_base_path RSE attribute to remove the unwanted leading prefix.

Example:
- RSE protocol prefix: /path/to/vo
- Storage expects: /vo Then add:
```
rucio rse attribute add --key oidc_base_path --value '/path/to' RSE_NAME
```

Transfer daemon token flow.

Transfer job submission to FTS For TPC transfer rucio sends 3 tokens to FTS. fts token [F], src storage token [S] and destination storage token [D].

Note: All tokens sent to FTS ([F], [S], [D]) are managed tokens.
This requires configuring FTS to perform token exchange and Just in Time token refresh, so long-running transfers continue even after the original tokens sent from Rucio expires. More info here and here.

Poller Daemon Token Support

For Poller to use tokens to communicate with FTS, modify to config with the following:
```
[conveyor]
poller_oidc_support = True
```
Reaper (deletion) Daemon Token support.

If the RSE has token enabled and meet the criteria then its enough for deletion via token.

Prerequisites​

Install via pip​

Install via Docker​

Environment Variables​

RUCIO_ENABLE_SSL​

RUCIO_SSL_PROTOCOL​

RUCIO_CA_PATH​

RUCIO_DEFINE_ALIASES​

RUCIO_ENABLE_LOGS​

RUCIO_LOG_LEVEL​

RUCIO_LOG_FORMAT​

RUCIO_HOSTNAME​

RUCIO_SERVER_ADMIN​

RUCIO_CFG configuration parameters​

Enable OIDC​

Registering OIDC Clients to Identity provider (IdP)​

Preparing idpsecrets.json​

Configuring Rucio Server for OIDC based authentication​

Enabling OIDC for Transfers & Deletions​

Defining <path> for Storage Capabilities:​

Transfer daemon token flow.​