Skip to main content

Configuration parameters

Description of the configuration parameters splitted in three different places: Rucio configuration file (rucio.cfg), Rucio configuration table and RSE attributes.

Rucio configuration file (rucio.cfg)​

  • accounts

    • special_accounts

  • alembic

    • cfg: Path to the configuration file (.ini) for Alembic. Example: /opt/rucio/etc/alembic.ini. No default.

  • api

    • endpoints: (Optional) Endpoints separated by commas. Default: ['accountlimits', 'accounts', 'config', 'credentials', 'dids', 'export', 'heartbeats', 'identities', 'import', 'lifetime_exceptions', 'locks', 'meta', 'ping', 'redirect', 'replicas', 'requests', 'rses', 'rules', 'scopes', 'subscriptions'].

  • auditor

    • cache: Path to the folder to store the rucio-auditor cache. Example: /opt/rucio/auditor-cache. No default.

    • results: Path to the folder to store the rucio-auditor results. Example: /opt/rucio/auditor-results. No default.

    • threshold: (Optional) Floating number used in a sanity check, comparing the number of entries with the total number of files on the RSE:

      if len(dark_replicas) > threshold * usage['files']
      if len(lost_replicas) > threshold * usage['files']

      Default: 0.2.

  • bb8

    • dump_production_day: (Optional) Day of the week of the most recent dump. Values: {Sunday, Monday, Tuesday, Wednesday, Thursday, Friday, Saturday}. Default: None.
    • dump_url_template: (Optional) URL of the template (structure) of a dump. Default: http://rucio-analytix.cern.ch:8080/LOCKS/GetFileFromHDFS?date=${date}&rse=${rse}.

  • bootstrap

    • gss_email: (Optional) Email of the Kerberos auth method which identity is specified in gss_identity.
    • gss_identity: (Optional) Identity of the Kerberos auth method.
    • saml_email
    • saml_id
    • ssh_email: (Optional) Email of the SSH auth method which identity is specified in ssh_identity. Default: ph-adp-ddm-lab@cern.ch.
    • ssh_identity: (Optional) SSH auth using an RSA key. Default: ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAq5LySllrQFpPL614sulXQ7wnIr1aGhGtl8b+HCB/0FhMSMTHwSjX78UbfqEorZV16rXrWPgUpvcbp2hqctw6eCbxwqcgu3uGWaeS5A0iWRw7oXUh6ydnVy89zGzX1FJFFDZ+AgiZ3ytp55tg1bjqqhK1OSC0pJxdNe878TRVVo5MLI0S/rZY2UovCSGFaQG2iLj14wz/YqI7NFMUuJFR4e6xmNsOP7fCZ4bGMsmnhR0GmY0dWYTupNiP5WdYXAfKExlnvFLTlDI5Mgh4Z11NraQ8pv4YE1woolYpqOc/IMMBBXFniTT4tC7cgikxWb9ZmFe+r4t6yCDpX4IL8L5GOQ== ddmlab
    • userpass_email: (Optional) Email of the root account which name is specified in userpass_identity. Default: ph-adp-ddm-lab@cern.ch.
    • userpass_identity: (Optional) Name of the root account. Default: ddmlab.
    • userpass_pwd: (Optional) Password of the root account which name is specified in userpass_identity. Default: secret.
    • x509_email: (Optional) Email of the X.509 identity specified in x509_identity. Default: ph-adp-ddm-lab@cern.ch.
    • x509_identity: (Optional) Identity of the X.509 certificate. Default: emailAddress=ph-adp-ddm-lab@cern.ch,CN=DDMLAB Client Certificate,OU=PH-ADP-CO,O=CERN,ST=Geneva,C=CH.

  • c3po

    • algorithms: (Optional) Placement algorithm.

      Only allows multiple algorithms if the dry_run mode is enabled (separated by commas).

      Values: {simple, t2_free_space, t2_free_space_only_pop, t2_free_space_only_pop_with_network}.

      Default: None.

    • ca_cert: (Optional) Path of the certificate for Elasticsearch. Default: False.

    • elastic_index: (Optional) Index (database) to use in Elasticsearch.

    • elastic_pass: (Optional) Password of the username defined in elastic_user to authenticate to Elasticsearch. No default.

    • elastic_url: URL of Elasticsearch. Example: http://aianalytics01.cern.ch:9200.

    • elastic_user: (Optional) Username to authenticate to Elasticsearch. No default.

    • placement_algorithm

    • redis_host: URL of the Redis database.

    • redis_port: Port of the Redis database defined in redis_host.

  • c3po-popularity

    • elastic_url

  • c3po-site-mapper

    • ddm_url: URL for DDM. Example: http://atlas-agis-api.cern.ch/request/ddmendpoint/query/list/?json.
    • panda_url: URL for PanDA. Example: http://atlas-agis-api.cern.ch/request/pandaqueue/query/list/?json.

  • c3po-workload

    • panda_url: URL for PanDA. Example: http://bigpanda.cern.ch/jobs/?category=analysis&jobstatus=running.
    • window: Integer.

  • c3po-network-metrics

    • prefix:
    • redis_host: URL of the Redis database.
    • redis_port: Port of the Redis database defined in redis_host.

  • cache

    • url: (Optional) URL of the cache. Default: 127.0.0.1:11211.
    • use_external_cache_for_auth_tokens: (Optional) if True, use remote cache provider for auth tokens. If False, use a private in-memory cache. Default: False

  • common

    • extract_scope: (Optional) Default: atlas.

    • logdir: Path of the directory for logs. Contains auditor.log.

    • logformat: (Optional) Formatter of the log. See: https://docs.python.org/3/library/logging.html#logging.Formatter

    • loglevel: (Optional). Set the root logger level to the specified level.

      Values: {'NOTSET', 'DEBUG', 'INFO', 'WARNING', 'ERROR', 'CRITICAL'}. See: https://docs.python.org/3/library/logging.html#levels

      Default: DEBUG. For rucio-auditor default is WARNING.

    • mailtemplatedir: (Optional) Path of the folder with mail templates (.tmpl). Example: /opt/rucio/etc/mail_templates.

    • multi_vo: (Optional) Default: False.

  • conveyor

    • allow_user_oidc_tokens: (Optional) Boolean. Default: False.
    • bring_online: Integer, bring online timeout. Default: 43200.
    • cacert
    • cache_time: (Optional) Integer, expiration time in seconds passed to the dogpile system. Default: 600.
    • failover_scheme: Failover schemes. Default: None.
    • filter_transfertool: (Optional) Default: None.
    • ftshosts: URL of the File Transfer Service (FTS) hosts (separated by commas). Example: https://fts3-pilot.cern.ch:8446, https://fts3-pilot.cern.ch:8446.
    • globus_auth_app: (Optional) Default: None.
    • max_time_in_queue: (Optional) (separated by commas). Default: {}.
    • poll_timeout: Float, timeout. Default: None.
    • queue_mode: (Optional) Values: {'strict', 'default'}Default: default.
    • request_oidc_audience: (Optional). Default: fts:example.
    • request_oidc_scope: (Optional). Default: fts:submit-transfer.
    • scheme: (Optional) Schemes to process (separated by commas). Default: None.
    • submit_timeout: (Optional) Timeout. Default: None.
    • transfertool: (Optional) Default: None.
    • transfertype: (Optional). Values: {bulk, single}. Default: single.
    • usercert: Path to the certificate for the FTS3 implementation of a Rucio transfertool. Default: None.
    • user_activities: (Optional) Default: ['user', 'user_test'].
    • user_transfers: (Optional) Default: None.
    • using_memcache: (Optional) Boolean. Default: False.
    • webdav_transfer_mode: (Optional). Default: None.

  • core

    • default_mail_from: (Optional) Default email. Default: None.
    • geoip_licence_key: (Optional) License key for GeoLite2. Get a free licence key at https://www.maxmind.com/en/geolite2/signup. Default: NOLICENCE.
    • use_temp_tables: (Optional) Use Rucio with temporary table workflows. Default: False.

  • client

    • account: Rucio account. Example: root.
    • auth_host: URL of the host of the rucio authentication server. Example: https://rucio-auth-prod.cern.ch:443.
    • auth_token_file_path: (Optional) If token file path is defined in the rucio.cfg file, use that file. Currently this prevents authenticating as another user or VO. Default: None.
    • auth_type: Type of authentication in rucio. Values: {userpass, x509, x509_proxy, gss, ssh, saml, oidc}.
    • ca_cert: Path of the cert file for HTTPS. Example: /opt/rucio/etc/web/ca.crt.
    • client_cert: Path of the X.509 client cert file. This can be overwitten by the RUCIO_CLIENT_CERT environment variable. Example: /opt/rucio/etc/web/client.crt.
    • client_key: Path of the X.509 client key file for the cert defined in client_cert. This can be overwitten by the RUCIO_CLIENT_KEY environment variable. Example: /opt/rucio/etc/web/client.key.
    • client_x509_proxy: Path of the X.509 client proxy. Mandatory if auth_type = x509_proxy.
    • oidc_audience: (Optional) Only used if auth_type = oidc. Default: None.
    • oidc_auto: (Optional) Boolean. Only used if auth_type = oidc. Default: False.
    • oidc_issuer: (Optional) Only used if auth_type = oidc. Default: None.
    • oidc_password: (Optional) Only used if auth_type = oidc. Default: None.
    • oidc_refresh_lifetime: (Optional) Only used if auth_type = oidc. Default: None.
    • oidc_scope: (Optional) Only used if auth_type = oidc. Default: openid profile.
    • oidc_username: (Optional) Only used if auth_type = oidc. Default: None.
    • password: Password of the user specified in username. Mandatory if auth_type = userpass or auth_type = saml.
    • protocol_stat_retries: (Optional) Integer, number of retries if stat file fails. The time of the retries are: 1s, 2s, 4s, 8s, 16s, 32s later. Default: 6.
    • request_retries: (Optional) Integer, number of retries if an unauthorized error is returned. Default: 3.
    • rucio_host: URL of rucio host. Example: https://rucio-server-prod.cern.ch:443.
    • ssh_private_key: Path of the SSH private key. Mandatory if auth_type = ssh. Example: $HOME/.ssh/id_rsa.
    • username: Mandatory if auth_type = userpass or auth_type = saml.
    • vo: (Optional) VO name. Default: def.

  • credentials

    • gcs: (Optional) Path of the Google Cloud Storage credentials. Default: /opt/rucio/etc/google-cloud-storage-test.json.
    • signature_lifetime: (Optional) ?. Default: 600.

  • database

  • download

    • transfer_speed_timeout: (Optional) Minimum allowed average transfer speed (in KBps). Default: 500. Used to dynamically compute the timeout if --transfer-timeout not set. Is not supported for --pfn.
    • transfer_timeout: (Optional) Transfer timeout (in seconds). Default: computed dynamically from --transfer-speed-timeout. If set to any value >= 0, --transfer-speed-timeout is ignored.

  • es-atlas

    • ca_cert: (Optional) Path of the certificate for Elasticsearch. No default.
    • password: (Optional) Password of the username defined in username to authenticate to Elasticsearch. No default.
    • url: (Optional) URL of Elasticsearch. Example: http://aianalytics01.cern.ch:9200. No default.
    • username: (Optional) Username to authenticate to Elasticsearch. No default.

  • hermes

    • elastic_endpoint: (Optional) URL of Elasticsearch. Example: http://aianalytics01.cern.ch:9200. Mandatory if elastic is specified in services_list.
    • influxdb_endpoint: (Optional) URL of InfluxDB. Mandatory if influx is specified in services_list.

  • importer

    • attr_sync_method: (Optional) Values: {append, edit, hard}. Default: edit.
    • rse_sync_method: (Optional) Values: {append, edit, hard}. Default: edit.

  • injector

    • adler32
    • bytes
    • file
    • md5

  • lifetime

    • directory: (Optional) Path to the policies directory with JSON files named config_DTYPE.json, where DTYPE is a value in {data, mc, valid, other}. Default: /opt/rucio/etc/policies.

  • logging

    • CFG_OPTION: (Optional) ?. Default: None.

  • messaging-cache

    • account
    • broker_virtual_host: (Optional) ?
    • brokers: Default message broker name for rucio-cache-client. Ignored if rucio-cache-client executed with --broker.
    • destination: Default message broker topic for rucio-cache-client. Ignored if rucio-cache-client executed with --destination.
    • port
    • ssl_cert_file: Default certificate file for rucio-cache-client. Ignored if rucio-cache-client executed with --certificate.
    • ssl_key_file: Default certificate key file for rucio-cache-client. Ignored if rucio-cache-client executed with --certificate-key.
    • voname

  • messaging-fts3

    • broker_virtual_host: (Optional) ?. No default.
    • brokers: Brokers separated by commas. Example: dashb-test-mb.cern.ch.
    • destination: Name of the destination topic. Example: /topic/transfer.fts_monitoring_queue_state.
    • nonssl_port: (Optional) Port of the broker if use_ssl is not set.
    • password: (Optional) Password of the username. Only used if use_ssl is not set. No default.
    • port
    • ssl_cert_file: (Optional) Path of the certificate file. No default.
    • ssl_key_file: (Optional) Path of the certificate key file defined in ssl_cert_file. No default.
    • use_ssl: (Optional) Boolean. Default: True.
    • username: (Optional) Username of the broker. Only used if use_ssl is not set. No default.
    • voname

  • messaging-hermes

    • broker_virtual_host: (Optional) No default.
    • brokers: Brokers separated by commas. Example: atlas-test-mb.cern.ch.
    • destination: Name of the destination topic. Example: /topic/rucio.events.
    • email_from: Example: Rucio <spamspamspam@cern.ch>.
    • email_test: Example: spamspamspam@cern.ch.
    • nonssl_port: (Optional) Port of the broker if use_ssl is not set.
    • password: (Optional) Password of the username. Mandatory if use_ssl is not set. No default.
    • port: (Optional) Port of the broker if use_ssl is set.
    • ssl_cert_file: (Optional) Path of the certificate file. No default. Mandatory if use_ssl is set.
    • ssl_key_file: (Optional) Path of the certificate key file defined in ssl_cert_file. No default. Mandatory if use_ssl is set.
    • use_ssl: (Optional) Boolean. Default: True.
    • username: (Optional) Username of the broker. Mandatory if use_ssl is not set. No default.
    • voname

  • metadata

    • plugins: (Optional) Metadata handler modules separated by commas. Default: rucio.core.did_meta_plugins.json_meta.JSONDidMeta.

  • monitor

    • enable_metrics: (Optional) Enable statsd metrics. Boolean. Default: False.
    • carbon_server: (Optional) Hostname or IP address of the statsd server. Default: localhost
    • carbon_port: (Optional) Port of the statsd server. Default: 8125.
    • user_scope: (Optional) Prefix to distinguish and group stats from an application or environment. Default: rucio.
    • metrics_port: (Optional) Port of Prometheus Python Client. Default: 8080.

  • nagios

    • fts_servers
    • proxy
    • rfcproxy

  • nongrid-trace

    • broker_virtual_host: (Optional) ?. No default.

    • brokers: Brokers separated by commas. Example: atlas-test-mb.cern.ch.

    • logformat: (Optional) Formatter of the log. See: https://docs.python.org/3/library/logging.html#logging.Formatter

    • loglevel: (Optional) Set the root logger level to the specified level.

      Values: {'NOTSET', 'DEBUG', 'INFO', 'WARNING', 'ERROR', 'CRITICAL'}. See: https://docs.python.org/3/library/logging.html#levels

      Default: DEBUG.

    • password: Password of the username.

    • topic: Name of the destination topic.

    • tracedir: (Optional) Path of the directory for traces. Default: /var/log/rucio.

    • username: Username of the broker.

  • oidc

    • admin_issuer: Example: wlcg.
    • default_jwt_refresh_lifetime: (Optional) Integer. Default: 96.
    • exchange_grant_type: (Optional) Default: urn:ietf:params:oauth:grant-type:token-exchange.
    • expected_audience: (Optional) Default: rucio.
    • expected_scope: (Optional) Default: openid profile.
    • idpsecrets: Path of the idpsecrets JSON. Example: /opt/rucio/etc/idpsecrets.json.

  • permission

    • policy: (Optional) Permission policy. Values: {atlas, belleii, cms, generic, generic_multi_vo}. Default: generic.

  • policy

    • lfn2pfn_algorithm_default: (Optional) Default algorithm name for LFN2PFN translation for this server. Default: hash.
    • package
    • package-VO
    • permission: Same as policy/permission.
    • schema
    • scratchdisk_lifetime: (Optional) Integer. Default: 14.
    • support: (Optional) Contact information.
    • support_rucio: (Optional) Rucio contact information. Default: https://github.com/rucio/rucio/issues.

  • saml

    • config_path: Path to the SAML config folder. Example: /opt/rucio/lib/rucio/web/ui/common/saml/.

  • test

    • cacert: Path of the CA certificate for tests. Example: /opt/rucio/etc/web/ca.crt
    • usercert Path of the user certificate for tests. Example: /opt/rucio/etc/web/usercert.pem
    • userkey: Path of the user certificate key for tests.

  • trace

    • broker_virtual_host: (Optional) No default.

    • brokers: Brokers separated by commas. Example: atlas-test-mb.cern.ch.

    • logformat: (Optional) Formatter of the log. See: https://docs.python.org/3/library/logging.html#logging.Formatter

    • loglevel: (Optional) Set the root logger level to the specified level

      Values: {'NOTSET', 'DEBUG', 'INFO', 'WARNING', 'ERROR', 'CRITICAL'}. See: https://docs.python.org/3/library/logging.html#levels

      Default: DEBUG.

    • password: Password of the username.

    • port: (Optional) Port of the broker. Example: 61013.

    • topic: Name of the destination topic. Example: /topic/rucio.tracer.

    • tracedir: (Optional) Path of the directory for traces. Default: /var/log/rucio/trace.

    • username: Username of the broker.

  • tracer-kronos

    • broker_virtual_host: (Optional) No default.
    • brokers: Brokers separated by commas. Example: atlas-test-mb.cern.ch.
    • chunksize: Integer
    • excluded_usrdns: Separated by commas. Example: CN=proxy,CN=Robot: Ganga Robot,CN=722147,CN=gangarbt,OU=Users,OU=Organic Units,DC=cern,DC=ch.
    • password: (Optional) Password of the username. Mandatory if use_ssl is not set. No default.
    • port: Port of the broker.
    • prefetch_size: activemq.prefetchSize, see https://activemq.apache.org/what-is-the-prefetch-limit-for
    • queue: The topic or queue to subscribe to. Example: /queue/Consumer.kronos.rucio.tracer.
    • reconnect_attempts: Maximum attempts to reconnect. Integer. Example: 100.
    • ssl_cert_file: (Optional) Path of the certificate file. No default. Mandatory if use_ssl is set.
    • ssl_key_file: (Optional) Path of the certificate key file defined in ssl_cert_file. No default. Mandatory if use_ssl is set.
    • subscription_id: A unique id to represent the subscription. Example: rucio-tracer-listener.
    • use_ssl: (Optional) Boolean. Default: True.
    • username: (Optional) Username of the broker. Mandatory if use_ssl is not set. No default.

  • transmogrifier

    • maxdids

  • upload

    • transfer_timeout: (Optional) Transfer timeout (seconds, integer). Default: 360.

  • webui

    • auth_issuer: (Optional) Mandatory if auth_type = oidc. No default.
    • auth_type: (Optional) Preferred server side config for webui authentication. Values: {oidc, None}. Default: None.
    • usercert
    • urls: A CSV specifying urls of Rucio WebUI 2.0 clients. Required for correctly handling pre-flight CORS requests.

Rucio configuration table​

  • automatix
    • account: (Optional) Account to use. Default: root.
    • dataset_lifetime: (Optional). Default: None.
    • did_prefix: (Optional) Default: .
    • DIDTYPE_pattern: (Optional) Separated by separator char. No default.
    • scope: (Optional) Default: False.
    • separator: (Optional) Separator char. No default.
    • set_metadata: (Optional) Default: False.
    • sites: Separated by commas.
    • sleep_time: (Optional) Integer. Default: 30.
  • clientcachemap
    • client_location['site']
  • conveyor
    • activity-source-strategy: (Optional) Default: {}.
    • default-source-strategy: (Optional) Default: orderly.
  • hermes
    • services_list: List of services separated by commas. Values: {activemq, elastic, influx}.
  • kronos
    • bad_files_patterns: (Optional) Patterns (regular expression) separated by commas for bad files. Default: [].
  • lifetime_model
    • approvers_email: (Optional) Separated by commas. Default: [].
  • reaper
    • auto_exclude_threshold: (Optional) Number of service unavailable exceptions after which the RSE gets temporarily excluded. Integer. Default: 100.
    • auto_exclude_timeout: Timeout for temporarily excluded RSEs. Integer. Default: 600.
    • maxdeletion_threadsHOSTNAME: (Optional) Max number of deletion threads (integer). If nb_workers_by_hostname is also not defined, default: 5.
    • max_evaluator_backlog_count: (Optional) Integer. Default: None.
    • max_evaluator_backlog_duration: (Optional) Minutes (integer). Default: None.
    • nb_workers_by_hostname: (Optional) Integer. Default: 5.
  • root-proxy-internal
    • client_location['site']
  • rules
    • apply_rule_max_partition_size: (Optional) Integer . Default: 2000.
    • force_epoch_when_detach: (Optional) Purge setting of the rule. Boolean. Default: False.
    • use_new_rule_algorithm: (Optional) Boolean. Default: False.
  • subscriptions
    • keep_history: (Optional) Boolean. Default: False.
    • reevaluate_dids_at_close: (Optional) Flag to reevaluate the DID against all the subscriptions when the DID is closed. Boolean. Default: False.
  • transfers
    • hop_penalty: (Optional) Penalty to be applied to each further hop. Integer. Default: 10.
    • multihop_tombstone_delay: Seconds (integer). Default: 7200.
    • use_multihop: Boolean. Default: False.
    • fts3tape_metadata_plugins: (Optional) Plugins to use with FTS3 to include archive metadata in the transfer process. List[String]. Default: None.
    • metadata_byte_limit: (Optional) Limit applied to archive_metadata during a transfer. Only used with archive metadata plugins using FTS3. Integer. Defalt None.
  • virtual_placement
    • vp_endpoint: (Optional) Virtual Placement server. Once VP is integrated in Rucio it won't be needed. Default: .
  • vo-map
    • VO: (Optional) Internal short VO name. No default.

RSE attributes​

  • associated_sites: Separated by commas. Default: None.
  • auto_approve_bytes: Auto Approve Limit. Example: 500GB. No default.
  • auto_approve_files: No default.
  • available_for_multihop: Boolean. If True, allow to use this RSE as an intermediate hop in a multi-hop transfer. Default: False.
  • block_manual_approval: Boolean. Default: False.
  • country: No default.
  • greedyDeletion: Boolean. Default: False.
  • hop_penalty: Integer. Cost of passing via this RSE in multi-hop transfers. Overrides the global transfers/hop_penalty configuration value for this particular RSE. Has only meaning if available_for_multihop attribute is True on the RSE. No default value.
  • istape: Boolean. Default: False.
  • mock: Boolean. Default: False.
  • naming_convention: Default: None.
  • physgroup: Default: .
  • quota_approvers: Separated by commas. Default: None.
  • restricted_read Boolean. If True, only allow transfers from this RSE if started by an account with admin privileges. Default: False
  • restricted_write Boolean. Same as restricted_read, but for transfers towards this RSE. Default: False
  • rule_approvers: Separated by commas. No default.
  • rule_deleters: Separated by commas. No default.
  • site
  • skip_upload_stat: Boolean. Default: False.
  • source_for_total_space: Default: storage.
  • source_for_used_space: Default: storage.
  • staging_buffer
  • tier: Datacenter TIER. Integer (from 1 to 4). No default.
  • type: Values: {LOCALGROUPDISK, LOCALGROUPTAPE, GROUPDISK, SCRATCHDISK, MOCK, TEST, DATADISK}. Default: .
  • verify_checksum: Boolean. No default.