1.30.0rc1

Release Candidate

This is a release candidate! Only to be used in testing environments!

Upgrade Instructions

This feature release requires a database schema upgrade. Please consult the documentation about upgrading your database schema.

The following changes are necessary and are covered by the alembic upgrade head command:

1. Correcting length of rse_expression in rule history tables (Alembic revision 83f991c63a93)

2. Conveyor throttler evolution (Alembic revision 13d4f70c66a9)

   • Creation of transfer_limits table
   • Drop and re-creation of rse_transfer_limits table

3. Dropping/Renaming of several columns in the distances table (Alembic revision 140fef722e91)

Important changes

• Most metric names were changed in this release. If you collect prometheus and/or statsd metrics, the monitoring dashboards and alerting rules will have to be updated.

  • Important Note (unrelated to this particular release)
    • The prometheus metrics are the main supported way of collecting metrics since 1.28.
    • There are no plans to deprecate carbon metrics yet, but there is no active development in this field anymore.
    • This update may be a good opportunity to switch to native prometheus metrics. Especially if you use prometheus anyway and rely on the statsd exporter to bridge the gap between statsd and prometheus.

• Deactivation of automatic https-\>davs translation for transfers

  • This automatic translation was necessary in the past but should be largely deprecated due to more recent versions of storage software
  • If you notice issues related to this, you can turn translation back on by setting the config option [transfers]rewrite_https_to_davs=True

• Support RFC format for X509 identities

  • There are two formats: an RFC-compliant (e.g. CN=John Doe,OU=Users,DC=example,DC=com) and a legacy (e.g. /DC=com/DC=example/OU=Users/CN=John Doe)
  • In Rucio 1.29 and prior, X509 identities must be stored in the legacy format and the authentication server tries to silently convert them from the RFC to the legacy format, if necessary
  • In Rucio 1.30, this conversion has been removed
    • Communities who would like to keep using the legacy format must ensure that LegacyDNStringFormat in mod_ssl is enabled

• Almost all fields from the distances table are removed. The only important field preserved from 1.29 to 1.30 is ranking (renamed to distance). Rucio 1.30 is compatible both with the old 1.29 table format and the 1.30 table format. If you desire to run Rucio 1.29 and 1.30 in parallel, database migrations associated with the "distances" table must be delayed until the full migration of all daemons and servers to 1.30

• The throttler rules format has changed in an incompatible way. The update of the database schema will result in removal of all existing throttling rules. Rules in the new format must be created before running throttler again, otherwise all waiting requests will be transitioned to queued state and be submitted without throttling.

General

Features

• Authentication & Authorisation: Support RFC format for X509 identities #5842
• Core & Internals: Introduce heartbeat handler to all daemons #5443

Enhancements

• Core & Internals: Remove retrying dependency #5344
• Core & Internals: Proper versioning for policy packages #5652
• Core & Internals: Enforce tokens by default in Flask #5822
• Core & Internals: switch to lazy-filling of account_usage counters #5914
• Core & Internals: fix type of 'session' function arguments #5947
• Monitoring & Logging: rework prometheus/statsd metric names #5804
• Protocols: Remove pysftp protocol #5347
• REST & API: remove traces endpoint from the default list in flask #5958
• Subscriptions: Introduce new exclude_site algorithm for chained subscriptions #5935
• Testing: Fix unit tests for non-ATLAS policies round 2/3 #5839
• Transfers: Re-work/cleanup distance table #5454
• Transfers: rework conveyor-throttler #5805
• Transfers: remove https->davs translation on FTS submission #5993

Bugs

• Database: Alembic context issue with move_rse_artributes_to_rse_attributes migration #5823
• Database: incorrect rse_expression length in rule_history* migrations #5846
• globus RSE protocol silently ignores errors #5987
• Protocols: webdav.py Default.stat() module: Match error when finding size of current file #5308
• Protocols: WebDAV protocol XML parser doesn't handle XML namespaces correctly #5700
• Protocols: bad-str-strip-call errors in pylint #6002

Clients

Enhancements

• Clients: Fix Potential for time zone confusion #5781

WebUI

Enhancements

• WebUI: r2d2 list_rules link is broken, gives 404 #5970