usage: rucio-conveyor-submitter [-h] [--run-once]
[--total-threads TOTAL_THREADS] [--bulk BULK]
[--group-bulk GROUP_BULK]
[--group-policy GROUP_POLICY]
[--source-strategy SOURCE_STRATEGY]
[--exclude-rses EXCLUDE_RSES]
[--include-rses INCLUDE_RSES]
[--rses RSES [RSES ...]] [--vos VOS [VOS ...]]
[--activities ACTIVITIES [ACTIVITIES ...]]
[--exclude-activities EXCLUDE_ACTIVITIES [EXCLUDE_ACTIVITIES ...]]
[--ignore-availability]
[--sleep-time SLEEP_TIME]
[--max-sources MAX_SOURCES]
[--archive-timeout-override INTEGER_SECONDS]
The Conveyor-Submitter daemon is responsible for managing non-tape file transfers. It prepares transfer jobs and submits them to the transfertool.
optional arguments:
-h, --help show this help message and exit
--run-once One iteration only
--total-threads TOTAL_THREADS
Concurrency control: total number of threads per
process
--bulk BULK Bulk control: number of requests
--group-bulk GROUP_BULK
Group control: number of requests per group
--group-policy GROUP_POLICY
Group control: policy used to group. enum{rule, dest,
src_dest, rule_src_dest, activity_dest,
activity_src_dest}
--source-strategy SOURCE_STRATEGY
Source strategy. Overload the strategy defined in
config DB.
--exclude-rses EXCLUDE_RSES
RSE expression to exclude
--include-rses INCLUDE_RSES
RSE expression to include
--rses RSES [RSES ...]
Explicit list of RSEs to include
--vos VOS [VOS ...] Optional list of VOs to consider. Only used in multi-
VO mode.
--activities ACTIVITIES [ACTIVITIES ...]
Explicit list of activities to include
--exclude-activities EXCLUDE_ACTIVITIES [EXCLUDE_ACTIVITIES ...]
Explicit list of activities to exclude
--ignore-availability
If True, will also try to submit transfers having
blocklisted RSEs as sources
--sleep-time SLEEP_TIME
Seconds to sleep if few requests
--max-sources MAX_SOURCES
Maximum source replicas per multi-source FTS job
--archive-timeout-override INTEGER_SECONDS
Override the archive_timeout parameter for any
transfers with it set (0 to unset)
Upload a file and create a replication rule::
$ rucio upload --scope mock --rse MOCK --name file filename.txt
$ rucio add-rule mock:file 1 MOCK2
$ rucio-admin rse add-distance MOCK2 MOCK --distance 1
The rule should replicate the file from RSE MOCK to RSE MOCK2. Therefor a distance between these RSEs is needed.
Check transfer requests for the DID::
$ python
from rucio.db.sqla import session,models
session.get_session().query(models.Request).filter_by(scope='mock', name='file').first()
# {'request_type': TRANSFER, 'state': QUEUED', ...}
A queued request was created which can be picked uped by the Conveyor-Submiter daemon.
Run the daemon::
$ rucio-conveyor-submitter --run-once
Check again the transfer requests for the DID::
$ python
from rucio.db.sqla import session,models
session.get_session().query(models.Request).filter_by(scope='mock', name='file').first()
# {'request_type': TRANSFER, 'state': SUBMITTED', ...}
A tranfer request got created by executing the transfer. Depending on the transfer submission, the request state can be different. In this example the transfer got submitted successfully.
When run in multi-VO mode, by default the daemon will run on RSEs from all VOs::
$ rucio-conveyor-submitter --run-once
2020-07-29 13:51:09,436 5784 INFO This instance will work on VOs: def, abc, xyz, 123
2020-07-29 13:51:13,315 5784 INFO RSE selection: automatic for relevant VOs
2020-07-29 13:51:13,316 5784 INFO starting submitter threads
By using the ``--vos`` argument only the VO or VOs specified will be affected::
$ rucio-conveyor-submitter --run-once --vos abc xyz
2020-07-29 13:51:09,436 5784 INFO This instance will work on VOs: abc, xyz
2020-07-29 13:51:13,315 5784 INFO RSE selection: automatic for relevant VOs
2020-07-29 13:51:13,316 5784 INFO starting submitter threads
Note that attempting the use the ``--vos`` argument when in single-VO mode will have no affect::
$ rucio-conveyor-submitter --run-once --vos abc xyz
2020-07-29 13:39:37,263 5752 INFO RSE selection: automatic
2020-07-29 13:39:37,264 5752 INFO starting submitter threads