Replicaclient

Classes¶

ReplicaClient ¶

ReplicaClient(
    rucio_host=None,
    auth_host=None,
    account=None,
    ca_cert=None,
    auth_type=None,
    creds=None,
    timeout=600,
    user_agent="rucio-clients",
    vo=None,
    logger=LOG,
)

Replica client class for working with replicas

Functions¶

quarantine_replicas ¶

quarantine_replicas(replicas, rse=None, rse_id=None)

Add quaratined replicas for RSE.

PARAMETER	DESCRIPTION
`replicas`	List of replica infos: {'scope': (optional), 'name': (optional), 'path': (required)}.
`rse`	RSE name. DEFAULT: `None`
`rse_id`	RSE id. Either RSE name or RSE id must be specified, but not both DEFAULT: `None`

declare_bad_file_replicas ¶

declare_bad_file_replicas(replicas, reason, force=False)

Declare a list of bad replicas.

PARAMETER	DESCRIPTION
`replicas`	Either a list of PFNs (string) or a list of dicts {'scope': , 'name': , 'rse_id': or 'rse': }
`reason`	The reason of the loss.
`force`	boolean, tell the serrver to ignore existing replica status in the bad_replicas table. Default: False DEFAULT: `False`

RETURNS	DESCRIPTION
	Dictionary {"rse_name": ["did: error",...]} - list of strings for DIDs failed to declare, by RSE

declare_bad_did_replicas ¶

declare_bad_did_replicas(rse, dids, reason)

Declare a list of bad replicas.

PARAMETER	DESCRIPTION
`rse`	The RSE where the bad replicas reside
`dids`	The DIDs of the bad replicas
`reason`	The reason of the loss.

declare_suspicious_file_replicas ¶

declare_suspicious_file_replicas(pfns, reason)

Declare a list of bad replicas.

PARAMETER	DESCRIPTION
`pfns`	The list of PFNs.
`reason`	The reason of the loss.

get_did_from_pfns ¶

get_did_from_pfns(pfns, rse=None)

Get the DIDs associated to a PFN on one given RSE

PARAMETER	DESCRIPTION
`pfns`	The list of PFNs.
`rse`	The RSE name. DEFAULT: `None`

RETURNS	DESCRIPTION
	A list of dictionaries {pfn: {'scope': scope, 'name': name}}

list_replicas ¶

list_replicas(
    dids,
    schemes=None,
    ignore_availability=True,
    all_states=False,
    metalink=False,
    rse_expression=None,
    client_location=None,
    sort=None,
    domain=None,
    signature_lifetime=None,
    nrandom=None,
    resolve_archives=True,
    resolve_parents=False,
    updated_after=None,
)

List file replicas for a list of data identifiers (DIDs).

PARAMETER	DESCRIPTION
`dids`	The list of data identifiers (DIDs) like : [{'scope': , 'name': }, {'scope': , 'name': }, ...]
`schemes`	A list of schemes to filter the replicas. (e.g. file, http, ...) DEFAULT: `None`
`ignore_availability`	Also include replicas from blocked RSEs into the list DEFAULT: `True`
`metalink`	`False` (default) retrieves as JSON, `True` retrieves as metalink4+xml. DEFAULT: `False`
`rse_expression`	The RSE expression to restrict replicas on a set of RSEs. DEFAULT: `None`
`client_location`	Client location dictionary for PFN modification {'ip', 'fqdn', 'site', 'latitude', 'longitude'} DEFAULT: `None`
`sort`	Sort the replicas: `geoip` - based on src/dst IP topographical distance DEFAULT: `None`
`domain`	Define the domain. None is fallback to 'wan', otherwise 'wan, 'lan', or 'all' DEFAULT: `None`
`signature_lifetime`	If supported, in seconds, restrict the lifetime of the signed PFN. DEFAULT: `None`
`nrandom`	pick N random replicas. If the initial number of replicas is smaller than N, returns all replicas. DEFAULT: `None`
`resolve_archives`	When set to True, find archives which contain the replicas. DEFAULT: `True`
`resolve_parents`	When set to True, find all parent datasets which contain the replicas. DEFAULT: `False`
`updated_after`	epoch timestamp or datetime object (UTC time), only return replicas updated after this time DEFAULT: `None`

RETURNS	DESCRIPTION
	A list of dictionaries with replica information.

list_suspicious_replicas ¶

list_suspicious_replicas(
    rse_expression=None, younger_than=None, nattempts=None
)

List file replicas tagged as suspicious.

PARAMETER	DESCRIPTION
`rse_expression`	The RSE expression to restrict replicas on a set of RSEs. DEFAULT: `None`
`younger_than`	Datetime object to select the replicas which were declared since younger_than date. Default value = 10 days ago. DEFAULT: `None`
`nattempts`	The minimum number of replica appearances in the bad_replica DB table from younger_than date. Default value = 0. DEFAULT: `None`
`state`	State of the replica, either 'BAD' or 'SUSPICIOUS'. No value returns replicas with either state.

add_replica ¶

add_replica(
    rse,
    scope,
    name,
    bytes_,
    adler32,
    pfn=None,
    md5=None,
    meta=None,
)

Add file replicas to a RSE.

PARAMETER	DESCRIPTION
`rse`	the RSE name. TYPE: `str`
`scope`	The scope of the file. TYPE: `str`
`name`	The name of the file. TYPE: `str`
`bytes_`	The size in bytes. TYPE: `int`
`adler32`	adler32 checksum. TYPE: `str`
`pfn`	PFN of the file for non deterministic RSE. TYPE: `Optional[str]` DEFAULT: `None`
`md5`	md5 checksum. TYPE: `Optional[str]` DEFAULT: `None`
`meta`	Metadata attributes. TYPE: `Optional[dict[str, Any]]` DEFAULT: `None`

RETURNS	DESCRIPTION
`bool`	True if files were created successfully.

add_replicas ¶

add_replicas(rse, files, ignore_availability=True)

Bulk add file replicas to a RSE.

PARAMETER	DESCRIPTION
`rse`	the RSE name.
`files`	The list of files. This is a list of DIDs like : [{'scope': , 'name': }, {'scope': , 'name': }, ...]
`ignore_availability`	Ignore the RSE blocklsit. DEFAULT: `True`

RETURNS	DESCRIPTION
	True if files were created successfully.

delete_replicas ¶

delete_replicas(rse, files, ignore_availability=True)

Bulk delete file replicas from a RSE.

PARAMETER	DESCRIPTION
`rse`	the RSE name.
`files`	The list of files. This is a list of DIDs like : [{'scope': , 'name': }, {'scope': , 'name': }, ...]
`ignore_availability`	Ignore the RSE blocklist. DEFAULT: `True`

RETURNS	DESCRIPTION
	True if files have been deleted successfully.

update_replicas_states ¶

update_replicas_states(rse, files)

Bulk update the file replicas states from a RSE.

PARAMETER	DESCRIPTION
`rse`	the RSE name.
`files`	The list of files. This is a list of DIDs like : [{'scope': , 'name': , 'state': }, {'scope': , 'name': , 'state': }, ...], where a state value can be either of: 'A' (AVAILABLE) 'U' (UNAVAILABLE) 'C' (COPYING) 'B' (BEING_DELETED) 'D' (BAD) 'T' (TEMPORARY_UNAVAILABLE)

RETURNS	DESCRIPTION
	True if replica states have been updated successfully, otherwise an exception is raised.

list_dataset_replicas ¶

list_dataset_replicas(scope, name, deep=False)

List dataset replicas for a did (scope:name).

PARAMETER	DESCRIPTION
`scope`	The scope of the dataset.
`name`	The name of the dataset.
`deep`	Lookup at the file level. DEFAULT: `False`

RETURNS	DESCRIPTION
	A list of dict dataset replicas.

list_dataset_replicas_bulk ¶

list_dataset_replicas_bulk(dids)

List dataset replicas for a did (scope:name).

PARAMETER	DESCRIPTION
`dids`	The list of DIDs of the datasets.

RETURNS	DESCRIPTION
	A list of dict dataset replicas.

list_dataset_replicas_vp ¶

list_dataset_replicas_vp(scope, name, deep=False)

List dataset replicas for a DID (scope:name) using the Virtual Placement service.

NOTICE: This is an RnD function and might change or go away at any time.

PARAMETER	DESCRIPTION
`scope`	The scope of the dataset.
`name`	The name of the dataset.
`deep`	Lookup at the file level. DEFAULT: `False`

RETURNS	DESCRIPTION
	If VP exists a list of dicts of sites

list_datasets_per_rse ¶

list_datasets_per_rse(rse, filters=None, limit=None)

List datasets at a RSE.

PARAMETER	DESCRIPTION
`rse`	the rse name.
`filters`	dictionary of attributes by which the results should be filtered. DEFAULT: `None`
`limit`	limit number. DEFAULT: `None`

RETURNS	DESCRIPTION
	A list of dict dataset replicas.

add_bad_pfns ¶

add_bad_pfns(pfns, reason, state, expires_at)

Declare a list of bad replicas.

PARAMETER	DESCRIPTION
`pfns`	The list of PFNs.
`reason`	The reason of the loss.
`state`	The state of the replica. Either BAD, SUSPICIOUS, TEMPORARY_UNAVAILABLE
`expires_at`	Specify a timeout for the TEMPORARY_UNAVAILABLE replicas. None for BAD files.

RETURNS	DESCRIPTION
	True if PFNs were created successfully.

set_tombstone ¶

set_tombstone(replicas)

Set a tombstone on a list of replicas.

PARAMETER	DESCRIPTION
`replicas`	list of replicas.