Data Identifier Methods

Copyright European Organization for Nuclear Research (CERN)

Licensed under the Apache License, Version 2.0 (the “License”); You may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0

Authors: - Vincent Garonne, <vincent.garonne@cern.ch>, 2012-2015 - Mario Lassnig, <mario.lassnig@cern.ch>, 2012-2013 - Thomas Beermann, <thomas.beermann@cern.ch> 2013 - Yun-Pin Sun, <yun-pin.sun@cern.ch>, 2013 - Cedric Serfon, <cedric.serfon@cern.ch>, 2014-2015 - Ralph Vigne, <ralph.vigne@cern.ch>, 2015 - Martin Barisits, <martin.barisits@cern.ch>, 2014-2015

class rucio.client.didclient.DIDClient(rucio_host=None, auth_host=None, account=None, ca_cert=None, auth_type=None, creds=None, timeout=None, user_agent='rucio-clients')[source]

Bases: rucio.client.baseclient.BaseClient

DataIdentifier client class for working with data identifiers

DIDS_BASEURL = 'dids'
add_container(scope, name, statuses=None, meta=None, rules=None, lifetime=None)[source]

Add data identifier for a container.

Parameters:
  • scope – The scope name.
  • name – The data identifier name.
  • statuses – Dictionary with statuses, e.g.g {‘monotonic’:True}.
  • lifetime – DID’s lifetime (in seconds).
Meta:

Meta-data associated with the data identifier is represented using key/value pairs in a dictionary.

Rules:

Replication rules associated with the data identifier. A list of dictionaries, e.g., [{‘copies’: 2, ‘rse_expression’: ‘TIERS1’}, ].

add_containers(cnts)[source]

Bulk add containers.

Parameters:cnts – A list of containers.
add_containers_to_container(scope, name, cnts)[source]

Add containers to container.

Parameters:
  • scope – The scope name.
  • name – The dataset name.
  • dsns – The content.
add_containers_to_containers(attachments)[source]

Add containers_to_containers.

Parameters:attachments – The attachments. attachments is: [attachment, attachment, ...] attachment is: {‘scope’: scope, ‘name’: name, ‘dids’: dids} dids is: [{‘scope’: scope, ‘name’: name}, ...]
add_dataset(scope, name, statuses=None, meta=None, rules=None, lifetime=None, files=None, rse=None)[source]

Add data identifier for a dataset.

Parameters:
  • scope – The scope name.
  • name – The data identifier name.
  • statuses – Dictionary with statuses, e.g.g {‘monotonic’:True}.
  • lifetime – DID’s lifetime (in seconds).
  • files – The content.
  • rse – The RSE name when registering replicas.
Meta:

Meta-data associated with the data identifier is represented using key/value pairs in a dictionary.

Rules:

Replication rules associated with the data identifier. A list of dictionaries, e.g., [{‘copies’: 2, ‘rse_expression’: ‘TIERS1’}, ].

add_datasets(dsns)[source]

Bulk add datasets.

Parameters:dsns – A list of datasets.
add_datasets_to_container(scope, name, dsns)[source]

Add datasets to container.

Parameters:
  • scope – The scope name.
  • name – The dataset name.
  • dsns – The content.
add_datasets_to_containers(attachments)[source]

Add datasets_to_containers.

Parameters:attachments – The attachments. attachments is: [attachment, attachment, ...] attachment is: {‘scope’: scope, ‘name’: name, ‘dids’: dids} dids is: [{‘scope’: scope, ‘name’: name}, ...]
add_did(scope, name, type, statuses=None, meta=None, rules=None, lifetime=None, dids=None, rse=None)[source]

Add data identifier for a dataset or container.

Parameters:
  • scope – The scope name.
  • name – The data identifier name.
  • statuses – Dictionary with statuses, e.g.g {‘monotonic’:True}.
  • lifetime – DID’s lifetime (in seconds).
  • dids – The content.
  • rse – The RSE name when registering replicas.
Paran type:

The data identifier type (file|dataset|container).

Meta:

Meta-data associated with the data identifier is represented using key/value pairs in a dictionary.

Rules:

Replication rules associated with the data identifier. A list of dictionaries, e.g., [{‘copies’: 2, ‘rse_expression’: ‘TIERS1’}, ].

add_dids(dids)[source]

Bulk add datasets/containers.

add_files_to_dataset(scope, name, files, rse=None)[source]

Add files to datasets.

Parameters:
  • scope – The scope name.
  • name – The dataset name.
  • files – The content.
  • rse – The RSE name when registering replicas.
add_files_to_datasets(attachments, ignore_duplicate=False)[source]

Add files to datasets.

Parameters:
  • attachments – The attachments. attachments is: [attachment, attachment, ...] attachment is: {‘scope’: scope, ‘name’: name, ‘dids’: dids} dids is: [{‘scope’: scope, ‘name’: name}, ...]
  • ignore_duplicate – If True, ignore duplicate entries.
attach_dids(scope, name, dids, rse=None)[source]

Attach data identifier.

Parameters:
  • scope – The scope name.
  • name – The data identifier name.
  • dids – The content.
  • rse – The RSE name when registering replicas.
attach_dids_to_dids(attachments, ignore_duplicate=False)[source]

Add dids to dids.

Parameters:attachments – The attachments. attachments is: [attachment, attachment, ...] attachment is: {‘scope’: scope, ‘name’: name, ‘dids’: dids} dids is: [{‘scope’: scope, ‘name’: name}, ...] :param ignore_duplicate: If True, ignore duplicate entries.
close(scope, name)[source]

close dataset/container

Parameters:
  • scope – The scope name.
  • name – The dataset/container name.
create_did_sample(input_scope, input_name, output_scope, output_name, nbfiles)[source]

Create a sample from an input collection.

Parameters:
  • input_scope – The scope of the input DID.
  • input_name – The name of the input DID.
  • output_scope – The scope of the output dataset.
  • output_name – The name of the output dataset.
  • account – The account.
  • nbfiles – The number of files to register in the output dataset.
  • session – The database session in use.
delete_metadata(scope, name, key)[source]

Delete data identifier metadata

Parameters:
  • scope – The scope name.
  • name – The data identifier.
  • key – the key.
detach_dids(scope, name, dids)[source]

Detach data identifier

Parameters:
  • scope – The scope name.
  • name – The data identifier name.
  • dids – The content.
get_dataset_by_guid(guid)[source]
Get the parent datasets for a given GUID.
Parameters:guid

The GUID.

returns:A did
get_did(scope, name)[source]

Retrieve a single data identifier.

Parameters:
  • scope – The scope name.
  • name – The data identifier name.
get_metadata(scope, name)[source]

Get data identifier metadata

Parameters:
  • scope – The scope name.
  • name – The data identifier name.
list_associated_rules_for_file(scope, name)[source]

List the associated rules a file is affected from..

Parameters:
  • scope – The scope name.
  • name – The file name.
list_content(scope, name)[source]

List data identifier contents.

Parameters:
  • scope – The scope name.
  • name – The data identifier name.
list_did_rules(scope, name)[source]

List the associated rules of a data identifier.

Parameters:
  • scope – The scope name.
  • name – The data identifier name.
list_dids(scope, filters, type='collection', long=False)[source]

List all data identifiers in a scope which match a given pattern.

Parameters:
  • scope – The scope name.
  • filters – A dictionary of key/value pairs like {‘name’: ‘file_name’,’rse-expression’: ‘tier0’}.
  • type – The type of the did: ‘all’(container, dataset or file)|’collection’(dataset or container)|’dataset’|’container’|’file’
  • long – Long format option to display more information for each DID.
list_files(scope, name, long=None)[source]

List data identifier file contents.

Parameters:
  • scope – The scope name.
  • name – The data identifier name.
  • long – A boolean to choose if GUID is returned or not.
list_parent_dids(scope, name)[source]

List parent dataset/containers of a did.

Parameters:
  • scope – The scope.
  • name – The name.
scope_list(scope, name=None, recursive=False)[source]

List data identifiers in a scope.

Parameters:
  • scope – The scope name.
  • name – The data identifier name.
  • recursive – boolean, True or False.
set_metadata(scope, name, key, value)[source]

Set data identifier metadata

Parameters:
  • scope – The scope name.
  • name – The data identifier name.
  • key – the key.
  • value – the value.
set_status(scope, name, **kwargs)[source]

Set data identifier status

Parameters:
  • scope – The scope name.
  • name – The data identifier name.
  • kwargs – Keyword arguments of the form status_name=value.