RSE Classes

Client

class rucio.client.rseclient.RSEClient(rucio_host=None, auth_host=None, account=None, ca_cert=None, auth_type=None, creds=None, timeout=None, user_agent='rucio-clients')[source]

Bases: rucio.client.baseclient.BaseClient

RSE client class for working with rucio RSEs

RSE_BASEURL = 'rses'
add_protocol(rse, params)[source]

Sends the request to create a new protocol for the given RSE.

Parameters:
  • rse – the name of the rse.
  • scheme – identifier of this protocol
  • params

    Attributes of the protocol. Supported are: hostname: hostname for this protocol (default = localhost) port: port for this protocol (default = 0) prefix: string used as a prfeix for this protocol when generating

    the PFN (default = None)
    impl: qualified name of the implementation class for this
    protocol (mandatory)
    read: integer representing the priority of this procotol for
    read operations (default = -1)
    write: integer representing the priority of this procotol for
    write operations (default = -1)
    delete: integer representing the priority of this procotol for
    delet operations (default = -1)
    extended_attributes: miscellaneous protocol specific information e.g. spacetoken
    for SRM (default = None)
Returns:

True if protocol was created successfully else False.

Raises:
  • Duplicate – if protocol with same hostname, port and protocol identifier already exists for the given RSE.
  • RSENotFound – if the RSE doesn’t exist.
  • KeyNotFound – if params is missing manadtory attributes to create the protocol.
  • AccessDenied – if not authorized.
add_rse(rse, **kwargs)[source]

Sends the request to create a new RSE.

Parameters:
  • rse – the name of the rse.
  • deterministic – Boolean to know if the pfn is generated deterministically.
  • volatile – Boolean for RSE cache.
  • city – City for the RSE.
  • region_code – The region code for the RSE.
  • country_name – The country.
  • continent – The continent.
  • time_zone – Timezone.
  • staging_area – Staging area.
  • ISP – Internet service provider.
Returns:

True if location was created successfully else False.

Raises Duplicate:
 

if rse already exists.

add_rse_attribute(rse, key, value)[source]

Sends the request to add a RSE attribute.

Parameters:
  • rse – the name of the rse.
  • key – the attribute key.
  • value – the attribute value.
Returns:

True if RSE attribute was created successfully else False.

Raises Duplicate:
 

if RSE attribute already exists.

delete_protocols(rse, scheme, hostname=None, port=None)[source]

Deletes matching protocols from RSE. Protocols using the same identifier can be distinguished by hostname and port.

Parameters:
  • rse – the RSE name.
  • scheme – identifier of the protocol.
  • hostname – hostname of the protocol.
  • port – port of the protocol.
Returns:

True if success.

Raises:
  • RSEProtocolNotSupported – if no matching protocol entry could be found.
  • RSENotFound – if the RSE doesn’t exist.
  • AccessDenied – if not authorized.
delete_rse(rse)[source]

Sends the request to delete a rse.

Parameters:rse – the name of the rse.
Returns:True if location was created successfully else False.
delete_rse_attribute(rse, key)[source]

Sends the request to delete a RSE attribute.

Parameters:
  • rse – the RSE name.
  • key – the attribute key.
Returns:

True if RSE attribute was deleted successfully else False.

get_protocols(rse, protocol_domain='ALL', operation=None, default=False, scheme=None)[source]

Returns protocol information. Parameter comibantions are: (operation OR default) XOR protocol.

Parameters:
  • rse – the RSE name.
  • protocol_domain – The scope of the protocol. Supported are ‘LAN’, ‘WAN’, and ‘ALL’ (as default).
  • operation – The name of the requested operation (read, write, or delete). If None, all operations are queried.
  • default – Indicates if only the default operations should be returned.
  • scheme – The identifier of the requested protocol.
Returns:

A list with details about each matching protocol.

Raises:
  • RSENotFound – if the RSE doesn’t exist.
  • RSEProtocolNotSupported – if no matching protocol entry could be found.
  • RSEOperationNotSupported – if no matching protocol entry for the requested operation could be found.
get_rse(rse)[source]

Returns details about the referred RSE.

Parameters:rse – Name of the referred RSE
Returns:A dict containing all attributes of the referred RSE
Raises RSENotFound:
 if the referred RSE was not found in the database
get_rse_limits(rse)[source]

Get RSE limits.

Parameters:rse – The RSE name.
Returns:True if successful, otherwise false.
get_rse_usage(rse, filters=None)[source]

Get RSE usage information.

Parameters:
  • rse – the RSE name.
  • filters – dictionary of attributes by which the results should be filtered
Returns:

True if successful, otherwise false.

list_rse_attributes(rse)[source]

Sends the request to get RSE attributes.

Parameters:rse – The RSE name.
Returns:True if RSE attribute was created successfully else False.
list_rse_usage_history(rse, filters=None)[source]

List RSE usage history information.

Parameters:
  • rse – The RSE name.
  • filters – dictionary of attributes by which the results should be filtered.
Returns:

list of dictionnaries.

list_rses(rse_expression=None)[source]

Sends the request to list all rucio locations(RSEs).

Rse_expression:RSE Expression to use as filter.
Returns:a list containing the names of all rucio locations.
set_rse_limits(rse, name, value)[source]

Set RSE limit information.

Parameters:
  • rse – The RSE name.
  • name – The name of the limit.
  • value – The feature value. Set to -1 to remove the limit.
Returns:

True if successful, otherwise false.

set_rse_usage(rse, source, used, free)[source]

Set RSE usage information.

Parameters:
  • rse – the RSE name.
  • source – the information source, e.g. srm.
  • used – the used space in bytes.
  • free – the free in bytes.
Returns:

True if successful, otherwise false.

swap_protocols(rse, domain, operation, scheme_a, scheme_b)[source]

Swaps the priorities of the provided operation.

Parameters:
  • rse – the RSE name.
  • domain – the domain in which priorities should be swapped i.e. wan or lan.
  • operation – the operation that should be swapped i.e. read, write, or delete.
  • scheme_a – the scheme of one of the two protocols to be swapped, e.g. srm.
  • scheme_b – the scheme of the other of the two protocols to be swapped, e.g. http.
Returns:

True if success.

Raises:
  • RSEProtocolNotSupported – if no matching protocol entry could be found.
  • RSENotFound – if the RSE doesn’t exist.
  • KeyNotFound – if invalid data was provided for update.
  • AccessDenied – if not authorized.
update_protocols(rse, scheme, data, hostname=None, port=None)[source]

Updates matching protocols from RSE. Protocol using the same identifier can be distinguished by hostname and port.

Parameters:
  • rse – the RSE name.
  • scheme – identifier of the protocol.
  • data – A dict providing the new values of the protocol attibutes. Keys must match column names in database.
  • hostname – hostname of the protocol.
  • port – port of the protocol.
Returns:

True if success.

Raises:
  • RSEProtocolNotSupported – if no matching protocol entry could be found.
  • RSENotFound – if the RSE doesn’t exist.
  • KeyNotFound – if invalid data was provided for update.
  • AccessDenied – if not authorized.
update_rse(rse, parameters)[source]

Update RSE properties like availability or name.

Parameters:
  • rse – the name of the new rse.
  • parameters – A dictionnary with property (name, read, write, delete as keys).

Manager

Storage

class rucio.rse.protocols.protocol.RSEProtocol(protocol_attr, rse_settings)[source]

Bases: object

This class is virtual and acts as a base to inherit new protocols from. It further provides some common functionality which applies for the amjority of the protocols.

close()[source]

Closes the connection to RSE.

connect()[source]

Establishes the actual connection to the referred RSE.

Raises RSEAccessDenied:
 if no connection could be established.
delete(path)[source]

Deletes a file from the connected RSE.

Parameters:

path – path to the to be deleted file

Raises:
  • ServiceUnavailable – if some generic error occured in the library.
  • SourceNotFound – if the source file was not found on the referred storage.
exists(path)[source]

Checks if the requested file is known by the referred RSE.

Parameters:path – Physical file name
Returns:True if the file exists, False if it doesn’t
Raises SourceNotFound:
 if the source file was not found on the referred storage.
get(path, dest)[source]

Provides access to files stored inside connected the RSE.

Parameters:
  • path – Physical file name of requested file
  • dest – Name and path of the files when stored at the client
Raises:
  • DestinationNotAccessible – if the destination storage was not accessible.
  • ServiceUnavailable – if some generic error occured in the library.
  • SourceNotFound – if the source file was not found on the referred storage.
get_space_usage()[source]

Get RSE space usage information.

Returns:a list with dict containing ‘totalsize’ and ‘unusedsize’
Raises ServiceUnavailable:
 if some generic error occured in the library.
lfns2pfns(lfns)[source]

Retruns a fully qualified PFN for the file referred by path.

Parameters:path – The path to the file.
Returns:Fully qualified PFN.
parse_pfns(pfns)[source]

Splits the given PFN into the parts known by the protocol. It is also checked if the provided protocol supportes the given PFNs.

Parameters:pfns – a list of a fully qualified PFNs
Returns:dic with PFN as key and a dict with path and name as value
Raises RSEFileNameNotSupported:
 if the provided PFN doesn’t match with the protocol settings
put(source, target, source_dir)[source]

Allows to store files inside the referred RSE.

Parameters:
  • source – path to the source file on the client file system
  • target – path to the destination file on the storage
  • source_dir – Path where the to be transferred files are stored in the local file system
Raises:
  • DestinationNotAccessible – if the destination storage was not accessible.
  • ServiceUnavailable – if some generic error occured in the library.
  • SourceNotFound – if the source file was not found on the referred storage.
rename(path, new_path)[source]

Allows to rename a file stored inside the connected RSE.

Parameters:
  • path – path to the current file on the storage
  • new_path – path to the new file on the storage
Raises:
  • DestinationNotAccessible – if the destination storage was not accessible.
  • ServiceUnavailable – if some generic error occured in the library.
  • SourceNotFound – if the source file was not found on the referred storage.
stat(path)[source]

Returns the stats of a file.

Parameters:

path – path to file

Raises:
  • ServiceUnavailable – if some generic error occured in the library.
  • SourceNotFound – if the source file was not found on the referred storage.
Returns:

a dict with two keys, filesize and adler32 of the file provided in path.