Gfal

Classes¶

Default ¶

Default(protocol_attr, rse_settings, logger=logging.log)

Implementing access to RSEs using the srm protocol.

Functions¶

lfns2pfns ¶

lfns2pfns(lfns)

Returns a fully qualified PFN for the file referred by path.

PARAMETER	DESCRIPTION
`path`	The path to the file.

RETURNS	DESCRIPTION
	Fully qualified PFN.

parse_pfns ¶

parse_pfns(pfns)

Splits the given PFN into the parts known by the protocol. During parsing the PFN is also checked for validity on the given RSE with the given protocol.

PARAMETER	DESCRIPTION
`pfn`	a fully qualified PFN

RETURNS	DESCRIPTION
	a dict containing all known parts of the PFN for the protocol e.g. scheme, path, filename

RAISES	DESCRIPTION
`RSEFileNameNotSupported`	if the provided PFN doesn't match with the protocol settings

path2pfn ¶

path2pfn(path)

Returns a fully qualified PFN for the file referred by path.

PARAMETER	DESCRIPTION
`path`	The path to the file.

RETURNS	DESCRIPTION
	Fully qualified PFN.

connect ¶

connect()

Establishes the actual connection to the referred RSE. If we decide to use gfal, init should be done here.

get ¶

get(path, dest, transfer_timeout=None)

Provides access to files stored inside connected the RSE.

PARAMETER	DESCRIPTION
`path`	Physical file name of requested file
`dest`	Name and path of the files when stored at the client
`transfer_timeout`	Transfer timeout (in seconds) DEFAULT: `None`

RAISES	DESCRIPTION
`DestinationNotAccessible`	if the destination storage was not accessible.
`ServiceUnavailable`	if some generic error occurred in the library.
`SourceNotFound`	if the source file was not found on the referred storage.

put ¶

put(source, target, source_dir, transfer_timeout=None)

Allows to store files inside the referred RSE.

PARAMETER	DESCRIPTION
`source`	path to the source file on the client file system
`target`	path to the destination file on the storage
`source_dir`	Path where the to be transferred files are stored in the local file system
`transfer_timeout`	Transfer timeout (in seconds) DEFAULT: `None`

RAISES	DESCRIPTION
`DestinationNotAccessible`	if the destination storage was not accessible.
`ServiceUnavailable`	if some generic error occurred in the library.
`SourceNotFound`	if the source file was not found on the referred storage.

delete ¶

delete(path)

Deletes a file from the connected RSE.

PARAMETER	DESCRIPTION
`path`	path to the to be deleted file

RAISES	DESCRIPTION
`ServiceUnavailable`	if some generic error occurred in the library.
`SourceNotFound`	if the source file was not found on the referred storage.

rename ¶

rename(path, new_path)

Allows to rename a file stored inside the connected RSE.

PARAMETER	DESCRIPTION
`path`	path to the current file on the storage
`new_path`	path to the new file on the storage

RAISES	DESCRIPTION
`DestinationNotAccessible`	if the destination storage was not accessible.
`ServiceUnavailable`	if some generic error occurred in the library.
`SourceNotFound`	if the source file was not found on the referred storage.

exists ¶

exists(path)

Checks if the requested file is known by the referred RSE.

PARAMETER	DESCRIPTION
`path`	Physical file name

RETURNS	DESCRIPTION
	True if the file exists, False if it doesn't

RAISES	DESCRIPTION
`SourceNotFound`	if the source file was not found on the referred storage.

close ¶

close()

Closes the connection to RSE.

stat ¶

stat(path)

Returns the stats of a file.

PARAMETER	DESCRIPTION
`path`	path to file

RETURNS	DESCRIPTION
	a dict with two keys, filesize and an element of GLOBALLY_SUPPORTED_CHECKSUMS.

RAISES	DESCRIPTION
`ServiceUnavailable`	if some generic error occurred in the library.

__gfal2_cancel ¶

__gfal2_cancel()

Cancel all gfal operations in progress.

__gfal2_copy ¶

__gfal2_copy(
    src,
    dest,
    src_spacetoken=None,
    dest_spacetoken=None,
    transfer_timeout=None,
)

Uses gfal2 to copy file from src to dest.

PARAMETER	DESCRIPTION
`src`	Physical source file name
`src_spacetoken`	The source file's space token DEFAULT: `None`
`dest`	Physical destination file name
`dest_spacetoken`	The destination file's space token DEFAULT: `None`
`transfer_timeout`	Transfer timeout (in seconds) DEFAULT: `None`

RETURNS	DESCRIPTION
	0 if copied successfully, other than 0 if failed

RAISES	DESCRIPTION
`SourceNotFound`	if source file cannot be found.
`RucioException`	if it failed to copy the file.

__gfal2_rm ¶

__gfal2_rm(paths)

Uses gfal2 to remove the file.

PARAMETER	DESCRIPTION
`path`	Physical file name

RETURNS	DESCRIPTION
	0 if removed successfully, other than 0 if failed

RAISES	DESCRIPTION
`SourceNotFound`	if the source file was not found.
`RucioException`	if it failed to remove the file.

__gfal2_exist ¶

__gfal2_exist(path)

Uses gfal2 to check whether the file exists.

PARAMETER	DESCRIPTION
`path`	Physical file name

RETURNS	DESCRIPTION
	0 if it exists, -1 if it doesn't

RAISES	DESCRIPTION
`RucioException`	if the error is not source not found.

__gfal2_rename ¶

__gfal2_rename(path, new_path)

Uses gfal2 to rename a file.

PARAMETER	DESCRIPTION
`path`	path to the current file on the storage
`new_path`	path to the new file on the storage

RETURNS	DESCRIPTION
	0 if it exists, -1 if it doesn't

RAISES	DESCRIPTION
`RucioException`	if failed.

get_space_usage ¶

get_space_usage()

Get RSE space usage information.

RETURNS	DESCRIPTION
	a list with dict containing 'totalsize' and 'unusedsize'

RAISES	DESCRIPTION
`ServiceUnavailable`	if some generic error occurred in the library.

__gfal2_get_space_usage ¶

__gfal2_get_space_usage(path, space_token)

Uses gfal2 to get space usage info with space token.

PARAMETER	DESCRIPTION
`path`	the endpoint path
`space_token`	a string space token. E.g. "ATLASDATADISK"

RETURNS	DESCRIPTION
	a list with dict containing 'totalsize' and 'unusedsize'

RAISES	DESCRIPTION
`ServiceUnavailable`	if failed.

NoRename ¶

NoRename(protocol_attr, rse_settings, logger=logging.log)

Implementing access to RSEs using the srm protocol without renaming files on upload/download. Necessary for some storage endpoints.

Initializes the object with information about the referred RSE.

PARAMETER	DESCRIPTION
`protocol_attr`	Properties of the requested protocol.
`rse_settting`	The RSE settings.
`logger`	Optional decorated logger that can be passed from the calling daemons or servers. DEFAULT: `log`

Functions¶

rename ¶

rename(pfn, new_pfn)

Allows to rename a file stored inside the connected RSE.

PARAMETER	DESCRIPTION
`pfn`	Current physical file name

CLI ¶

CLI(protocol_attr, rse_settings, logger=logging.log)

Implementing access to RSEs using the srm protocol through CLI with 'gfal' commands.

Initializes the object with information about the referred RSE.

PARAMETER	DESCRIPTION
`props`	Properties derived from the RSE Repository

Functions¶

get ¶

get(path, dest, transfer_timeout=None)

Provides access to files stored inside connected the RSE.

PARAMETER	DESCRIPTION
`path`	Physical file name of requested file
`dest`	Name and path of the files when stored at the client
`transfer_timeout`	Transfer timeout (in seconds) DEFAULT: `None`

RAISES	DESCRIPTION
`RucioException`	Passthrough of gfal-copy error message.

put ¶

put(source, target, source_dir, transfer_timeout=None)

Allows to store files inside the referred RSE.

PARAMETER	DESCRIPTION
`source`	path to the source file on the client file system
`target`	path to the destination file on the storage
`source_dir`	Path where the to be transferred files are stored in the local file system
`transfer_timeout`	Transfer timeout (in seconds) DEFAULT: `None`

RAISES	DESCRIPTION
`RucioException`	Passthrough of gfal-copy error message.