gdayf.persistence.persistencehandler.PersistenceHandler Class Reference

Class to manage trasient information between all persistence options and models on an unified way. More...

Inheritance diagram for gdayf.persistence.persistencehandler.PersistenceHandler:

Collaboration diagram for gdayf.persistence.persistencehandler.PersistenceHandler:

Public Member Functions
def	__init__ (self, e_c)
	Class Constructor. More...
def	store_file (self, storage_json, filename)
	Method used to store a file on one persistence system ['localfs', ' hdfs'] using mmap structure to manage multi-persistence features. More...
def	remove_file (self, load_path)
	Method used to remove a file on one persistence system ['localfs',' hdfs']. More...
def	store_json (self, storage_json, ar_json, other=None)
	Method used to store a json on all persistence system ['localfs', ' hdfs', ' mongoDB'] oriented to store full Analysis_results json but useful on whole json. More...
def	recover_experiment_mongoDB (self, client=None)
	Method used to recover an experiment as [ar_metadata] oriented to store full Analysis_results json but useful on whole json. More...
def	mkdir (self, type, path, grants)
	Method used to check and make directory os similar path structures on all persistence system ['localfs', ' hdfs', ' mongoDB'] over agnostic way. More...
def	get_ar_from_engine (self, path)
	Method base to get an ArMetadata Structure from file. More...

Detailed Description

Class to manage trasient information between all persistence options and models on an unified way.

Definition at line 38 of file persistencehandler.py.

Constructor & Destructor Documentation

__init__()

def gdayf.persistence.persistencehandler.PersistenceHandler.__init__	(		self,
			e_c
	)

Class Constructor.

Parameters

e_c	context pointer

Definition at line 41 of file persistencehandler.py.

Member Function Documentation

get_ar_from_engine()

def gdayf.persistence.persistencehandler.PersistenceHandler.get_ar_from_engine	(		self,
			path
	)

Method base to get an ArMetadata Structure from file.

Parameters

self	object pointer
path	FilePath for localfs or hdfs and /user_id/workflow_id/analysis_id/model_id for MongoDB

Returns

operation status (0 success /1 error, ArMetadata/None)

Definition at line 510 of file persistencehandler.py.

mkdir()

def gdayf.persistence.persistencehandler.PersistenceHandler.mkdir	(		self,
			type,
			path,
			grants
	)

Method used to check and make directory os similar path structures on all persistence system ['localfs', ' hdfs', ' mongoDB'] over agnostic way.

Parameters

self	object pointer
type	['localfs', ' hdfs', ' mongoDB']
path	directory or persistence structure to be created
grants	on a 0o#### format (octalpython format)

Returns

operation status (0 success) (1 error)

Definition at line 440 of file persistencehandler.py.

Here is the call graph for this function:

recover_experiment_mongoDB()

def gdayf.persistence.persistencehandler.PersistenceHandler.recover_experiment_mongoDB	(		self,
			client = None
	)

Method used to recover an experiment as [ar_metadata] oriented to store full Analysis_results json but useful on whole json.

Parameters

self	object pointer
client	Cliente MongoClient()

Returns

[ArMetadata]

Definition at line 405 of file persistencehandler.py.

remove_file()

def gdayf.persistence.persistencehandler.PersistenceHandler.remove_file	(		self,
			load_path
	)

Method used to remove a file on one persistence system ['localfs',' hdfs'].

Parameters

self	object pointer
storage_json	(list of storagemetadata objects or OrderedDict() compatible objects)
ar_metadata	model_structure

Returns

global_op state (0 success) (n number of errors)

Definition at line 150 of file persistencehandler.py.

Here is the call graph for this function:

store_file()

def gdayf.persistence.persistencehandler.PersistenceHandler.store_file	(		self,
			storage_json,
			filename
	)

Method used to store a file on one persistence system ['localfs', ' hdfs'] using mmap structure to manage multi-persistence features.

Parameters

self	object pointer
storage_json	(list of storagemetadata objects or OrderedDict() compatible objects)
filename	file full path string

Returns

global_op state (0 success) (n number of errors)

Definition at line 52 of file persistencehandler.py.

Here is the call graph for this function:

store_json()

def gdayf.persistence.persistencehandler.PersistenceHandler.store_json	(		self,
			storage_json,
			ar_json,
			other = None
	)

Method used to store a json on all persistence system ['localfs', ' hdfs', ' mongoDB'] oriented to store full Analysis_results json but useful on whole json.

Parameters

self	object pointer
storage_json	(list of storagemetadata objects or OrderedDict() compatible objects)
ar_json	file ArMetadata Class or OrderedDict() compatible object
other	file OrderedDict() compatible object to be stored

Returns

global_op status (0 success) (n number of errors)

assert isinstance(storage_json, StorageMetadata)

Definition at line 231 of file persistencehandler.py.

Here is the call graph for this function:

---

The documentation for this class was generated from the following file:

• D:/OneDrive/DayF/git/gdayf-core/src/gdayf/persistence/persistencehandler.py