pygromos.simulations.hpc_queuing.submission_systems package

Submodules

pygromos.simulations.hpc_queuing.submission_systems.dummy module

class pygromos.simulations.hpc_queuing.submission_systems.dummy.DUMMY(verbose: bool = False, nomp: int = 1, nmpi: int = 1, job_duration: str = '24:00', submission: bool = True, environment=None)[source]

Bases: pygromos.simulations.hpc_queuing.submission_systems._submission_system._SubmissionSystem

This SubmissionSystem is for testing submission systems. It basically prints everything out.

get_jobs_from_queue(job_text: str, **kwargs) → List[int][source]

search_queue_for_jobname

this jobs searches the job queue for a certain job id.

Parameters

  • job_text (str)

  • regex (bool, optional) – if the string is a Regular Expression

Returns

output contains all ids of fitting jobs to the querry

Return type

List[int]

job_queue_list: pandas.core.frame.DataFrame
search_queue_for_jobname(job_name: str, **kwargs) → List[str][source]

this jobs searches the job queue for a certain job id.

Parameters

  • job_name (str)

  • regex (bool, optional) – if the string is a Regular Expression

Returns

the output of the queue containing the jobname

Return type

List[str]

submit_jobAarray_to_queue(sub_job: pygromos.simulations.hpc_queuing.submission_systems.submission_job.Submission_job) → Optional[int][source]

this function is submitting :Parameters: sub_job (Submission_job) – submission job parameters

Returns

if a job was submitted the jobID is returned else None.

Return type

int, None

submit_to_queue(sub_job: pygromos.simulations.hpc_queuing.submission_systems.submission_job.Submission_job) → Optional[int][source]

This function submits a str command to the submission system.

Parameters

sub_job (Submission_job) – submission job parameters

Returns

if a job was submitted the jobID is returned else None.

Return type

int, None

verbose: bool

pygromos.simulations.hpc_queuing.submission_systems.local module

class pygromos.simulations.hpc_queuing.submission_systems.local.LOCAL(submission: bool = True, nomp: int = 1, nmpi: int = 1, job_duration: str = '24:00', verbose: bool = False, environment=None, zip_trajectories: bool = True)[source]

Bases: pygromos.simulations.hpc_queuing.submission_systems._submission_system._SubmissionSystem

This class handles local submission without a queueing system

job_queue_list: pandas.core.frame.DataFrame
search_queue_for_jobid(job_id: int, **kwargs) → pandas.core.frame.DataFrame[source]

this jobs searches the job queue for a certain job id. DUMMY FUNCTION!

Parameters

job_id (int) – id of the job

Raises

NotImplemented – Needs to be implemented in subclasses

search_queue_for_jobname(job_name: str, **kwargs) → List[str][source]

this jobs searches the job queue for a certain job name. DUMMY FUNCTION!

Parameters

  • job_name (str)

  • regex (bool, optional) – if the string is a Regular Expression

Returns

the output of the queue containing the jobname

Return type

List[str]

submit_jobAarray_to_queue(sub_job: pygromos.simulations.hpc_queuing.submission_systems.submission_job.Submission_job) → int[source]

submitt a local job array

submit_to_queue(sub_job: pygromos.simulations.hpc_queuing.submission_systems.submission_job.Submission_job) → int[source]

submitt a local job

verbose: bool

pygromos.simulations.hpc_queuing.submission_systems.lsf module

class pygromos.simulations.hpc_queuing.submission_systems.lsf.LSF(submission: bool = True, nomp: int = 1, nmpi: int = 1, job_duration: str = '24:00', max_storage: float = 1000, verbose: bool = False, environment=None, block_double_submission: bool = True, bjobs_only_same_host: bool = False, chain_prefix: str = 'done', begin_mail: bool = False, end_mail: bool = False, zip_trajectories: bool = True)[source]

Bases: pygromos.simulations.hpc_queuing.submission_systems._submission_system._SubmissionSystem

This class is a wrapper for the LSF queueing system by IBM, like it is used on Euler.

get_queued_jobs() → pandas.core.frame.DataFrame[source]

This function updates the job-list of the queueing system in the class.

Returns

returns the job_queue as pandas dataFrame.

Return type

pd.DataFrame

kill_jobs(job_name: Optional[str] = None, regex: bool = False, job_ids: Optional[Union[int, List[int]]] = None)[source]

this function can be used to terminate or remove pending jobs from the queue.

Parameters

  • job_name (str) – name of the job to be killed

  • regex (bool) – if true, all jobs matching job_name get killed!

  • job_ids (Union[List[int], int]) – job Ids to be killed

search_queue_for_jobid(job_id: int) → pandas.core.frame.DataFrame[source]

this jobs searches the job queue for a certain job id.

Parameters

job_id (int) – id of the job

Raises

NotImplementedError – Needs to be implemented in subclasses

search_queue_for_jobname(job_name: str, regex: bool = False) → pandas.core.frame.DataFrame[source]

this jobs searches the job queue for a certain job name.

Parameters

  • job_name (str)

  • regex (bool, optional) – if the string is a Regular Expression

Returns

the output of the queue containing the jobname

Return type

List[str]

submit_jobAarray_to_queue(sub_job: pygromos.simulations.hpc_queuing.submission_systems.submission_job.Submission_job) → int[source]
This functioncan be used for submission of a job array. The ammount of jobs is determined by the difference:

end_job-start_job

An array index variable is defined called ${JOBID} inside the command representing job x in the array.

Parameters

sub_job (Submission_job) – the job to be submitted

Returns

return job ID

Return type

int

submit_to_queue(sub_job: pygromos.simulations.hpc_queuing.submission_systems.submission_job.Submission_job) → int[source]

This function submits the given command to the LSF QUEUE

Parameters

  • submission_job (Submission_job) – the job to be submitted

  • ——-

pygromos.simulations.hpc_queuing.submission_systems.submission_job module

class pygromos.simulations.hpc_queuing.submission_systems.submission_job.Submission_job(command: Optional[str] = None, jobName: Optional[str] = None, outLog: Optional[str] = None, errLog: Optional[str] = None, start_job: Optional[int] = None, end_job: Optional[int] = None, jobLim: Optional[int] = None, queue_after_jobID: Optional[int] = None, post_execution_command: Optional[str] = None, submit_from_dir: Optional[str] = None, sumbit_from_file: bool = True, jobGroup: Optional[str] = None, jobID=None)[source]

Bases: object

Description: This class stores parameters for the submission of jobs. It is used by the submission_systems: - submission dummy - submission local - submission lsf

It should handle all the information required for a single job, while the submission_systems handles more permanent settings.

It should provide an easy way to modify jobs, even from high level modules (e.g. the simulation module like TI, EMIN, …).

Author: Marc Lehner

property command: str
property end_job: int
property errLog: str
property jobGroup: str
property jobID: int
property jobLim: int
property jobName: str
property outLog: str
property post_execution_command: str
property queue_after_jobID: int
property start_job: int
property submit_from_dir: str
property sumbit_from_file: bool

Module contents

pygromos.simulations.hpc_queuing.submission_systems.get_submission_system(testing: bool = False)[source]