Nextflow documentation is migrating

Nextflow documentation is being migrated to docs.seqera.io/nextflow. This site will remain available throughout the migration.

nextflow.executor

The nextflow.executor package defines the executor interface and implements several built-in executors.

Class Diagram

        classDiagram
    %%
    %% nextflow.executor
    %%
    ExecutorFactory --> Executor : getExecutor

    class Executor {
        name : String
        monitor : TaskMonitor
    }
    Executor --* TaskMonitor
    Executor --> TaskHandler : submit

    TaskMonitor <|-- TaskPollingMonitor

    class TaskPollingMonitor {
        capacity : int
        submitRateLimit : RateLimiter
        pollIntervalMillis : long
        dumpInterval : Duration
    }

    TaskPollingMonitor <|-- LocalPollingMonitor

    class LocalPollingMonitor {
        maxCpus : int
        maxMemory : long
    }

    Executor <|-- AbstractGridExecutor
    Executor <|-- LocalExecutor
    %% Executor <|-- NopeExecutor
    %% AbstractGridExecutor <|-- CondorExecutor
    %% AbstractGridExecutor <|-- HyperQueueExecutor
    %% AbstractGridExecutor <|-- LsfExecutor
    %% AbstractGridExecutor <|-- MoabExecutor
    %% AbstractGridExecutor <|-- NqsiiExecutor
    %% AbstractGridExecutor <|-- OarExecutor
    %% AbstractGridExecutor <|-- PbsExecutor
    %% AbstractGridExecutor <|-- SgeExecutor
    %% AbstractGridExecutor <|-- SlurmExecutor
    %% PbsExecutor <|-- PbsProExecutor
    %% SgeExecutor <|-- CrgExecutor

    %% TaskHandler <|-- CachedTaskHandler
    TaskHandler <|-- GridTaskHandler
    TaskHandler <|-- LocalTaskHandler
    TaskHandler <|-- NativeTaskHandler
    %% TaskHandler <|-- NopeTaskHandler
    %% TaskHandler <|-- StoredTaskHandler

    class BashWrapperBuilder {
        bean : TaskBean
        copyStrategy : ScriptFileCopyStrategy
    }
    BashWrapperBuilder --* TaskBean
    BashWrapperBuilder --* ScriptFileCopyStrategy

    ScriptFileCopyStrategy <|-- SimpleFileCopyStrategy

    class SimpleFileCopyStrategy {
        stageinMode : String
        stageoutMode : String
        targetDir : Path
        workDir : Path
    }

    

Note

Some classes may be excluded from the above diagram for brevity.

Notes

The Executor class is the base class for all Nextflow executors. The main purpose of an Executor is to submit tasks to an underlying compute environment, such as an HPC scheduler or cloud batch executor. It uses a TaskMonitor to manage the lifecycle of all tasks and a TaskHandler to manage each individual task. Most executors use the same polling monitor, but each executor implements its own task handler to customize it for a particular compute environment. See nextflow.processor for more details about these classes.

The built-in executors include the local executor (LocalExecutor) and the various grid executors (SLURM, PBS, LSF, etc), all of which extend AbstractGridExecutor. The LocalExecutor implements both “script” tasks (processes with a script or shell block) and “native” tasks (processes with an exec block).

The BashWrapperBuilder is used by executors to generate the wrapper script (.command.run) for a task, from a template script called command-run.txt, as well as the task configuration and the execution environment.