Skip to main content

Seqera

Added in version 26.04
warning

Preview feature: may change in a future release.

The seqera executor runs pipeline tasks using Seqera Intelligent Compute. It submits each task to the Seqera scheduler service.

To enable this executor, set process.executor = 'seqera' in the nextflow.config file. Every pipeline process must specify a Docker image using the container directive. The pipeline work directory must be located in an S3 bucket.

Use the following process directives to control resource requests and other job characteristics:

The following hints are supported:

  • machineRequirement.capacityMode
  • machineRequirement.diskAllocation
  • machineRequirement.diskEncrypted
  • machineRequirement.diskIops
  • machineRequirement.diskMountPath
  • machineRequirement.diskSize
  • machineRequirement.diskThroughputMiBps
  • machineRequirement.diskType
  • machineRequirement.machineTypes
  • machineRequirement.maxSpotAttempts
  • machineRequirement.onDemandCostRatio
  • machineRequirement.provisioning
  • predictionModel

The machineRequirement.* hints override the corresponding field of the seqera.executor.machineRequirement config scope on a per-process basis. The predictionModel hint overrides the run-level seqera.executor.predictionModel for a single process. Set it to qr/v1 or qr/v2 to select a model, or to none to disable resource prediction for that process even when the run enables a model. When omitted, the process inherits the run-level prediction model. Use keys as-is, or with the seqera/ prefix to restrict them to this executor.

For example, to override the provisioning mode for a single process:

process hello {
    hints 'seqera/machineRequirement.provisioning': 'spotFirst'

    script:
    """
    your_command --here
    """
}

To disable resource prediction for a single process when the run sets a model:

process hello {
    hints 'seqera/predictionModel': 'none'

    script:
    """
    your_command --here
    """
}

See the seqera scope for the full config reference.

On-demand fallback cost limit​

Added in version 26.10

With provisioning = 'spotFirst', the scheduler falls back to on-demand instances when spot capacity runs out, as long as on-demand stays within a cost limit. The limit is the maximum ratio of the highest on-demand hourly price to the highest spot price, across the instance types selected for a task. If the ratio exceeds the limit, the scheduler retries the task on spot instead. If spot capacity stays unavailable, the task fails with an error message that starts with on-demand fallback refused.

The scheduler applies a default limit for each cloud provider. To set a different limit for all tasks in the run, use onDemandCostRatio:

seqera {
    executor {
        machineRequirement {
            provisioning = 'spotFirst'
            onDemandCostRatio = 2.5    // fall back only while on-demand costs at most 2.5x spot
        }
    }
}

Leaving onDemandCostRatio unset is not the same as setting it to 0:

onDemandCostRatioBehavior
Not setThe scheduler's default limit for the cloud provider applies.
0No limit. Tasks always fall back to on-demand, regardless of price.
Greater than 0The scheduler uses this value instead of its default limit.

Setting 0 does not disable the fallback. To disable it, set provisioning = 'spot'.

To override the limit for a single process, use a hint, as with provisioning. For example, to let a long-running process always fall back to on-demand:

process align {
    hints 'seqera/machineRequirement.onDemandCostRatio': 0

    script:
    """
    your_command --here
    """
}

Tasks with different limits run on separate clusters. Tasks that set a limit, including 0, do not use the compute environment's warm pool.

Disk support​

When a process specifies the disk directive, the Seqera executor provisions storage for the task container. It supports two disk allocation strategies:

  • task (default): Nextflow creates a dedicated EBS volume for each task at launch time. This provides isolated storage with configurable volume type, IOPS, throughput, and encryption.

  • node: Uses the instance storage attached at the cluster level. Tasks running on the same node share this storage, which does not support EBS-specific options.

Task allocation (EBS volumes)​

By default, Nextflow uses a gp3 volume with 325 MiB/s throughput, the recommended settings for Fusion. You can customize the EBS volume configuration:

seqera {
    executor {
        machineRequirement {
            diskAllocation = 'task'    // Per-task EBS volume (default)
            diskType = 'ebs/io1'       // Use provisioned IOPS SSD
            diskIops = 10000           // Required for io1/io2
            diskThroughputMiBps = 500  // Throughput for gp3 volumes
            diskEncrypted = true       // Enable KMS encryption
            diskMountPath = '/data'    // Container mount path (default: /tmp)
        }
    }
}

Supported volume types: ebs/gp3 (default), ebs/gp2, ebs/io1, ebs/io2, ebs/st1, ebs/sc1.

Node allocation (instance storage)​

To use instance storage instead of per-task EBS volumes:

seqera {
    executor {
        machineRequirement {
            diskAllocation = 'node'    // Use instance storage
        }
    }
}
note

The EBS-specific options (diskType, diskIops, diskThroughputMiBps, diskEncrypted) do not apply to node allocation. Specifying them causes an error.

On this Page