What Are Flink Slots?

A slot in Flink is a parallelism unit. It defines how many concurrent tasks can be executed in parallel.

A task is the smallest unit of work that can be scheduled by the Flink framework. A slot can be occupied by one or more tasks, depending on the level of parallelism (the number of slots) that is configured for the job.

The number of slots available in a Flink cluster depends on the number of TaskManagers and the number of slots per TaskManager that is configured. For example, if a Flink cluster has 4 TaskManagers and each TaskManager has 8 slots, then the total number of slots available in the cluster is 32.

When a job is submitted to a Flink cluster, the job manager will try to allocate enough slots for the job so that all tasks can be executed in parallel. If there are not enough slots available, some tasks will have to wait until other tasks have finished and freed up a slot.

The slot concept in Flink is similar to the concept of containers in Hadoop YARN or Mesos. In those systems, each task is executed in its own container. In Flink, each task is executed in its own slot. The advantage of this approach is that it allows for a much finer-grained level of parallelism.

For example, it is possible to have 1000 tasks running in parallel on 4 TaskManagers, each with 8 slots. With YARN or Mesos, this would require at least 125 containers (1000 / 8).

Slots can be statically or dynamically allocated to jobs. When static allocation is used, each job is allocated a fixed number of slots when it is submitted. This can lead to under-utilization of resources if the job does not use all of its allocated slots (for example, if it only has 10 tasks and is allocated 20 slots).

When dynamic allocation is used, jobs are only allocated as many slots as they need at any given time. This ensures that resources are always used as efficiently as possible.