Jobs

In the previous chapter, we successfully deployed and updated our application to a Kubernetes cluster. As we are progressing, we have a requirement to send scheduled reminder mail to our customers. In modern applications, this is usually achieved with background tasks, as they don’t require direct user interaction. This can be accomplished in Kubernetes using jobs and cron jobs. Jobs and cron jobs create one or more pods and ensure that they are successfully terminated.

Jobs track the successful completion of pods by tracking their exit and making sure that the defined number of pods are executed successfully. If any pod fails during the execution, the job makes sure that a new pod is created to complete the task and monitors it through to successful completion. This is very useful for background scenarios where there is no direct monitoring, and you can rely on the system to complete the task.

Job specification

Jobs also follow the same declarative style specification as other Kubernetes components. The following sample specification file, called Jobspec.yaml, creates a job for our remindme service. This job will print simulated email messages in the console and delete the reminders that are processed by the job using the delete API we created in Chapter 5.

Code Listing 68: Remindme job

The specification asks Kubernetes to create a job named remindmejob using the given image. The Job specification requires apiversion, kind, and metadata.

The spec.template is exactly same as the pod template we used in Chapter 3. It defines the pod details and supports a backofflimit number, which is the same as a number of retries in case of failure. Once this number is reached, the job is moved to a failed state. The default value for this is 6. The job can also be terminated by setting the .spec.activeDeadlineSeconds field, which ensures that the job completes within the specified time (or it fails).

You can apply the create job specification to your cluster using the familiar kubectl command.

Code Listing 69: Applying job to cluster

Figure 53: Output create job

After a successful execution, you can execute the following command to check the status of the job provisioned in your cluster.

Code Listing 70: Get job details

Figure 54: Output get pods

Job types

There are mainly three types of jobs in Kubernetes: nonparallel jobs, parallel jobs with a fixed completion count, and parallel jobs with a work queue. Let’s discuss them in a little more detail.

Nonparallel jobs

These are single-execution jobs, and only one pod is created. If there is an error during the execution, then a new pod is created to complete the task. Otherwise, the job is considered complete once the pod terminates successfully.

Parallel jobs with a fixed completion count

These jobs require a .spec.completions value in the spec template. Multiple pods get created, not necessarily simultaneously, but equal to the value of .spec.completions. The job is considered complete when the same number of pods terminate successfully.

The default value of .spec.completions is 1.

Parallel jobs with a work queue

These jobs require a .spec.parallelism value in the spec template. Multiple pods get created simultaneously, equal to the value of .spec.parallelism, and the job is considered complete when the same amount of pods terminate successfully. The default value of .spec.parallelism is 1.

Note: Once a single pod has exited, all other pods in the job must also be in the exiting phase.

Jobs are great for one-time background processing, but for recurring tasks, we have the concept of cron jobs in Kubernetes.

Cron jobs

We now understand how we can process a task in the background in Kubernetes. However, in our case, we want to schedule a job to send emails periodically. Enter cron jobs. Cron jobs are like background jobs that are used to schedule low-priority tasks and background tasks, like sending emails or purging logs.

We will create a cron job for sending email periodically to users at midnight. A cron job is used to create jobs on a scheduled basis. It uses the popular cron format to specify schedules.

A cron expression is specified as a string of five numbers in the following order:

• Minute (0–59)
• Hour (0–23)
• Day of month (1–31)
• Month (1–12)
• Day of week (0–6)

We can also use a * or ? character to denote any value. Some examples of cron expressions are:

• 5 0 * 8 *         -          At 00:05 in August.
• 15 14 1 * *     -          At 14:15 on day-of-month 1.
• 0 0 * * 1-5      -          At midnight on every day of the week, Monday–Friday.

Let’s create our config file cronjobspec.yaml to define the cron job that will run every minute and send the reminder email. Notice the kind property.

Code Listing 71: CronJob

You can apply the cronjobspec.yaml to the cluster using the following command.

Code Listing 72: Apply cron job to cluster

Figure 55: Output cron job created

We can also use the imperative style to achieve the same.

Code Listing 73: Declare and apply cron job to cluster

After it is successfully scheduled, we can check the status of the job using the following command.

Code Listing 74: Check job status

Figure 56: Output job status

Similarly, if we want to delete the job, we can use the following command:

Code Listing 75: Delete job

Figure 57: Output delete cron job

The previous command will delete the reminderwebjob from our cluster.

Note: The startingDeadlineSeconds field is optional.  It stands for the deadline, in seconds, for starting the job if it misses its scheduled time for any reason. After the deadline, the cron job does not start the job. Jobs that do not meet their deadline in this way count as failed jobs. If this field is not specified, the jobs have no deadline. The cronjob controller counts how many missed schedules happen for a cron job (source).

The cron job is a powerful feature that is heavily utilized for background processing and ensures your application can handle the increased load.

Summary

In this chapter, we learned about creating and deploying background jobs to the Kubernetes cluster. We also learned about scheduling the jobs as cron jobs for the cluster.

In the next chapter, we will discuss how you can gather infrastructure and application logs when you are running a cluster at scale. We will also study some of the available solutions for application and cluster monitoring.