71

I have the following Kubernetes Job configuration:

---
apiVersion: batch/v1
kind: Job
metadata:
  name: dbload
  creationTimestamp: 
spec:
  template:
    metadata:
      name: dbload
    spec:
      containers:
      - name: dbload
        image: sdvl3prox001:7001/pbench/tdload
        command: ["/opt/pbench/loadTpcdsData.sh",  "qas0063", "dbc", "dbc", "1"]
      restartPolicy: Never
      imagePullSecrets: 
        - name: pbenchregkey
status: {}

When I do kubectl create -f dbload-deployment.yml --record the job and a pod are created, Docker container runs to completion and I get this status:

$ kubectl get job dbload
NAME      DESIRED   SUCCESSFUL   AGE
dbload    1         1            1h
$ kubectl get pods -a
NAME           READY     STATUS      RESTARTS   AGE
dbload-0mk0d   0/1       Completed   0          1h

This job is one time deal and I need to be able to rerun it. If I attempt to rerun it with kubectl create command I get this error

$ kubectl create -f dbload-deployment.yml --record
Error from server: error when creating "dbload-deployment.yml": jobs.batch "dbload" already exists

Of course I can do kubectl delete job dbload and then run kubectl create but I'm wondering if I can somehow re-awaken the job that already exists?

43

No. There is definitely no way to rerun a kubernetes job. You need to delete it first.

2
  • 5
    Those like me who need more details - this is two a step process. First delete your job with kubectl delete job <job_name> and then kubectl apply -f <job_yml> – Niks Jun 12 '20 at 14:58
  • It can be done in one step with kubectl replace --force - which will delete the job if it exists, and then (re-)create it unconditionally. See below. – Caesar Mar 31 at 0:58
56

You can simulate a rerun by replacing the job with itself:

  • kubectl get job "your-job" -o json | kubectl replace --force -f -

If you get errors due to auto-generated labels or selectors, you can delete or edit them with jq:

  • kubectl get job "your-job" -o json | jq 'del(.spec.selector)' | jq 'del(.spec.template.metadata.labels)' | kubectl replace --force -f -
3
  • 4
    Would strongly recommend saving a copy of the job json to a file first. kubectl replace deletes the job before running into the errors recreating it. – Jeremy Huiskamp Apr 4 '19 at 12:40
  • 1
    Save the json first and then recreate!! – deepdive Nov 14 '19 at 23:48
  • No need to save the json first if he still has the original job definition file? – Caesar Mar 31 at 0:56
36

You can also avoid the error you mentioned by specifying

  generateName: dbload

instead of simply name

In that case, each job you submit with this yaml file will have a unique name that will look something like dbloada1b2c. Then you can decide whether you need to delete the old jobs, but you won't have to do it.

Here is a working yaml example:

apiVersion: batch/v1
kind: Job
metadata:
  generateName: netutils-
spec:
  parallelism: 1
  template:
    spec:
      containers:
      - image: amouat/network-utils 
        name: netutil
      restartPolicy: Never

This is the output from kubectl get job after two kubectl create -f example.yaml commands:

NAME             COMPLETIONS   DURATION   AGE
netutils-5bs2s   0/1           14s        14s
netutils-dsvfk   0/1           10s        10s

9
  • I believe generateName only applies to kind=pod and NOT job. – user518066 Aug 30 '17 at 14:28
  • 4
    No, it's a standard part of ObjectMeta and applies to both pod and job: k8s reference. I've been using it all the time, it's core to what I'm doing. – vp124 Sep 1 '17 at 14:52
  • 5
    Thank you very much for this dodge. Just for documentation this does only work with kubectl create – Ohmen Nov 29 '17 at 14:27
  • disagree. Tried to do this right now and it causes error "resource name may not be empty". – rudolfdobias Feb 8 '20 at 0:48
  • 2
    Are you creating your job with kubectl create or kubectl apply? The latter will not work, as per this reference: github.com/kubernetes/kubernetes/issues/44501. So you have to do kubectl create. I have also just tried on Azure, it works there as well. kubectl version shows v1.15.10 on AKS and v1.14.10-gke.27 ok GKE, but like I said, I've been doing this for quite a while, so it worked on earlier versions, too. – vp124 May 4 '20 at 16:06
1

As an improvement on @F. Santiago's idea, you can simply use the value stored at the annotation "kubectl.kubernetes.io/last-applied-configuration" that holds the initial applied configuration without any auto-generated field:

kubectl get <job name> -o json | \
jq -r '.metadata.annotations."kubectl.kubernetes.io/last-applied-configuration"' | \
kubectl replace --save-config --force -f -

Note: for kubectl replace, remember to pass --save-config so it updates the annotation field with the last config applied.

1

There is no way to run a job that completed but you can simulat a rerun by doing the following

  1. Get yaml file of the existing job:

    kubectl get job <job_name> -o yaml > <job_name>.yaml
    
  2. Delete the existing job:

    kubectl delete job <job_name>
    
  3. Run the job again:

    kubectl apply -f <job_name>.yaml
    
1

Based on @Marcelo's idea I made it work with the following, without any processing of the template:

kubectl get job <job-name> -o custom-columns=:metadata.annotations.kubectl\.kubernetes\.io/last-applied-configuration > job.json
kubectl delete -f job.json
kubectl apply -f job.json

Please note the escaped dots (\.) in the annotation name: kubectl\.kubernetes\.io/last-applied-configuration. Without it, it returns <none>.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.