killing-dead-pipelines

This page lists ways to kill the unresponsive pipelines.

Killing the cluster

  • This is a most direct way of killing unresponsive pipelines by bringing the whole cluster down. This can be accomplished by executing the following command on the broker image.
  • vp-terminate-cluster –cluster=name_of_the_cluster
  • NOTE:
    If you have multiple pipelines running on the cluster, bringing down the cluster using vp-terminate-cluster will make you lose access to all other pipelines. To know how to kill individual pipelines, read on.
  • After bringing the cluster down, you still need to kill the wrapper pipeline on the broker image. To do this, execute the following command.
  • qdel -u “*”
  • NOTE:
    If you have multiple wrappers and/or pipelines running on the broker image, executing qdel -u “*” will delete all the wrappers and pipelines. To kill an individual wrapper and/or worker pipeline, read on.

Killing individual pipeline

  • To kill an individual pipeline in the cloud, you need to first login to the master in the cloud using SSH. Then execute qstat command to see the status of all jobs.
  • qstat -u “*”
  • Now go through the state of all jobs and note down the job ids whose state is hung or unresponsive. Now delete those jobs using the following command.
  • qdel job_id_1 job_id_2 …
  • To kill an individual wrapper pipeline on the broker image, execute qstat -u “*” and locate the wrapper pipeline’s job id. Now delete that job using qdel command as explained above.