We have restarted catalog service and tried deleting the locks from HMS directly, however the locks are coming back again.
Note: The cluster deployment has changed and the DM/env.sh mechanism no longer exists. To add environment variables, edit the kubernetes configmap and restart the Okera catalog pod.
This issue occurred due to RDS setting a write lock caused by MSCK repair and auto-recovery of Partitions being set at the same time. The solution is to disable auto-recovery of partitions on the clusters these jobs are running. The steps to do this are as follows
1) Edit the env.sh
2) Add to the property:
The following property:
To make sure you don't remove any presets, you can simply add
3) Restart the DM and start your cluster.