Question
Are there any performance improvements to be gained from using MSCK in Hive over ALTER TABLE RECOVER PARTITIONS natively in ODAS?
Answer
ALTER TABLE <DB.TB> RECOVER PARTITIONS is effectively an alias for the msck repair table statement.
The primary difference in usability is that `alter table recover partitions` works from dbcli or directly ODAS through the REST API. The `msck repair table` command must be run from Hive. Hive then translates the `msck` call to `Alter table add partition` and distributes it to the planner as a call to each partition.
The overhead of this translation and distribution results in slower performance from Hive vs. natively through ODAS.
For more information, refer to the documentation here.
Comments
0 comments
Please sign in to leave a comment.