Loading data from one Hive table to another with partition
In some cases you may need to set hive.exec.dynamic.partition.mode=nonstrict to be able to insert data into partitioned table, for example,
CREATE TABLE hivePartitionedTable
(
c1 int
, c2 int
, c3 string
)
PARTITIONED BY (year int)
ROW FORMAT DELIMITED
FIELDS TERMINATED BY ','
STORED AS SEQUENCEFILE
;
then this INSERT will work:
set hive.exec.dynamic.partition.mode=nonstrict;
INSERT INTO hivePartitionedTable PARTITION (year)
VALUES (1,2,'3', 1999);
Create a copy of your table
CREATE TABLE my_table_backup LIKE my_table;
Enable dynamic partitioning
SET hive.exec.dynamic.partition = true;
SET hive.exec.dynamic.partition.mode = nonstrict;
SET hive.mapred.mode = nonstrict;
Copy the table
INSERT OVERWRITE TABLE my_table_backup PARTITION (ds)
SELECT * FROM my_table
WHERE ds = ds;
The where clause is needed if you use strict mode.
this is because you are missing the partition info in your insert query
insert overwrite table reg_logs_org PARTITION (utc_date)
select id, region_code, sum(count), utc_date
from
reg_logs
group by
utc_date, id, region_code