Postgresql : Merge 2 similar databases
Not my idea, but one I have read in the past.
The source: Move table to new database
pg_dump -d db1 -t table1 |psql db2
then psql and do
insert into table2 (select * from table1);
I used the migrate_pkey_sequence()
function defined here to migrate the primary keys of multiple tables (with the same schema but different values from different databases) to distinct ranges so that I could merge the tables.
This is the approximate sequence I followed to merge two databases, d1
and d2
, each of which has a table example
:
Dump
d1
andd2
to the respective filesd1.sql.gz
andd2.sql.gz
. This is the command I used to dump:$ pg_dump --no-owner | gzip > d1.sql.gz
Create a new, empty database on a local PostgreSQL server.
- Load
d1.sql.gz
. Migrate
d1.example
using:# SELECT migrate_pkey_sequence('example', 'id', 'example_id_seq', 1); INFO: 00000: UPDATE example SET id = nextval('example_id_seq') + 0 INFO: 00000: ALTER SEQUENCE example_id_seq RESTART WITH 1 INFO: 00000: UPDATE example SET id = DEFAULT migrate_pkey_sequence ----------------------- 4 (1 row)
Record the value printed (4 in this example). That will be the start of the next sequence.
- Dump to the file
d1-new.sql.gz
in the same way as above. Repeat steps 2 through 4 with
d2.sql.gz
but use the value from step 5 as the argument tomigrate_pkey_sequence()
:# SELECT migrate_pkey_sequence('example', 'id', 'example_id_seq', 4);
Dump to the file
d2-new-without-schema.sql.gz
without saving the schema and disable triggers. This is the command I used:$ pg_dump --no-owner --data-only --disable-triggers | \ gzip > d2-new-without-schema.sql.gz
Create a new, empty database.
Load
d1-new.sql.gz
andd2-new-without-schema.sql.gz
in order. If everything worked as expected, the primary keys should be disjoint and not conflict.Dump the database if you want to load it into a remote server.
First, load the tables into two separate schemas in the same database.
CREATE SCHEMA db1;
CREATE TABLE db1.table1 (id INT PRIMARY KEY, value VARCHAR(10));
-- load the contents of Table1 @ Database1 into db1.table1
CREATE SCHEMA db2;
CREATE TABLE db2.table1 (id INT PRIMARY KEY, value VARCHAR(10));
-- load the contents of Table1 @ Database2 into db2.table1
Then you can merge the two, by prioritizing db2 over db1.
SELECT
DISTINCT ON (id)
id,
value
FROM (
SELECT
*,
1 AS db
FROM
db1.table1
UNION
SELECT
*,
2 AS db
FROM
db2.table1) AS mix
ORDER BY
id,
db DESC;