Skip to main content

pglogical on PostgreSQL12( logical replication )



pglogical is a logical replication system implemented entirely as a PostgreSQL extension. Fully integrated, it requires no triggers or external programs. Logical Replication extension for PostgreSQL 12, 11, 10, 9.6, 9.5, 9.4 (Postgres), providing much faster replication than Slony, Bucardo or Londiste, as well as cross-version upgrades.
In my example to configure pglogical I used two systems 192.168.1.93 for provider(publish) and 192.168.1.120 for subscriber (subscribe).Both systems are on Centos7.
Step 1. Install PostgreSQL 
I assume that on both systems PostgreSQL12 has already installed. If not installed use below link
[root@pc1 ~]#
yum install https://download.postgresql.org/pub/repos/yum/reporpms/EL-7-x86_64/pgdg-redhat-repo-latestpglogical.noarch.rpm
[root@pc1 ~]#yum install postgresql12-server postgresql12-contrib
[root@pc1 ~]#/usr/pgsql-12/bin/postgresql-12-setup initdb
[root@pc1 ~]#systemctl enable postgresql-12
[root@pc1 ~]#systemctl start postgresql-12 
Step 2.Then install the “2ndQuadrant’s General Public” repository for your PostgreSQL version, by running the
following instructions as root on the destination Linux server:
[root@pc1 ~]#curl https://access.2ndquadrant.com/api/repository/dl/default/release/12/rpm
| bash
[root@pc1 ~]# yum install postgresql12-pglogical
Step 3. Open postgresql.conf file and edit below parameters
[root@pc1 ~]#cd /var/lib/pgsql/12/data
root@pc1 data]# vi postgresql.conf
wal_level = 'logical'
max_worker_processes = 10   # one per database needed on provider node
                                                   # one per node needed on subscriber node
max_replication_slots = 10      # one per node needed on provider node
max_wal_senders = 10             # one per node needed on provider node
shared_preload_libraries = 'pglogical'
Save and exit from postgresql.conf file
[root@pc1 data]# systemctl restart postgresql-12.service
[root@pc1 data]# systemctl status postgresql-12.service
pg_hba.conf has to allow replication
connections from localhost.
Step 4. Next the pglogical extension has to be installed on all nodes(192.168.1.93 and 192.168.1.120):
[root@pc1 ~]#su Postgres [Hint :You can use pgAdmin tool for it]
bash-4.2$psql
Postgres=#
CREATE EXTENSION pglogical;
To check run this
Postgres=# \dx+ pglogicalll
Objects in extension "pglogical" Object description ---------------------------------------------------------------
 function pglogical.alter_node_add_interface(name,name,text)
 function pglogical.alter_node_drop_interface(name,name)
 function pglogical.alter_replication_set(name,boolean,boolean,boolean,boolean)
 function pglogical.alter_subscription_add_replication_set(name,name)
 .
 .
table pglogical.sequence_state
 table pglogical.subscription
 view pglogical.tables
(51 rows)
Step 5. Now create a table testpglogical on both nodes for testing purpose .
CREATE TABLE testpglogical (empno serial primary key, empname text);
Now create the provider node(On 192.168.1.93):
SELECT pglogical.create_node(
node_name := 'provider1',
            dsn := 'host=192.168.1.93 port=5432 dbname=postgres'
);
select pglogical.create_replication_set('replication_set');
select pglogical.replication_set_add_table(
                                           set_name := 'replication_set',
                                           relation := 'testpglogical',
                                           synchronize_data := true
                              );
[ If we want to Add all tables in public schema to the default replication set.
SELECT pglogical.replication_set_add_all_tables('default', ARRAY['public']);]
Optionally you can also create additional replication sets and add tables to them (see Replication sets)It's usually better to create replication sets before subscribing so that all tables are synchronized during initial replication setup in a single initial transaction. However, users of bigger databases may instead wish to create them incrementally for better control.
Step 6. On subscriber Node (On192.168.1.120) execute below scripts:
[root@pc1 ~]# cd /var/lib/pgsql/
vi .pgpass
*:*:*:postgres:welcome   [Hint : welcome is password of user postgres]
Save and exit
[root@pc1 ~]# su Postgres [Hint : You can use pgAdmin tool for it]
bash-4.2$psql
Postgres=#
Create table testpglogical (empno serial primary key, empname text);
Now create the subscriber node(On 192.168.1.120):
select pglogical.create_node(
  node_name := 'subscriber',
  dsn := 'host=192.168.1.120 port=5432 dbname=postgres'
);

select pglogical.create_subscription(
  subscription_name := 'subscription',
  replication_sets := array['replication_set'],
  provider_dsn := 'host=192.168.1.93 port=5432 dbname=postgres'
);

Now your pglogical has configured.
Whatever you will do with table testpglogical on provider node(192.168.1.93) it will reflecte in table testpglogical on subscriber node(192.168.1.120).
e.g.
On provider node(192.168.1.93)
INSERT INTO testpglogical VALUES (1,'SAGAR'),(2,'VISHWAJEET'),(3,'DINESH');
On subscriber node(192.168.1.120)
SELECT *  FROM testpglogical;

Requirements
To use pglogical the provider and subscriber must be running PostgreSQL 9.4 or newer.
The pglogical extension must be installed on both provider and subscriber. You must CREATE EXTENSION pglogical on both.
Tables on the provider and subscriber must have the same names and be in the same schema. Future revisions may add mapping features.
Tables on the provider and subscriber must have the same columns, with the same data types in each column. CHECK constraints, NOT NULL constraints, etc must be the same or weaker (more permissive) on the subscriber than the provider.
Tables must have the same PRIMARY KEYs. It is not recommended to add additional UNIQUE constraints other than the PRIMARY KEY (see below).
To explore more on it use below links:
Use cases supported are:
  • Upgrades between major versions (given the above restrictions)
  • Full database replication
  • Selective replication of sets of tables using replication sets
  • Selective replication of table rows at either publisher or subscriber side (row_filter)
  • Selective replication of table columns at publisher side
  • Data gather/merge from multiple upstream servers
Architectural details:
  • pglogical works on a per-database level, not whole server level like physical streaming replication
  • One Provider may feed multiple Subscribers without incurring additional disk write overhead
  • One Subscriber can merge changes from several origins and detect conflict between changes with automatic and configurable conflict resolution (some, but not all aspects required for multi-master).
  • Cascading replication is implemented in the form of change set forwarding.
To explore more on it go through below links:
https://github.com/2ndQuadrant/pglogical

Comments

  1. Thanks for sharing good article prakash

    ReplyDelete
  2. Thanks Prakash Sir for sharing this. It is really helpful.

    ReplyDelete
  3. Very helpful information. Keep rocking...

    ReplyDelete
  4. I am always getting a good stuff from your side sir and with proper example that how we make copies from logical replication, Your articles are very good sir keep writing and keep updating sir...Thank you for sharing this sir, As we got lots of knowledge from your article sir.

    ReplyDelete
  5. Thank you for sharing such a useful information. I appreciate you the detail you went into the topic. I am grateful for the amount of time and effort you put into this helping us. Your insights and summary are beneficial for all of us.

    ReplyDelete
  6. Simple and essy steps to configue. Good one

    ReplyDelete
  7. Informatic and easy to implement

    ReplyDelete

Post a Comment

Popular posts from this blog

SSL configuration in PostgreSQL for PgAdmin

SSL configuration in PostgreSQL for PgAdmin This document describes how to set up SSL to enable ssl connections from PgAdmin on some client machine to postgresql on a server machine. In my example, I am using Centos 7 for PostgreSql-9.5 server and Windows for PgAdmin.The assumption is that Postgresql (compiled with ssl support) and openssl are already installed on the server (Linux Centos7). PgAdmin is already installed on the client (either Windows or Linux). On the server, three certificates are required in the data directory. root.crt (trusted root certificate) server.crt (server certificate) server.key (private key)  On CentOS7 default Data directory is / var/lib/pgsql/9.5/data/ : Now, connect your system with user root. Step 1 Su – Go to data directory [root@localhost ~]# cd /var/lib/pgsql/9.5/data/ Out put [root@ localhost data]# Step 2 : Generate a private key (you must provide a passphrase). [root@ localhost data]# ope...

Partition and Subpartition in PostgreSQL12

Partitioning refers to splitting what is logically one large table into smaller physical pieces. Partitioning can provide several benefits: • Query performance can be improved dramatically in certain situations, particularly when most of the heavily accessed rows of the table are in a single partition or a small number of partitions.  • When queries or updates access a large percentage of a single partition, performance can be improved by taking advantage of sequential scan of that partition instead of using an index and random access reads scattered across the whole table. • Bulk loads and deletes can be accomplished by adding or removing partitions, if that requirement is planned into the partitioning design. Doing ALTER TABLE DETACH PARTITION or dropping an individual partition using DROP TABLE is far faster than a bulk operation. These commands also entirely avoid the VACUUM overhead caused by a bulk DELETE. • Seldom-used data can be migrated to cheaper and slower...

PostgreSQL Foreign Data Wrapper for Oracle on Windows OS

The foreign data wrapper is responsible for fetching data from the remote data source and returning it to the PostgreSQL executor. If wrapper must support in updating foreign tables. In this post, I'll demonstrate, how you may connect PostgreSQL database to Oracle database using oracle_fdw . Before going ahead kindly ensure that PostgreSQL is installed in your system . Let’s begin: Step-1 Software’s used in demo - Windows, PostgreSQL 9.5, 9.6   instantclient-basic-win-x86-64-11.2.0.1.0 instantclient-jdbc-win-x86-64-11.2.0.1.0   instantclient-odbc-win-x86-64-11.2.0.1.0 instantclient-sqlplus-win-x86-64-11.2.0.1.0 & Oracle_FDW as per version ( http://www.oracle.com/technetwork/topics/linuxx86-64soft-092277.html   , https://pgxn.org/dist/oracle_fdw/ or version wise)   Download all the files mentioned above, in a single folder and unzip entire files to this single folder. Suppose, we create a folder Oracle in c: drive ...