Stack Overflow
Stack Overflow

Questions tagged [wal]

WAL, or write-ahead log, is a form of journalling used as part of crash-safety and atomic commit support in databases like PostgreSQL and SQLite.

PostgreSQL's write-ahead log is a crucial feature that permits crash-safety and atomic commit to be implemented. It's also heavily used in PostgreSQL's point-in-time recovery, streaming replication, and archive based replication features, as well as hot physical backup.

In SQLite, the default method by which SQLite implements atomic commit and rollback is a rollback journal. Beginning with version 3.7.0, a new "Write-Ahead Log" option (hereafter referred to as "WAL") is available.

229 questions
1
vote
0 answers

Copying values from STDIN to a Postgres table is not reflected in WAL

I'm using copy from STDIN command to insert data into my Postgres table. However, this is not being reflected in the WriteAheadLogs (however, using an INSERT INTO statement reflects on the WriteAheadLogs). Am I missing something? Should the WAL not…
ghost
  • 1,107
  • 3
  • 12
  • 31
1
vote
2 answers

Can't get new postgres config file settings to take effect

I have a somewhat large table in my database and I am inserting new records to it. As the number of records grow, I started having issues and can't insert. My postgresql log files suggest I increase WAL size: [700] LOG: checkpoints are occurring too…
Lou Salome
  • 59
  • 1
  • 8
1
vote
0 answers

What is the /var/lib/postgresql/10/main/ equivilent in PSQL-12?

I'm working on setting up point in time backups for a PSQL server that I have running, and I'm following a tutorial for an earlier version. I'm trying to figure out what the specific directory is for the DB cluster in PSQL-12 so that I can clear out…
Aideux
  • 109
  • 3
  • 9
1
vote
2 answers

PostgreSQL 9.6 understanding wal files

I am trying to understand the behaviour of wal files. The wal related settings of the database are as follows: "min_wal_size" "2GB" "max_wal_size" "20GB" "wal_segment_size" "16MB" "wal_keep_segments" "0" "checkpoint_completion_target" …
SteZe
  • 43
  • 7
1
vote
0 answers

Communication between two containers with Static IP

Guy, I am trying to create an WAL redundancy on my machine with postgres and WAL technique, but I need to set some configurations with static ip into .conf files on postgres, I am using Docker and docker-compose to do that, but I am with a problem…
Oracy Martos
  • 451
  • 1
  • 4
  • 19
1
vote
1 answer

wal-g backup-push tries to use wrong database

When trying to use wal-g with Patroni, I've got the following configuration as the Postgres section of patroni.yml: postgresql: use_pg_rewind: true use_slots: true parameters: wal_level: logical hot_standby: "on" …
NealeU
  • 1,264
  • 11
  • 23
1
vote
1 answer

Transaction Log in Aerospilke

What I have? A lot of different microservices managing by different teams. All microservices persist data in Aerospike database. What I want to achieve? I'm building new microservice that relies on data handled by another services. I want to listen…
Serge Harnyk
  • 1,279
  • 10
  • 19
1
vote
1 answer

wal writer vs bg writer vs checkpointer

Please explain difference between these processes in short words. Official documentation didn't give me any grasp =( All of them for flushing dirty pages from shared buffers to disk (wal or db disk?), but what is actual difference? How is the…
Temirlan Kabylbekov
  • 13
  • 5
1
vote
1 answer

Is it possible to read the header of an archived WAL segment?

I would like to read the PostgreSQL archived WAL segment header to get some information to the content of the file. In compare, under MS SQL I can run a restore headeronly ... command and get information about each backup file, including…
SN83
  • 11
  • 1
1
vote
2 answers

Make wal2json listen to tables without Primary Key

When I create a table with a Primary Key on PostgreSQL 10 create table test (t1 int, t2 varchar, primary key(id)); and stream data from PostgreSQL to Kafka through Debezium using wal2json with these SQL insert into test values (1, 'Test1'); update…
Prince Vegeta
  • 719
  • 6
  • 21
1
vote
1 answer

Conversion of numeric in wal2json

I have a column X, which has datatype numeric, in a table. After streaming changes from PostgreSQL 10 to Kafka using debezium together with the logical decoding plugin, wal2json; I read the json output with a Java Kafka Consumer. I noticed that the…
Prince Vegeta
  • 719
  • 6
  • 21
1
vote
1 answer

Is it possible to get the current 64 bit transactionId (with the epoch) in a postgreSQL logical replication plugin

I am building a Change Capture system that consumes the Postgres WAL through a logical decoding output plugin. In the callbacks it is very easy to access the current transactionId…
fpacifici
  • 523
  • 2
  • 11
1
vote
0 answers

SQLite - Is it possible to increase database in chunks during wal mode?

I am recording data quite rapidly in a SQLite Database, which results in a lot of fragments on the disc (typically 2500 frags/file). When I later on try to search through this database it takes a long time, mostly due to the fact that I have a hard…
Richard N
  • 11
  • 1
1
vote
1 answer

Postgres 11 Standby never catches up

Since upgrading to Postgres 11 I cannot get my production standby server to catch up. In the logs things look fine eventually: 2019-02-06 19:23:53.659 UTC [14021] LOG: consistent recovery state reached at 3C772/8912C508 2019-02-06 19:23:53.660 UTC…
Gordon Seidoh Worley
  • 7,839
  • 6
  • 45
  • 82
1
vote
1 answer

reusing old master as slave in PostgreSQL

I have a Master node (A) and a slave node (B). Assume a checkpoint has occurred at instant T and some transactions have been completed since the checkpoint (The dirty pages are not yet persisted on the disk). Assume the WAL position after the…
krithikaGopalakrishnan
  • 1,217
  • 10
  • 28
1 2 3
…
15 16