Stack Overflow
Stack Overflow

Questions tagged [wal]

WAL, or write-ahead log, is a form of journalling used as part of crash-safety and atomic commit support in databases like PostgreSQL and SQLite.

PostgreSQL's write-ahead log is a crucial feature that permits crash-safety and atomic commit to be implemented. It's also heavily used in PostgreSQL's point-in-time recovery, streaming replication, and archive based replication features, as well as hot physical backup.

In SQLite, the default method by which SQLite implements atomic commit and rollback is a rollback journal. Beginning with version 3.7.0, a new "Write-Ahead Log" option (hereafter referred to as "WAL") is available.

229 questions
2
votes
5 answers

Why does PostgreSQL need both WAL buffer and WAL segment file?

I'm trying to understand more about the apacheAGE extension therefore I'm reading the inner workings of PostgreSQL. From what I understand every operation that alters the table is written at the WAL buffer, but after it is commited/aborted it is…
Panagiotis Foliadis
  • 254
  • 1
  • 1
  • 8
2
votes
2 answers

PostgreSQL restoration throwing error : replication slot does not exist

Environment: Postgresql 13.x (dockerized) I was trying to test the DR setup for PostgreSQL nodes. pg_basebackup and wal_files archive was taken from the standby mode. Done restoration on a new node by copying pg_basebackup and configured…
Anto
  • 3,128
  • 1
  • 20
  • 20
2
votes
0 answers

GCP - size of CloudSQL Postgres Database logs are increasing rapidly and are not being purged

I'm currently working with Cloud SQL to store data for my database, which is around 14GB in size. However, over monitoring for the past few weeks, it seems like the storage capacity of the database has been continually increasing. When looking at…
Jainil Sutaria
  • 21
  • 2
2
votes
1 answer

What is the need for undo log if redo(WAL) exists?

If there is WAL (redo) log available, then it would allow us to persist to disk the records and commit the transaction without needing to update the data structures first. Since you can always re-apply from the redo log and update the data…
user855
  • 19,048
  • 38
  • 98
  • 162
2
votes
1 answer

Do stand by servers also have check points when they flush the data from WAL logs?

I was curious to know how does end to end replication takes place, The primary replicates WAL to stand by servers, and then the data is committed to the disk, as part of checkpoints also in standby?
Puneet
  • 113
  • 5
2
votes
1 answer

Why data is lost in SQLite database with WAL mode on when connection is not closed properly?

Problem: data is not available to SELECT from new connection to SQLite DB with WAL mode (Write Ahead Log) on if previous connection hasn't been closed properly. Main question is: why data is lost and is there some way to get lost transactions? I'm…
chromesearcher
  • 61
  • 1
  • 10
2
votes
1 answer

Setting up WAL archiving by passing archive_command on the postgres command line in docker compose

I'm trying to set up WAL archiving in Postgres 13.1 using the official docker images. I'm setting the archive_command setting on the command line in a docker compose file, but the command says 'not found' in the logs. When I run the command directly…
Paul Johnson
  • 1,329
  • 1
  • 12
  • 25
2
votes
1 answer

How to restore data from MapDB wal (write ahead log) files?

My objective is to enable MapDB crash protection and if a crash happens, how to restore data. I have enabled transaction for MapDB DB db; HTreeMap< String, String > mapDb; db = DBMaker.fileDB( "walTest/file1.db" ) .transactionEnable() …
Tilak De
  • 141
  • 2
2
votes
2 answers

What does pg_replication_slots.active f for the WAL?

Postgres documentation states the following regarding pg_replication_slots.active: True if this slot is currently actively being used. What does this actually imply for a false value? For example, will the WAL still stick around if the value is…
Awesome-o
  • 2,002
  • 1
  • 26
  • 38
2
votes
1 answer

PostgreSQL 13 - WAL - last file update date doesn't change

I'm using a recent version of PostgreSQL on Windows and trying to understand the internal mechanisms arround WAL files. The scenario I want to test is the following. Look at the active wal file on the file system SELECT…
Laurent GRENIER
  • 612
  • 1
  • 6
  • 13
2
votes
1 answer

What could be preventing SQLite WAL files from being cleared?

I've had three cases of WAL files growing to massive sizes of ~3 GB on two different machines with the same hardware and software setup. The database file itself is ~7 GB. Under normal runtime, WAL is either non-existent or a few kilobytes. I know…
relatively_random
  • 4,505
  • 1
  • 26
  • 48
2
votes
0 answers

How do I provide resiliency in AWS-DMS?

I'm using AWS-DMS for replicating from PostgreSQL (as source endpoint) to S3 (as target endpoint). I want to provide resiliency for the process. Particularly, if the DMS goes down due to some reason, how do I ask it to start from the last point…
ghost
  • 1,107
  • 3
  • 12
  • 31
2
votes
2 answers

prometheus eating huge memory

Prometheus deployed on kubernetes using prometheus operator is eating too much memory and it is at present at ~12G. I see /prometheus/wal directory is at ~12G. I have removed all *.tmp files but that couldn't help. Unable to figure out the solution…
Yogesh Jilhawar
  • 5,605
  • 8
  • 44
  • 59
2
votes
2 answers

Can't find wal backup files

I have two servers, a master and a replica that work together in asynchronous replication mode; slave server seems to be working fine, since any change in the master is mirrored in the slave right away. Moreover, there is also an archive process…
sergi_bcn
  • 21
  • 1
2
votes
1 answer

wal-e/wal-g any benefit for simple backup and restore via S3

I'm using AWS RDS and have a need to replicate "database_a" in an RDS instance to "database_a" in a different RDS instance. The replication only needs to be once every 24 hours. I'm currently solving this with pg_dump and pg_restore but am…
rix
  • 10,104
  • 14
  • 65
  • 92
1 2
3
…
15 16