0

How to make ddrescue rescue unreadable files along with rsync run when you backup files, when they do exist only at the source?

I think that major programs that work on backup disks must "merge" with ddrescue for platter disks.

I have a lot of disks that are faulty. I would like to rescue these files because they may not exist elsewhere.

I would like to merge rsync with ddrescue. If I have a disk with million files, it is painful to check all input/output errors of files and rescue them manually.

How I can make rsync, when input/output error appears, to run ddrescue instead?

Greg Askew
  • 35,880
  • 5
  • 54
  • 82
Elias Estatistics
  • 121
  • 4
  • 1
    get in touch with a data rescue service. don't try on your own if the data are business related for your business environment. and if there's no backup for the file in question.... you know the words ;) – djdomi Aug 11 '23 at 19:08

1 Answers1

0

If source media is unstable, address that problem (dump and replace it, or restore from backups, whatever) first, before doing any other operations, including regular backup, with rsync or anything else.

ddrescue is not designed to be a part of standard backup procedure, nor it is wise to rely on it for any regular procedures. It's for block device recovery only.

Use dd if you want to dump an image of the block device in a backup procedure, and if it throws any errors, you know, that's a time to use those backups.

Speaking of backups, I disagree that rsync alone can be considered a backup tool. Where's history of previous backup snapshots (so you can store multiple past versions of data e.g. yesterday, two days ago, week ago and month ago, all available)? Where's cleanup for them? Where's problem reporting? It can be a part of a backup solution which does all this and uses rsync to transfer files, but it is not to be used alone. Like RAID, rsync is not a backup — but a mere synchronization tool, this time at a file level.

Nikita Kipriyanov
  • 10,947
  • 2
  • 24
  • 45
  • For example, Tarring files does have rescue procedure for files (but not so good as in ddrescue) which refers "file was shrunk by "x" bytes", and this procedure is automatic. I found this to be a very good idea! – Elias Estatistics Aug 12 '23 at 14:05
  • **This is not the "rescue procedure" which "tar does have"**. When tar started it saw the file X had certain size, it proceeded for some time and stored some data, but by the time to archive the file X it has different size, so in now files in the archive may belong to different generations and **this means the newly generated archive is inconsistent and, probably, will not restore data into any meaningful state should recovery operation be started using this archive**. This is also not stored into the archive, it is only shown as a warning to the user *during archiving*. – Nikita Kipriyanov Aug 13 '23 at 11:42
  • sadly, I dont have these archives anymore. So i cant answer you – Elias Estatistics Aug 13 '23 at 11:50
  • 1
    You completely lack of understanding how backup tools work, why it requires such a complexity, which difficulties you can encounter and so on. You don't understand the principle of operation of ddrescue, tar and other low level tools, to the point to fail to recognize that ddrescue is not a backup tool (and it never said it was). Most likely, you shouldn't design backup procedures on you own, because you **will** lose data if you do. Learn your system to know its backup techniques, implement one of them which suits you better and try to understand why it is like that. – Nikita Kipriyanov Aug 13 '23 at 11:51
  • Also, this discussion does not have anything on the topic of using ddrescue on a regular basis (which you never do unless you are the data recovery service). I explained what to do in the answer, wasn't it clear? – Nikita Kipriyanov Aug 13 '23 at 11:54
  • I asked how to incorporate ddrescue with rsync. This question has not been answered yet. I dont want to use dd, making image i want to back up files using rsync as many do, and recover as many data as i can. Rsync is used frequently to make backups, yes? why to use dd? it can back up files in the same manner as rsync? how? – Elias Estatistics Aug 13 '23 at 12:45
  • On ServerFault we're talking about reasonable practices for managing live business information systems, and what you were asking about is not reasonable, **in my answer I explained, why**. Yes, the answer is, basically: "you are thinking it wrong, screwed way; it doesn't work like you imagine it and you shouldn't do it that way", and it's normal to receive such answers when you come up with ridiculous ideas. – Nikita Kipriyanov Aug 13 '23 at 17:22
  • And, to address that: ddrescue is called **dd**rescue for a reason. It's a **block device recovery tool**, it can be thought as a special kind of dd (which knows nothing about files, except how to read and write them sequentially), but with special read techniques to extract as much data as possible. So, if you want block-level dumper, use dd in a backup procedure (probably that is wrong idea too, but this is not the topic of the question); but, dd doesn't suit you, ddrescue is **already** not suitable, even ignoring the fact it shouldn't be used in any regular automatic backup procedure. – Nikita Kipriyanov Aug 13 '23 at 17:30
  • The most important, however, the idea in the answer: a proper backup tool **must not** attempt extensive recovery machinery at all. If it can't do a backup easy way, for whatever reason, it must hard fail and notify the operator about the problem encountered; if backup can't be done for any reason, especially due to bad blocks on he source, this is severe issue ignoring which you'd guarantee yourself bigger problems. If source device returns read errors (why doesn't it have RAID in the first place?), you'd recover it (with ddrescue or whatever), and restart your normal file-level backup. – Nikita Kipriyanov Aug 13 '23 at 17:34