Rename files with space with very large directory structure

Question

I've already read through and tried many of the answers here for how to rename files with spaces in a nested directory structure. However, they do not seem to work for my situation. They all segfault.

I have just shy of 1,000,000 files in a directory structure of 32,768 directories. This is also on Windows (Server 2008 R2), and I'm running MINGW32 giving me Bash 3.1.

The directories are within a structure like 00/00/file1 01/00/file2 where each sub-directory "series" varies from 00 to zz. I believe the directory structure only goes 2 levels deep, but I could be wrong. Generating a file count from Windows Explorer's "properties" takes about 45 minutes.

I'm thinking the posted answers here are segfaulting because they are running out of memory building or walking these directories. This is my last attempt before posting here:

find . -maxdepth -1 -mindepth 1 -type -d -printf '%p\n' |
   while IFS= read -r g; do
      find "$g" -depth -name '* *' | while IFS= read -r f ; do 
         mv -i "$f" "$(dirname "$f")/$(basename "$f"|tr ' ' _) 
      done
   done

My last attempt tries walking into at least the first level of subdirectories and then using this solution from another post on this topic:

find . -depth -name '* *' | while IFS= read -r f ; do 
    mv -i "$f" "$(dirname "$f")/$(basename "$f"|tr ' ' _) 
done

Where you can see my version is simply that other post's answer embedded in an outer find loop. However, this also segfaults. I thought to try this because trying the above solution when one directory level in worked.

Two things 1) please don't include tags in the title - that is what tags are for and 2) please use the code, `{}`, option to format code. — Boris the Spider, Aug 30 '14 at 18:42
try writing the results of `find` to a file then use `split -l` to break the problem into smaller problems, then iterate over the content of the split files. — Victory, Aug 30 '14 at 18:43
Why BASH and MinGW? It seems like you're creating a mountain out of a task that is very simple. Use Python or Java or a _real_ language that doesn't have issues with spaces. — Boris the Spider, Aug 30 '14 at 18:43
Why not run `DIR /B /S > FILELIST.TXT` and then you can parse that easily without running it again? — Mark Setchell, Aug 30 '14 at 19:18

Cyrus · Accepted Answer · 2014-08-31T09:47:26.610

3

Create this shell script e.g. ./move.sh:

#!/bin/bash

echo mv "$1" "${1// /_}"

Make it executable.

Run:

find . -depth -name '* *' -exec ./move.sh "{}" \;

If output is okay, remove echo from ./move.sh

edited Aug 31 '14 at 09:47

answered Aug 30 '14 at 19:26

Cyrus

84,225
14
89
153

2

Worth noting that this solution works significantly faster than I expected. Usually doing anything with this directory structure takes 30+ hours. This completed in under two. I'm shocked. – Blake Senftner Aug 31 '14 at 02:18

Rename files with space with very large directory structure

1 Answers1