Pattern-based filename filtering in gnu shell command

Question

Lets say I have an active/ directory that contains these files

active/
foo.bar.abc
foo.bar.xyz
foo.bat.abc

archive/
foo.bat.xyz

I want to write a command to output only unique filenames in active/ (uniqueness based on the middle item) AND doesn't match to any files already in archive/ (again based on that middle term).

Sample output:

foo.bar.abc

Explanation: output either foo.bar.abc or foo.bar.xyz doesn't matter. Not foo.bat.abc since foo.bat.xyz exists in archive/

I've found this to help identify unique values based on a pattern but I can't figure out how to combine that with my additional clause of no match in archive/

Answer 1

Awk is actually not needed here, you can do it with simple grep/sed and sort:

(ls ./archive | sed 's/^/1 /'; ls ./active | sed 's/^/2 /') | \
  sort --field-separator="." --key="2,2" --uniq --stable | \
  grep '^2 ' | sed 's/^2 //'

Explanation:

First list both directories and mark which lines are from which directory. Then sort both listings together by their middle parts. Option --field-separator="." splits all lines into fields on dosts and option --key="2,2" tells to sort by the middle field, i.e. by the part between the dots. We use a stable sort to make sure lines from archive are the first and tell sort to print only the first matches of all duplicate lines.

Finally we filter only lines that we marked with 2, i.e. the lines from ./active.

Example:

active/
  foo.aaa.xxx
  foo.bar.abc
  foo.bar.xyz
  foo.bat.abc
  zoo.aaa.xxx
  zoo.bbb.aaa


archive/
  aaa.bbb.zoo
  foo.bat.xyz

Result:
  foo.aaa.xxx
  foo.bar.abc

Answer 2

Another attempt using GNU grep, awk and GNU findutils

$ grep -Fxvf <(find active/ -type f -printf '%P\n' | awk -F'.' '!seen[$2]++') <(find archive/ -type f -printf '%P\n' | awk -F'.' '!seen[$2]++')
foo.bar.xyz

Am using process-substitution <() to run the find/awk commands and pass it to grep for finding the difference.

While find command lists the file on the specified directory, one entry per line, the awk filters the list by retaining the ones which are not duplicated by 2nd word. For awk, with the delimiter as . !seen[$2]++ prints only unique lines by hashing it in the array only if it hasn't been seen before.

Do remember the -printf '%P in find is NOT POSIX compatible and will work with GNU findutils. Recommend upgrading to it for this to work.

Other possible solutions, with a similar logic, one with comm and join, both part of GNU coreutils are below:-

$ join -v 2 <(find active/ -type f -printf '%P\n' | awk -F'.' '!seen[$2]++') <(find archive/ -type f -printf '%P\n' | awk -F'.' '!seen[$2]++')
foo.bar.xyz

Another with comm

$ comm -13 <(find active/ -type f -printf '%P\n' | awk -F'.' '!seen[$2]++') <(find archive/ -type f -printf '%P\n' | awk -F'.' '!seen[$2]++')
foo.bar.xyz