Compare strings inside in the two different directories using array

Question

I don't get the scenario of this given code. All I wanted is to compare the files that is given below. But, in this script nothings happen. I assume that this given code can executed wherever like in /root and it will run. Please check this out.

#!/bin/bash
for file in /var/files/sub/old/*
do
# Strip path from file name
file="${file##*/}"

# Strip everything after the first hyphen
prefix="${file%%-*}-"

# Strip everything before the second-to-last dot
suffix="$(echo $file | awk -F. '{ print "."$(NF-1)"."$NF }')"

# Create new file name from $prefix and $suffix, and any version number
new=$(echo "/var/files/new/${prefix}"*"${suffix}")

# If file exists in the 'new' folder:
if test -f "${new}"
then
# Do string comparison to see if new file is lexicographically "greater than" old
if [[ "${new##*/}" > "${file}" ]]
then
  # If so, delete the old version.
  rm /var/sub/files/old/"${file}"
else
  # 'new' file is NOT newer, delete it instead.
  rm "${new}"
  fi
  fi
done

# Move all new files into the old folder.
mv /var/files/new/* /var/files/sub/old/

Example files inside of each sub- directories ..

/var/files/sub/old/
firefox-24.5.0-1.el5_10.i386.rpm
firefox-24.5.0-1.el5_10.x86_64.rpm
google-1.6.0-openjdk-1.6.0.0-5.1.13.3.el5_10.x86_64.rpm
google-1.6.0-openjdk-demo-1.6.0.0-5.1.13.3.el5_10.x86_64.rpm

/var/files/new/
firefox-25.5.0-1.el5_10.i386.rpm
firefox-25.5.0-1.el5_10.x86_64.rpm
ie-1.6.0-openjdk-devel-1.6.0.0-5.1.13.3.el5_10.x86_64.rpm
ie-1.6.0-openjdk-javadoc-1.6.0.0-5.1.13.3.el5_10.x86_64.rpm
ie-1.6.0-openjdk-src-1.6.0.0-5.1.13.3.el5_10.x86_64.rpm
google-2.6.0-openjdk-demo-1.6.0.0-5.1.13.3.el5_10.x86_64.rpm

In this instance, I want to get the files that are the same. So the files that are the same in the given example are:

firefox-24.5.0-1.el5_10.i386.rpm
firefox-24.5.0-1.el5_10.x86_64.rpm
google-1.6.0-openjdk-demo-1.6.0.0-5.1.13.3.el5_10.x86_64.rpm

in the old/ directory and for the new/ directory the equivalents are:

firefox-25.5.0-1.el5_10.i386.rpm
firefox-25.5.0-1.el5_10.x86_64.rpm
google-2.6.0-openjdk-demo-1.6.0.0-5.1.13.3.el5_10.x86_64.rpm

The files have similarity for their first characters. It will display in the terminal. After that, there will be another comparing again of the files and the comparison is about which file is more updated one by the number after the name of the file like: firefox-24.5.0-1.el5_10.i386.rpm compared with firefox-25.5.0-1.el5_10.i386.rpm. So in that instance the firefox-24.5.0-1.el5_10.i386.rpm will be replaced by firefox-25.5.0-1.el5_10.i386.rpm because it has a greater value and more updated one and same as other files that are similar. And if the old one is removed and the new will take replacement of it.

So at this moment after the script has been executed the output will be like this.

/var/files/sub/old/
google-1.6.0-openjdk-1.6.0.0-5.1.13.3.el5_10.x86_64.rpm
firefox-25.5.0-1.el5_10.i386.rpm
firefox-25.5.0-1.el5_10.x86_64.rpm
ie-1.6.0-openjdk-devel-1.6.0.0-5.1.13.3.el5_10.x86_64.rpm
ie-1.6.0-openjdk-javadoc-1.6.0.0-5.1.13.3.el5_10.x86_64.rpm
ie-1.6.0-openjdk-src-1.6.0.0-5.1.13.3.el5_10.x86_64.rpm
google-2.6.0-openjdk-demo-1.6.0.0-5.1.13.3.el5_10.x86_64.rpm

/var/files/new/
<<empty all files here  must to moved to other directory take as a replacement>>

Can anyone help me to make a script for this ? above is just an example. Let's assume that there are lots of files to considered as similar and need to removed and moved.

It will make your life easier if you learn how to indent code in SO (or text like the output of `ls`). Write (or copy/paste) the code in the edit box as you want to see it; no tabs (4 spaces per tabstop recommended but not mandatory). Then select it, and use the **`{}`** button above the edit box to indent it all. It saves a lot of XML/HTML markup! — Jonathan Leffler, Jun 03 '14 at 03:01
Not sure what you're asking. I think you want to remove files from the `new` directory if they're the same as the file in the `old` directory. So you need an `if` statement -- your script is removing them without testing the result of `diff`. Also, I suggest you use `cmp` rather than `diff`, it's more efficient unless you actually want to see all the differences. — Barmar, Jun 03 '14 at 03:08
It's a decidedly non-trivial exercise. You need to break the string into a sequence of non-version and version components, and make sure that for any two files, they have the same number of components, and that all the non-version components are the same and then compare the version components, probably from left-to-right. Comparing versions such as `9.10.1` and `10.9.1` requires some care; sequencing `9.9.9`, `9.9.10`, `9.10.9`, `9.10.10`, `10.0.0` also requires some care. — Jonathan Leffler, Jun 03 '14 at 03:19
Parsing an RPM filename string is, indeed, an ugly task but the specification for how to do so is fixed and fairly straightforward. That being said, and as I indicated in my answer, when you don't need to do that you shouldn't bother doing it. Comparing those values, once split, is an entirely different (and much uglier) matter. — Etan Reisner, Jun 03 '14 at 03:22
@EtanReisner: I think it likely that using `rpm` or an equivalent is the better approach. I'd be curious to know what it would give as the information for the three IE files, and for the Google files. There are three separate version numbers in them. — Jonathan Leffler, Jun 03 '14 at 03:24
@JonathanLeffler The filenames parse backwards. "x86_64", "5.1.13.3.el5_10", "1.6.0.0", and "ie-1.6.0-openjdk-src" are the constituent parts if my parsing was remembered (and performed manually) correctly. Those parts being arch, release, version, name. — Etan Reisner, Jun 03 '14 at 03:32
For the curious: yum code for this splitting is [here](http://yum.baseurl.org/gitweb?p=yum.git;a=blob;f=rpmUtils/miscutils.py;h=aea455082c91c51bcc5986174c2bd1f699ff1d94;hb=HEAD#l301) — Etan Reisner, Jun 03 '14 at 03:35

score 1 · Answer 1 · answered Jun 03 '14 at 03:16

You can use rpm to get the name of the package without version or architecture strings:

rpm -qi -p /firefox-25.5.0-1.el5_10.i386.rpm

Gives:

Name        : firefox
Version     : 25.5.0
Release     : 1.el5_10
Architecture: i386
....

So you can compare the Names to find related packages.

score 0 · Answer 2 · answered Jun 03 '14 at 03:14

If the goal here is to have the newrpms directory have only the newest version of each RPM from a combination of sources then you most likely want to simply combine all the files in a single directory and then use the repomanage tool (from the yum-utils package, at least on CentOS) to have it inform you which of the RPMS are old and remove them.

Something like:

repomanage --old combined_rpms_directory | xargs -r rm

As to your initial script

for i in $(\ls -d ./new/*);
do 
    diff ${i} newrpms/; 
    rm ${i}
done

You generally don't want to "parse" the output from ls, especially when a glob will do what you want just as easily (for i in ./new/* in this case).

diff ${i} newrpms/ is attempting to diff a file and a directory (or two directories if your ls/glob happened to catch a directory) but in neither case will diff do what you want there. That being said what diff does doesn't really matter because, as Barmar said in his comment

your script is removing them without testing the result of diff

@JonathanLeffler It is part of the yum-utils package in CentOS (and presumably other RH-based distributions). Edited my answer to indicate that. — Etan Reisner, Jun 03 '14 at 03:18

score 0 · Answer 3 · answered Jun 05 '14 at 06:56

A bash script that does the checking. Here's how it works:

Traverse over each file in the old files directory. Get the prefix (package name with no version, architecture, etc), eg. firefox-; get the suffix (architecture.rpm), eg. .i386.rpm.
Attempt to match prefix and suffix with any version number within the new files directory, ie. firefox-*.i386.rpm. If there is a match, $new will contain the file name, eg. firefox-25.5.0-1.el5_10.i386.rpm; if no match, $new will equal the literal string firefox-*.i386.rpm which is not a file.
Check new files directory for existence of $new.
If it exists, check that $new is indeed newer than the old version. This is done by lexicographical string comparison, ie. firefox-24.5.0-1.el5_10.i386.rpm is less than firefox-25.5.0-1.el5_10.i386.rpm because it comes earlier in the alphabet. Conveniently, sane versioning schemes also happen to be alphabetical. NB: this may fail, for example, when comparing version 2 to version 10.
A new version of a file in the old files directory has been found! In this case, get rid of the old file with rm. If the file in the new directory is not newer, then delete it instead.
Done removing old versions. Old files directory has only files without newer versions.
Move all new files into old directory, leaving newest files in old directory, and new directory empty.

#!/bin/bash
for file in /var/files/sub/old/*
do
  # Strip path from file name
  file="${file##*/}"

  # Strip everything after the first hyphen
  prefix="${file%%-*}-"

  # Strip everything before the second-to-last dot
  suffix="$(echo $file | awk -F. '{ print "."$(NF-1)"."$NF }')"

  # Create new file name from $prefix and $suffix, and any version number
  new=$(echo "/var/files/new/${prefix}"*"${suffix}")

  # If file exists in the 'new' folder:
  if test -f "${new}"
  then
    # Do string comparison to see if new file is lexicographically "greater than" old
    if [[ "${new##*/}" > "${file}" ]]
    then
      # If so, delete the old version.
      rm /var/sub/files/old/"${file}"
    else
      # 'new' file is NOT newer, delete it instead.
      rm "${new}"
    fi
  fi
done

# Move all new files into the old folder.
mv /var/files/new/* /var/files/sub/old/

`new=$(echo /var/files/new/${prefix}"*"${suffix})` changed into this, what does this code do ? — user3626800, Jun 10 '14 at 02:33
It looks for files matching a certain file name in /var/files/new/. For example, if `$prefix` is `firefox-` and `$suffix` is `.i386.rpm`, the `*` will be expanded by `bash` to match anything in between, for example `25.5.0-1.el5_10`. So `$new` becomes `firefox-25.5.0-1.el5_10.i386.rpm`. — savanto, Jun 10 '14 at 07:26
as what I've noticed when I executed it .. `./checker.sh: line 12: /var/files/new/firefox-*.i386.rpm: No such file or directory` whats wrong of this ? — user3626800, Jun 10 '14 at 09:07
I changed the line for `new=$(echo "/var/file/new/${prefix}"*"${suffix}")` to `new=$("/var/file/new/${prefix}"*"${suffix}")` because for the first one it will just display `/var/file/new` word — user3626800, Jun 10 '14 at 09:09

Compare strings inside in the two different directories using array

3 Answers3

Related