-1

I am trying to get a list of only directory names in order to store it in a variable for a foreach for processing later on.

However, just testing the ls first I am getting weird results.

For example: 

ls -1 /var/lib/mysql/ | grep -e '^d'
db_nagiosql
db_nagiosql_v32
discount-o-matic
drupal
drupal5

However, this does what it should do:

ls -l /var/lib/mysql | grep -e '^d' | awk '{print $9}'
alex
bugs
bugtracker
bugzilla
cacti
cerb5
db_nagiosql
db_nagiosql_v32
discount-o-matic
drupal
drupal5
earth
fft
final_function_test
firm_ware
flyspray
gallery2
graphics
jon
joomla
mysql
nconf
old_fft
opendocman
oreon
part-number
phpbb
phpbugtracker
phplist
postnuke
teldir
test
testing
vanilla
vision
wikidb
wordpress
zen

The issue is, I need this to be portable(ish) so the awk part I rather not have as it may not always be the ninth column. Why does the ls -1 not work while the ls -l /var/lib/mysql | grep -e '^d' | awk '{print $9}' does work?

Jonathan
  • 151
  • 2
  • 2
  • 6
  • 1
    Why have you called `grep` in the first example, if you don't only want directories whose names start with `d`? – Michael Hampton Dec 18 '15 at 17:11
  • That explains that lol. I only want to list directories. I thought the `^d` does that? No files. No information. Just the name of the directory regardless of name listed one per line. – Jonathan Dec 18 '15 at 17:14
  • 2
    No, this can't be true. You wrote a whole question on SF without doing some basic research (like reading the documentation) and realising that you had misread a basic command :( – user9517 Dec 18 '15 at 17:59
  • `ls -1` with a number one does not do a long listing of the entries, so the first character being a `d` only means the file/directory name starts with a `d` not that it is a directory. if you did a `-l` (lower case `L`) instead of `1` your grep would work, but as @inukshuk's answer shows there are better ways to get just the directories – Eric Renouf Dec 18 '15 at 17:59
  • I actually had done research but misread my own comments as to what `^d` was doing. I forgot it was from the d in the permissions rather than directory type, thus why the long listing worked and not the `ls -1`, thus why talking amongst others is a great learning experience for catching such silly mistakes. – Jonathan Dec 18 '15 at 19:50

2 Answers2

2

First, grep is matching a text pattern where ^d specifically matches all lines starting with the d character. When you do ls -l, the lines start with drwx if they are a directory, -rwx if they are a file, lrwx if they are a link, etc (give-or-take the read/write/execute permissions). When you ls -1, it's listing just the file or directory name with no other info, so grep'ing ^d will get you only files and directories that start with a d.

There are two ways to do what you're looking for:

To list only directories with the ls command, you need "ls -d */". To get your desired result, you list all the directories on one line, then remove the trailing / character if you want:

ls -1d */ | sed 's|/$||'

Or alternatively, you can use the ls -l command, grep for the directories like you did, and then awk the last column rather than the 9th column:

ls -l | grep ^d | awk '{print $NF}'
Inukshuk
  • 165
  • 1
  • 2
  • 10
  • In the last example you don't need the `grep` at all since `awk` can do pattern matching, e.g., `awk '/^d/ {print $NF}' – Eric Renouf Dec 18 '15 at 18:01
  • I am going with `ls -l /var/lib/mysql | awk '/^d/ {print $NF}'` as this does exactly what I need and more simplified than having multiple pipes. Thank you. – Jonathan Dec 18 '15 at 19:53
0

If you're trying to portably find all the directories you could use find like so:

 find /var/lib/mysql/ -maxdepth 1 -mindepth 1 -type d -exec basename {} \;

or if you have GNU (so not as portable)

 find /var/lib/mysql/ -maxdepth 1 -mindepth 1 -type d -printf "%f\n"

Assuming you want to further process these you might want to adapt what you're doing to be able to take the full path and let find help with -print0 and xargs -0 so you won't get tripped up with special characters in the path names. See why you shouldn't parse the output of ls for more info about the dangers of that road

Eric Renouf
  • 939
  • 8
  • 19
  • Problem with this it returns the parent directory `mysql` as top result, but a better approach than I had done. – Jonathan Dec 18 '15 at 19:47
  • @Jonathan I updated my answer to fix the inclusion of the parent `mysql` directory – Eric Renouf Dec 18 '15 at 20:05
  • Since these are the actual databases I would think the problem wouldn't occur, though I never tried making a mysql db with special chars. What I am ultimately doing, is `mysqldump`ing all the individual databases so I can recover individual databases easier rather than having EVERYTHING in one sql file dump by doing a `for` loop to iterate all the directories stored in a local variable for iteration. – Jonathan Dec 18 '15 at 20:11
  • @Jonathan first, you can put spaces in the names of databases like `create database \`with space\``, but it seems to not use the space in the directory name at least on the version of mysql I have running. Second, just as a caution, the 2 `find` commands I provided actually still fall under the `don't parse ls` rule I quoted, even though they aren't using `ls`, they have the same problems unless you use the `print0` or use the `-exec` to `find` but then it depends a little on what you do in the `exec` – Eric Renouf Dec 19 '15 at 01:11
  • I'm essentially exporting the contents to a variable in a shell script so I can do a for each on it where it mysqldump each individual database from the list in the variable on a daily cron job. – Jonathan Dec 19 '15 at 01:35
  • @Jonathan ok, just remember it is possible for the name of the directory on disk to be different from the database name (e.g., the `with space` name I created above has a directory name `with@0020space`, since you're already handling the username and password somehow for `mysqldump` you might want to use `mysql -sNe 'show databases'` and carefully parse that output instead, with proper `read` and quoting and all – Eric Renouf Dec 19 '15 at 13:33
  • That is an interesting method, which can be must more useful in this case. I just wish mysqldump had a better way to dump all individual databases at once, individually rather than one large dump, but i digress. Though interestingly enough, the information_schema database does not exist in `/var/lib/mysql` – Jonathan Dec 21 '15 at 15:10
  • @Jonathan I found the same basic question on SO: http://stackoverflow.com/questions/10867520/mysqldump-with-db-in-a-separate-file which seems to offer a suggestion similar to my last comment, but also includes and answer with a possible tool you might want to consider: http://stackoverflow.com/a/10867543/4687135 – Eric Renouf Dec 21 '15 at 16:20
  • I'm reading the on mobile and thus away from office but I'll certainly check that out tomorrow. Looks promising as like that person I'm reinventing the wheel unnecessarily. I have a decent bash script I'm rather proud of including retention but my only issue is obscure errors from cron. It's running on an old server and got a real panic recently. Thanks again. – Jonathan Dec 21 '15 at 23:19