Sometimes you might encounter cases when files are named somewhat weird and makes it hard to use them for automated analysis purposes. In such cases, wildcard expansion becomes a very useful concept. For example, some applications tend to write their logs into log files with some additional information in file names, like:
[root@linux ~]# ls -l /var/log/demo/
total 76
-rw-r--r-- 1 root root 19125 Mar 15 13:48 2024-03-13.demo.log
-rw-r--r-- 1 root root 29580 Mar 15 13:49 2024-03-14.demo.log
-rw-r--r-- 1 root root 22100 Mar 15 13:49 2024-03-15.demo.log
[root@linux ~]#Say log file is being written to as long as calendar date matches the current one. And what if you want to set some automated analysis of such log files, but to do it only for the relevant one, to which application currently writes?
Simulating log analysis
So here we have our small log analysis demo script, which would print us file size and name:
[root@linux ~]# cat analyze_log.sh
#!/bin/bash
my_log="${1}"
# analyze log here
wc -c "${my_log}"
exit 0
[root@linux ~]#If you just provide static file name for it, all is good:
[root@linux ~]# ./analyze_log.sh /var/log/demo/2024-03-15.demo.log
22100 /var/log/demo/2024-03-15.demo.log
[root@linux ~]#But things will get messy if you feed it wildcard:
[root@linux ~]# ./analyze_log.sh "/var/log/demo/*.demo.log"
wc: /var/log/demo/*.demo.log: No such file or directory
[root@linux ~]# ./analyze_log.sh /var/log/demo/*.demo.log
19125 /var/log/demo/2024-03-13.demo.log
[root@linux ~]#If quoted while passing it, it is treated as literal string you are passing. If unquoted, shell expands it already before passing it to the script. So you pass 3 (or as many files as you would have) arguments for the script, that is why it gives you first one.
What is wildcard expansion?
Wildcard expansion, also known as globbing, is a feature found in Unix-like operating systems. It allows you to specify patterns that match filenames or paths in order to select multiple files at once, rather than specifying each file individually.
ls to the rescue
So what if you don’t have luxury to detect current date before running the analysis script and would like to provide the wildcard instead to not have to care what date it is currently?
Adding just one additional line will do the magic:
#!/bin/bash
my_log="${1}"
my_log=$(ls -1t ${my_log} | head -1)
# analyze log here
wc -c "${my_log}"
exit 0
So now you can provide the file name with wildcard and get the expected result – newest file:
[root@linux ~]# ./analyze_log.sh "/var/log/demo/*.demo.log"
22100 /var/log/demo/2024-03-15.demo.log
[root@linux ~]#Just don’t forget to allow shell to expand it at correct place – use quotes as long as you reach that “magic” line.
The ls -1t will show all matching files sorted by modification time, newest first. It will also ensure this list has just the file names without any additional attributes. And then with the head -1 you will take just the first file name, which is what you need. That’s it!