Bulk renaming in a single pass with Go
NameChanger
Renaming multiple files following a pattern often requires small shell scripts. Mike Schilli looks to simplify this task with a Go program.
One popular interview question for system administrators is what is the easiest way to give a set of files a new extension. Take a directory of *.log
files, for example: How do you rename them all in one go to *.log.old
? It has reportedly happened that candidates suggested the shell command mv *.log *.log.old
for this – however, they were then not hired.
There are already quite a few tools lurking around on GitHub that handle such tasks, such as the Renamer tool written in Rust [1]. But such simple utilities make for great illustrative examples, so I wanted to explore some Go techniques for bulk renaming. Paying tribute to the original, the Go variant presented below will also go by the name of Renamer. For example, to rename an entire set of logfiles ending in .log
to .log.bak
, just use the call shown in line 1 of Listing 1.
Listing 1
Renaming Files
01 $ renamer -v '.log$/.log.bak' *.log 02 out.log -> out.log.bak 03 [...] 04 $ renamer -v '/hawaii2020-{seq}.jpg' *.JPG 05 IMG_8858.JPG -> hawaii2020-0001.jpg 06 IMG_8859.JPG -> hawaii2020-0002.jpg
Or how about renaming vacation photos currently named IMG_8858.JPG
through IMG_9091.JPG
to hawaii-2020-0001.jpg
through hawaii-2020-0234.jpg
? My Go program does that too with the call from line 4, replacing the placeholder {seq}
with a counter incremented by one for each renamed file, which it pads with leading zeros to four digits.
Mass Production
The renamer
main program (Listing 2) processes its command-line options -d
for a test run without consequences (dryrun
) and -v
for chatty (verbose
) status messages in lines 19 and 20. The standard flag package used for this purpose not only assigns the dryrun
and verbose
pointer variables the values true
or false
, respectively, but it also jumps to a Usage()
function defined in the Usage
attribute if the user tries to slip in an option that the program doesn't know.
Listing 2
renamer.go
01 package main 02 03 import ( 04 "flag" 05 "fmt" 06 "os" 07 "path" 08 ) 09 10 func usage() { 11 fmt.Fprintf(os.Stderr, 12 "Usage: %s 'search/replace' file ...\n", 13 path.Base(os.Args[0])) 14 flag.PrintDefaults() 15 os.Exit(1) 16 } 17 18 func main() { 19 dryrun := flag.Bool("d", false, "dryrun only") 20 verbose := flag.Bool("v", false, "verbose mode") 21 flag.Usage = usage 22 flag.Parse() 23 24 if *dryrun { 25 fmt.Printf("Dryrun mode\n") 26 } 27 28 if len(flag.Args()) < 2 { 29 usage() 30 } 31 32 cmd := flag.Args()[0] 33 files := flag.Args()[1:] 34 modifier, err := mkmodifier(cmd) 35 if err != nil { 36 fmt.Fprintf(os.Stderr, 37 "Invalid command: %s\n", cmd) 38 usage() 39 } 40 41 for _, file := range files { 42 modfile := modifier(file) 43 if file == modfile { 44 continue 45 } 46 if *verbose || *dryrun { 47 fmt.Printf("%s -> %s\n", file, modfile) 48 } 49 if *dryrun { 50 continue 51 } 52 err := os.Rename(file, modfile) 53 if err != nil { 54 fmt.Printf("Renaming %s -> %s failed: %v\n", 55 file, modfile, err) 56 break 57 } 58 } 59 }
In any case, the program expects a command to manipulate the file names and one or more files to rename later. Line 12 informs the user of the correct call syntax of the renamer
binary compiled from the source code.
The array slice arithmetic assigns the first command-line parameter with index number
to the cmd
variable. This is followed by one or more file names, which the shell is also welcome to expand using wildcards before passing them to the program. The arguments from the second to last position are fetched from the array slice by the expression [1:]
; line 33 assigns the list of files to the variable files
.
The instruction passed in at the command line to manipulate the file names (e.g., '.log$/.log.old'
) gets sent to the mkmodifier()
function defined further down in Listing 3. This turns the instruction into a Go function that manipulates input file names according to the user's instructions and returns a modified name.
Listing 3
mkmodifier.go
01 package main 02 03 import ( 04 "errors" 05 "fmt" 06 "regexp" 07 "strings" 08 ) 09 10 func mkmodifier(cmd string) (func(string) string, error) { 11 parts := strings.Split(cmd, "/") 12 if len(parts) != 2 { 13 return nil, errors.New("Invalid repl command") 14 } 15 search := parts[0] 16 repltmpl := parts[1] 17 seq := 1 18 19 var rex *regexp.Regexp 20 21 if len(search) == 0 { 22 search = ".*" 23 } 24 25 rex = regexp.MustCompile(search) 26 27 modifier := func(org string) string { 28 repl := strings.Replace(repltmpl, 29 "{seq}", fmt.Sprintf("%04d", seq), -1) 30 seq++ 31 res := rex.ReplaceAllString(org, repl) 32 return string(res) 33 } 34 35 return modifier, nil 36 }
Function Returns Function
You've read that correctly: The mkmodifier()
function actually returns a function in line 34 of Listing 2, which is assigned to the modifier
variable there. A few lines down, in the for
loop that iterates over all the files to be manipulated, the main program simply calls this function by referencing modifier
. With every call, the main program passes the returned file name the original name of the file and, in line 42, picks up the new name and stores it in modfile
.
If the user has chosen dryrun
mode (-d
), line 47 simply prints the intended rename action, and line 50 rings in the next round of the for
loop with continue
, skipping the call of rename()
in line 52.
In production mode, however, line 52 calls the Unix system rename()
function from the standard os package and renames the file to the new name from modfile
. If access rights prevent this, the function fails and os.Rename()
returns an error, which line 53 fields. The associated if
block prints a message and breaks the for
loop with break
, because in that case the end of the world is nigh.
Regular Expressions
Instead of requesting a plain vanilla string replacement, the user can also specify regular expressions to remodel file names. For example, the .log$
search expression illustrated earlier specifies that the .log
suffix must actually be at the end of the name – it would ignore foo.log.bak
. To enable this, Listing 3 draws on the standard regexp package and compiles the regular expression from the user input to create a rex
variable of the *regexp.Regexp
type using MustCompile()
in line 25. After that, the modifier defined in line 27 can call the ReplaceAllString()
function. It replaces all matches that match the expression in the original name org
with the replacement string stored in repl
.
Attentive readers may wonder about the mkmodifier()
function in Listing 3: It returns a function to the main program, to be called multiple times, but this function actually seems to maintain state between calls. For example, take a look at the function's local variable seq
: Each new call to the function injects a value incremented by one into the modified file name. How is this possible?
Buy this article as PDF
(incl. VAT)
Buy Linux Magazine
Subscribe to our Linux Newsletters
Find Linux and Open Source Jobs
Subscribe to our ADMIN Newsletters
Support Our Work
Linux Magazine content is made possible with support from readers like you. Please consider contributing when you’ve found an article to be beneficial.
News
-
Systemd Fixes Bug While Facing New Challenger in GNU Shepherd
The systemd developers have fixed a really nasty bug amid the release of the new GNU Shepherd init system.
-
AlmaLinux 10.0 Beta Released
The AlmaLinux OS Foundation has announced the availability of AlmaLinux 10.0 Beta ("Purple Lion") for all supported devices with significant changes.
-
Gnome 47.2 Now Available
Gnome 47.2 is now available for general use but don't expect much in the way of newness, as this is all about improvements and bug fixes.
-
Latest Cinnamon Desktop Releases with a Bold New Look
Just in time for the holidays, the developer of the Cinnamon desktop has shipped a new release to help spice up your eggnog with new features and a new look.
-
Armbian 24.11 Released with Expanded Hardware Support
If you've been waiting for Armbian to support OrangePi 5 Max and Radxa ROCK 5B+, the wait is over.
-
SUSE Renames Several Products for Better Name Recognition
SUSE has been a very powerful player in the European market, but it knows it must branch out to gain serious traction. Will a name change do the trick?
-
ESET Discovers New Linux Malware
WolfsBane is an all-in-one malware that has hit the Linux operating system and includes a dropper, a launcher, and a backdoor.
-
New Linux Kernel Patch Allows Forcing a CPU Mitigation
Even when CPU mitigations can consume precious CPU cycles, it might not be a bad idea to allow users to enable them, even if your machine isn't vulnerable.
-
Red Hat Enterprise Linux 9.5 Released
Notify your friends, loved ones, and colleagues that the latest version of RHEL is available with plenty of enhancements.
-
Linux Sees Massive Performance Increase from a Single Line of Code
With one line of code, Intel was able to increase the performance of the Linux kernel by 4,000 percent.