Organizing photos by date with Go
Keeping Things Tidy
In this issue, Mike conjures up a Go program to copy photos from a cell phone or SD card into a date-based file structure on a Linux box. To avoid wasting time, a cache using UUIDs ensures that only new photos are transferred.
I regularly import photos from my phone or the SD card of my brand new mirrorless camera (a Sony A7) to my home computer in order to archive the best shots. On the computer, a homegrown program sorts them into a folder structure that creates a separate directory for each year, month, and day. After importing, the images usually remain on the card or phone. Of course, I don't want the importer to recopy previously imported images the next time it is called, but instead pick up where it left off the last time. If several SD cards are used, it is important to keep track of them because they sometimes use conflicting file names.
The photos on the SD card are files with a name format of DSC<number>.JPG
. On the phone, they have a different file name, say, IMG_<number>.JPG
. Cameras and photo apps increment the consecutive number of newly taken photos by one for each shot. This process is described in the Design rule for Camera File system (DCF) [1] specification. The DCF specification defines the format of the file names along with their counters and specifies what happens if a counter overflows or the camera detects that the user has used other SD cards with separate counters in the meantime.
Figure 1 shows the typical, DCF-compliant file layout on the card. On a freshly formatted card, the camera saves the first images as DSC00001.JPG
, DSC00002.JPG
, and so on in the 100MSDCF/
subdirectory; this, in turn, is located in the DCIM
folder. Now, it's unlikely for anyone to store 99,999 pictures on a card, but if a crazy photographer actually shot that many photos, the camera would create a new directory named 101MSDCF/
and, after the next shot, would simply start again at DSC00001.JPG
.
Interesting things happen if a photographer changes SD cards without reformatting the freshly inserted card: The camera's internal counter jumps from the previously monotonously increasing value to the value of the image with the highest counter on the SD card. Imagine that, after taking DSC02001.JPG
, the photographer switches to an SD card that already contains a photo named DSC09541.JPG
. In this case, the camera would continue with DSC09542.JPG
even if DSC02002.JPG
still happened to be available. Depending on the camera model and software version, there can be some deviations.
Loose Standard
As an experiment, I manipulated an SD card serving in my Sony A7. Its directory 100MSDCF/
was filled with images ranging from DSC00205.JPG
to DSC00952.JPG
. When I manually inserted a new photo named DSC99999.JPG
into the card and reinserted the card into the camera, the camera software actually created the new directory 101MSDCF/
(as a peer to 100MSDCF/
) on the card and saved newly captured images there as DSC00953.JPG
, DSC00954.JPG
, and so on (see Figure 2)!
In other words, the camera remembers – even after it has been turned off and on again – the last image it took and the folder where it stored the shot. When I deleted the fake image DSC99999.JPG
from 100MSDCF/
again, the camera still continued with DSC00954.JPG
in the 101MSDCF/
directory.
However, if you routinely swap SD cards, you will often find new files on them with names that photos in your external storage archive already use. If my algorithm were to rely only on the original file name as a key when importing photos, it would either overwrite existing files in the computer archive or conclude that some files had already been imported previously and should therefore be ignored during the current import. It would be wrong on both counts. Instead, the importer has to store any photos that are not already in the archive, regardless of their original names.
Check and Save
How can an import application determine if a file on the SD card is actually new, even if there is already an image with the same name in the archive? The Go program presented here resorts to a cache file that makes use of the parent directories and a UUID of the respective SD card for imported photos.
Figure 3 shows the importer in action. Called up with the name of the photo directory (normally that of the SD card inserted), the importer works its way through the individual images, plumbing the depths of the card structure. It checks if the particular photo has been copied previously according to the cache data. If not, it archives it in a date-based file structure (Figure 4).
Knotted Handkerchief
Listing 1 implements the cache that helps the program to remember which photos importer
already copied. It relies on file names and file sizes to do this. The cache is a Go map of the type map[string]bool
; it assigns a value of true
to each photo path (as a string) if the respective photo has already been copied. The photo path not only includes the name of the photo file, but also the name of the directory in which it is located on the card (e.g., 100MSDCF/
in Figure 5).
Listing 1
cacher.go
01 package main 02 03 import ( 04 "bufio" 05 "fmt" 06 "github.com/google/uuid" 07 "io/ioutil" 08 "os" 09 "path" 10 "strings" 11 ) 12 13 const uuidFile = ".uuid" 14 const cacheFile = ".idb-import-cache" 15 16 type Cache struct { 17 uuid string 18 iPath string 19 uuidPath string 20 cachePath string 21 cache map[string]bool 22 } 23 24 func NewCache(ipath string) *Cache { 25 return &Cache{ 26 uuid: "", 27 uuidPath: path.Join(ipath, uuidFile), 28 iPath: ipath, 29 cachePath: "", 30 cache: map[string]bool{}, 31 } 32 } 33 34 func (cache *Cache) Init() { 35 buf, err := ioutil.ReadFile(cache.uuidPath) 36 if err == nil { 37 cache.uuid = strings.TrimSpace(string(buf)) 38 } else { 39 if os.IsNotExist(err) { 40 uuid := uuid.New().String() 41 err := ioutil.WriteFile(cache.uuidPath, []byte(uuid), 0644) 42 panicOnErr(err) 43 cache.uuid = uuid 44 } else { 45 panicOnErr(err) 46 } 47 } 48 49 homedir, err := os.UserHomeDir() 50 panicOnErr(err) 51 cache.cachePath = path.Join(homedir, cacheFile) 52 } 53 54 func (cache *Cache) Read() { 55 f, err := os.Open(cache.cachePath) 56 if os.IsNotExist(err) { 57 return 58 } 59 panicOnErr(err) 60 defer f.Close() 61 62 scanner := bufio.NewScanner(f) 63 for scanner.Scan() { 64 line := scanner.Text() 65 cache.cache[line] = true 66 } 67 68 return 69 } 70 71 func (cache Cache) Write() { 72 f, err := os.OpenFile(cache.cachePath, os.O_RDWR|os.O_CREATE|os.O_TRUNC, 0644) 73 panicOnErr(err) 74 defer f.Close() 75 76 for k, _ := range cache.cache { 77 fmt.Fprintf(f, "%s\n", k) 78 } 79 return 80 } 81 82 func (cache Cache) Exists(key string) bool { 83 _, ok := cache.cache[cache.uuid+":"+key] 84 return ok 85 } 86 87 func (cache Cache) Set(key string) { 88 cache.cache[cache.uuid+":"+key] = true 89 }
The program uses a 36-digit UUID to identify the SD card. During the first import of photos on a never-before-used card, it creates the UUID in the .uuid
file at the root level of the card's filesystem and rereads it from there for subsequent import attempts. As you can see in Figure 5, the card's UUID is also part of the key of already imported photos in the cache. This way, the importer knows exactly which card a specific image came from.
In Listing 1, the structure Cache
starting in line 16 defines the data of a cache instance for the card currently being processed. The NewCache()
constructor starting in line 24 returns the pre-initialized structure as a pointer to the caller. The caller stores the pointer in a variable such as cache
. If the programmer then types cache.Function()
, Go passes the structure pointer to the function, using its receiver mechanism – object orientation in Go.
Buy this article as PDF
(incl. VAT)
Buy Linux Magazine
Subscribe to our Linux Newsletters
Find Linux and Open Source Jobs
Subscribe to our ADMIN Newsletters
Support Our Work
Linux Magazine content is made possible with support from readers like you. Please consider contributing when you’ve found an article to be beneficial.
News
-
Canonical Bumps LTS Support to 12 years
If you're worried that your Ubuntu LTS release won't be supported long enough to last, Canonical has a surprise for you in the form of 12 years of security coverage.
-
Fedora 40 Beta Released Soon
With the official release of Fedora 40 coming in April, it's almost time to download the beta and see what's new.
-
New Pentesting Distribution to Compete with Kali Linux
SnoopGod is now available for your testing needs
-
Juno Computers Launches Another Linux Laptop
If you're looking for a powerhouse laptop that runs Ubuntu, the Juno Computers Neptune 17 v6 should be on your radar.
-
ZorinOS 17.1 Released, Includes Improved Windows App Support
If you need or desire to run Windows applications on Linux, there's one distribution intent on making that easier for you and its new release further improves that feature.
-
Linux Market Share Surpasses 4% for the First Time
Look out Windows and macOS, Linux is on the rise and has even topped ChromeOS to become the fourth most widely used OS around the globe.
-
KDE’s Plasma 6 Officially Available
KDE’s Plasma 6.0 "Megarelease" has happened, and it's brimming with new features, polish, and performance.
-
Latest Version of Tails Unleashed
Tails 6.0 is based on Debian 12 and includes GNOME 43.
-
KDE Announces New Slimbook V with Plenty of Power and KDE’s Plasma 6
If you're a fan of KDE Plasma, you'll be thrilled to hear they've announced a new Slimbook with an AMD CPU and the latest version of KDE Plasma desktop.
-
Monthly Sponsorship Includes Early Access to elementary OS 8
If you want to get a glimpse of what's in the pipeline for elementary OS 8, just set up a monthly sponsorship to help fund its continued existence.