Does this approach capture changes made to folder names/ file names?

[richardlehane]

Nice repo btw :)

[ross-spencer]

Hey Richard!! you've got eagle eyes! -- I was just writing up a blog about this repository today to introduce it to the world. As I was writing about it I was thinking about this very issue. Ideally (at least in one use-case for this tool) folder names wouldn't be important, but the current algorithm will register some changes to file names and is fairly likely to cause the collection checksum to change -- there's an initial alphabetical sort of reports before processing.

There's another more recursive algorithm for this work I can envision that doesn't sort by filename at all, and works its way up the tree by ordering folders alphabetically by checksum (like it does for files), and then it wont be sensitive to folder naming at all. This is closer to the use-case I was writing for.

There may be a case to be made for both approaches. Are you for or against capturing folder name changes?

[richardlehane]

It feels a little weird to me to verify that a folder is empty but not verify its name?

I'd suggest either:

1. go fully content only: i.e. ignore folder names, file names, and even file hierarchies. Just do the hashing, make a big long lexically sorted list of hashes, and then hash that. This has the advantage of being pretty simple and easy to reason about.
2. continue your current strategy but add file paths and folder names to the hashing i.e. hash NAME+HASH. This has the advantage of verifying more information about the collection but it is more complex as you now have to decide whether a file path is absolute or relative etc.

[ross-spencer]

Thanks Richard. I will keep the naming use-case in mind. It may be another tool for that, it may be an extension of this one.

I will treat the sensitivity to naming as the first major bug of sumfolder1's implementation.

What I am really trying to capture here is a content based analysis, there's a bit more info in the latest blog just in case that helps me elaborate more: https://openpreservation.org/blogs/what-is-the-checksum-of-a-directory/?q=1

I'd like to treat directories like we treat files - at some point it may be possible to take different abstractions as inputs into this algorithm:

E.g.

1. A DROID report.
2. The output of MD5DEEP.
3. An archival listing that includes sumfolder1 compatible data.

Numbers 1. and 2. maybe similar, perhaps one only deals in absolute paths and the other in relative paths. In listing 3. I have may have no paths per se, I may have a path renamed as a finding aid, Where the content doesn't change, they should each be comparable e.g.

1. C:\Collection\Political\Speeches\Opening_of_parliament (acdc1234)
2. speeches/opening_of_parliament (acdc1234)
3. Speeches -> 2021 -> Annual Ceremonial Opening of Parliament (acdc1234)

Unfortunately it took me this far working through it and writing about it to see the cascading effect of some initial sorting of the data I was working with. Fortunately I think it's fairly easy to fix (3-4 hours), and should help me remove some excess codes/checks.