rAudio 20240818: Thumbnail update stops after 304 successful creations

[klassikmann]

System: rAudio 20240818 + UPDATE regarding #1728 + UPDATE regarding #1746

For testing purposes I deleted all coverart.jpg and thumb.jpg to trigger a complete recreation of thumbnails.
Effect: Every time after 304 successful creations the system stops. During the creation of the next pair of thumbnails the corresponding "convert" process (part of imagemagick) does not finish. Interrupting plus restarting "Update thumbnails" produce the next bunch of 304 thumbnails.

[klassikmann]

Still present in rAudio 20240906 (process stops after exactly 304 successful thumbnail creations) but the problem is getting worse.

Interrupting plus restarting "Update thumbnails" produce the next bunch of 304 thumbnails.

This is no more possible, because the option not to override existing thumbnails in "Update Thumbnails" vanished.

[klassikmann]

@rern
Any advice for me to track down the issue. Thumbnails are generated in albumthumbnail.sh:

		else
			convert "$coverfile" -thumbnail 200x200\> -unsharp $unsharp "$dir/coverart.jpg"
			[[ $? == 0 ]] && convert "$coverfile" -thumbnail 80x80\> -unsharp $unsharp "$dir/thumb.jpg" || error=1
		fi

At run 304 of these two lines "convert" does not finish.

And I ask to reinstate the former behaviour providing a checkbox to override existing thumbnails explicitily.

[klassikmann]

@rern

Try context menu Update thumbnails at that directory.

That's what I already did. But the checkbox not to override existing thumbnails explicitily is no more available.

Try copy the cover.jpg to another directory and run the command in that directory:
...

Done and tested: No restriction at 304 conversions (btw. "convert" is now deprecated, use "magick")

Option Rebuild all = Replace all existings

Planned for tomorrow.

[klassikmann]

@rern
Finishing the tests:

Option Rebuild all = Replace all existings

Planned for tomorrow.

304 thumbnails (coverart.jpg + thumb.jpg) successfully built from folder.jpg. No 305 hangs indefinitely. The "304 bug" appears regardless of "global" or "local" thumbnails update.

As I already mentioned above: In the recent releases "Update thumbnails" in the context menu is missing a radio button "Only added or removed", prohibiting a "bunch-by-bunch" thumbnails update of big subtrees:

The relevant processes after "convert" stops:

F   UID     PID    PPID PRI  NI    VSZ   RSS WCHAN  STAT TTY        TIME COMMAND
4     0     315       1  20   0  81688 20608 do_epo Ss   ?          0:00 php-fpm: master process (/etc/php/php-fpm.conf)
5    33     379     315  20   0  81752 11480 pipe_r S    ?          0:00 php-fpm: pool www
4     0     995     379  20   0  17848  6784 do_sys S    ?          0:00 /usr/bin/sudo /srv/http/bash/albumthumbnail.sh NAS/Musik
4     0     997     995  20   0   5456  3524 do_wai S    ?          0:04 /bin/bash /srv/http/bash/albumthumbnail.sh NAS/Musik/Neu
0     0    3179     997  20   0  55460  8960 pipe_w S    ?          0:00 convert /mnt/MPD/NAS/Musik/Neue Musik/Satie, Erik/Olga S

[rern]

• Backup the 20210416 #305 directory then delete it
• Try update again

Options show only when that directory contains subdirectories otherwise always overwrite.

[klassikmann]

@rern
First, your reference to a very old discussion (#305) is a typo, isn't if?

I assume your advice tries to exclude a problem in a specific directory. My music archive contains a few directories at the top level. But below these are some very large sub-trees. The "304-bug" applies regardless which of these big trees or even the whole archive is input for a thumbnail update. So, a problem with one specific directory can be excluded.

Options show only when that directory contains subdirectories otherwise always overwrite.

That's not my experience which is described above.

[rern]

• Typo - Github markdowned from - Backup the #305 directory then delete it
• The overwrite option bug was found. Please try the UPDATE.

[klassikmann]

The overwrite option bug was found. Please try the UPDATE.

Fixed.

As I wrote above the "304-bug" appears at every part of my directory tree as long as more than 304 "folder.jpg" are present in the tree.

[klassikmann]

@rern
Only an wild speculation:
As you see in the last two lines of the "ps axl" output during the endless sleep at the 305th thumbnail generation, "convert" is sleeping at a write to a pipe, and the parent is waiting:

UID     PID    PPID PRI  NI    VSZ   RSS WCHAN  STAT TTY        TIME COMMAND
  0     997     995  20   0   5456  3524 do_wai S    ?          0:04 /bin/bash /srv/http/bash/albumthumbnail.sh NAS/Musik/Neu
  0    3179     997  20   0  55460  8960 pipe_w S    ?          0:00 convert /mnt/MPD/NAS/Musik/Neue Musik/Satie, Erik/Olga S

My (possible/probable incorrect interpretation): There might be an exhausted ressource after 304x2 (coverart.jpg + thumb.jpg) runs of "convert". A candidate for such a ressouce is the pipe buffer size (64k).

[klassikmann]

@rern
My speculation is valid, the problem is solved!

With your yesterday's change of albumthumbnail.sh in UPDATE replacing the deprecated "convert" with "magick" the warning message "WARNING: The convert command is deprecated in IMv7, use "magick" instead of "convert" or "magick convert"
vanished. The size of the message is 106 chars. 106 * 304 * 2 = 64.448 which is slightly below 64k=65536, the pipe buffer size.
I just repeated the generation of thumbnails in a subtree containing 913 folder.jpg. It performed successfully.

[klassikmann]

@rern
Add-on: I'm using rAudio 20240908 + yesterday's UPDATE. Trying to make a global "Update thumbnails" in the album windows yields to:

[ceejayemm]

@rern
With the same setup I am seeing the same problem.

Chris

[rern]

Please try the UPDATE again.

[klassikmann]

@rern @ceejayemm
UPDATE: Fixed

[ceejayemm]

Agreed, fixed with latest Update branch

Thanks

Chris

[ceejayemm]

Oops! Spoke too soon. The thumbnail update runs and produces output for all my existing albums but right at the very end I get:

342/342 NAS/NAS_Music/Jethro Tull/The Zealot Gene
      #339 - Thumbnail created.

 !  Coversion failed:

/mnt/MPD/NAS/NAS_Music/David Gilmour/Luck and Strange/cover.jpg

Duration: 00:01:57

followed by:

magick: unable to open image '/mnt/MPD/NAS/NAS_Music/David Gilmour/Luck and Strange/coverart.jpg': Permission denied @ error/blob.c/OpenBlob/3596.

This album is new to me today and is stored as .flac files with embedded album covers and existing cover.jpg, coverart.jpg and folder.jpg files already in the same folder as the album tracks. The album plays correctly in rAudio.

ls -lrt NAS_Music
........
drwxr-xr-x 2 mpd mpd 0 Sep 10 12:41 'David Gilmour'
......
RAOffice:/mnt/MPD/NAS/NAS_Music  #
RAOffice:/mnt/MPD/NAS/NAS_Music  #  cd David\ Gilmour/
RAOffice:/mnt/MPD/NAS/NAS_Music/David Gilmour  #  ls -lrt
total 0
drwxr-xr-x 2 mpd mpd 0 Sep 10 12:43 'Luck and Strange'
RAOffice:/mnt/MPD/NAS/NAS_Music/David Gilmour  #  cd Luck\ and\ Strange/
RAOffice:/mnt/MPD/NAS/NAS_Music/David Gilmour/Luck and Strange  #  ls -lrt
total 380563
-rwxr-xr-x 1 mpd mpd 45302986 Sep 10 11:17 '02 - Luck and Strange.flac'
-rwxr-xr-x 1 mpd mpd  5454348 Sep 10 11:17 '01 - Black Cat.flac'
-rwxr-xr-x 1 mpd mpd 36380509 Sep 10 11:17 "03 - The Piper's Call.flac"
-rwxr-xr-x 1 mpd mpd  3417682 Sep 10 11:17 '05 - Vita Brevis.flac'
-rwxr-xr-x 1 mpd mpd 40516273 Sep 10 11:17 '04 - A Single Spark.flac'
-rwxr-xr-x 1 mpd mpd 36966572 Sep 10 11:17 '06 - Between Two Points.flac'
-rwxr-xr-x 1 mpd mpd 35372740 Sep 10 11:17 '08 - Sings.flac'
-rwxr-xr-x 1 mpd mpd 32880580 Sep 10 11:17 '07 - Dark and Velvet Nights.flac'
-rwxr-xr-x 1 mpd mpd 23465366 Sep 10 11:17 '10 - Yes, I Have Ghosts.flac'
-rwxr-xr-x 1 mpd mpd 46664560 Sep 10 11:17 '09 - Scattered.flac'
-rwxr-xr-x 1 mpd mpd 83135861 Sep 10 12:31 '11 - Luck and Strange (original Barn Jam.flac'
-rwxr-xr-x 1 mpd mpd   115416 Sep 10 12:32  folder.jpg
-rwxr-xr-x 1 mpd mpd   115416 Sep 10 12:32  coverart.jpg
-rwxr-xr-x 1 mpd mpd   115416 Sep 10 12:32  cover.jpg

Chris