Skip to content

Commit

Permalink
Fixes Byte Order Mark in Downloaded file list, again. (#1137)
Browse files Browse the repository at this point in the history
codecs.open is stateless, so simple way doesn't work and we need to deal
with state, or we get a BOM on each newline. This is suboptimal and forces
more postprocessing of Downloaded file list for correct result.

All of this code could be deleted if irfan fixes it's unicode file bug.
Alas, bug persists in all text files irfan opens, including it's config.
So fix is unlikely. :^(

Note to self: test not only on one line, but on multiple lines.
  • Loading branch information
NHOrus authored Jul 22, 2022
1 parent 5c78fb1 commit c1992f9
Showing 1 changed file with 8 additions and 0 deletions.
8 changes: 8 additions & 0 deletions PixivDownloadHandler.py
Original file line number Diff line number Diff line change
Expand Up @@ -245,6 +245,14 @@ def download_image(caller,
raise
else:
PixivHelper.print_and_log('info', ' done.')

# codecs.open is stateless, so if platform_encoding == utf-8-sig each new line starts from utf-8-sig
# this is bad and I feel bad

if os.path.isfile(caller.dfilename):
dfile_encoding = 'utf-8'
else:
dfile_encoding = caller.platform_encoding

# write to downloaded lists
if caller.start_iv or config.createDownloadLists:
Expand Down

0 comments on commit c1992f9

Please sign in to comment.