Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[420 FLOOD_WAIT_X] would cause photos and videos which size is 0KB #495

Open
porridgexj opened this issue Jun 21, 2024 · 7 comments
Open

Comments

@porridgexj
Copy link

When files in the channel are too many, and will cause the ERROR below:

ERROR Telegram says: [420 FLOOD_WAIT_X] - A wait of 2919 seconds is required (caused by client.py:1021
"auth.ExportAuthorization")

and that generate many videos and photos which is 0KB, and the program will move on. It will cause many photos and videos are skipped
屏幕截图 2024-06-21 205416
, and the program will never redownload them again.

@porridgexj
Copy link
Author

porridgexj commented Jun 22, 2024

I think it is a bug from pyrogram, we just can't catch the FLOOD error from app.download_media function. So I find another way to solve this, when show a FloodWait error, the file downloaded size usually is 0KB, so we can test whether the file size is 0KB.

def is_file_empty(file_path):
    if os.path.exists(file_path):
        return os.path.getsize(file_path) == 0
    else:
        return True

flag = True
while flag:
    # 'download_media' is a function from 'pyrogram'
    file_path = app.download_media(file_id)
    if file_path and not is_file_empty(file_path):  
        flag = False
        # solve success situation
    else:
        time.sleep(10)
        # sleep for several seconds and retry

@AIRDOGE
Copy link

AIRDOGE commented Jul 15, 2024

the same problem

@AIRDOGE
Copy link

AIRDOGE commented Jul 15, 2024

I think it is a bug from pyrogram, we just can't catch the FLOOD error from app.download_media function. So I find another way to solve this, when show a FloodWait error, the file downloaded size usually is 0KB, so we can test whether the file size is 0KB.

def is_file_empty(file_path):
    if os.path.exists(file_path):
        return os.path.getsize(file_path) == 0
    else:
        return True

flag = True
while flag:
    # 'download_media' is a function from 'pyrogram'
    file_path = app.download_media(file_id)
    if file_path and not is_file_empty(file_path):  
        flag = False
        # solve success situation
    else:
        time.sleep(10)
        # sleep for several seconds and retry

Where should I need to insert this code?

@AIRDOGE
Copy link

AIRDOGE commented Jul 15, 2024

Is there any way to make the program automatically wait for the time required by telegram and than restarting the download?

@porridgexj
Copy link
Author

Is there any way to make the program automatically wait for the time required by telegram and than restarting the download?

We can’t get the seconds that telegram required, because we just can’t catch the FLOOD error info from pyrogram. If you want to make the program wait for the time required by telegram exactly, I think you can let time.sleep from 10 to 1…but i think it will put pressure on telegram’s server, i think check every 30 seconds is ok

by the way, I didn’t try to insert my code to TELEGRAM_MEIDA_DOWNLOADER, because I just want to download all pics from a channel, it is easy to achieve By just using PYROGRAM, so I wrote my own code, and not use TELEGRAM_MEDIA_DOWNLOADER

@gauravsuman007
Copy link

gauravsuman007 commented Aug 30, 2024

I just used the code from @porridgexj and put it in the media_downloader.py. I first defined a new function:

def is_file_undownloaded_or_empty(file_path):
    if os.path.exists(file_path):
    	logger.info("%s already exists, checking size", file_path)	
    	return os.path.getsize(file_path) == 0
    else:
        return True

Then I modified the download code (note that the commented lines are the old code. Also moved the Media downloaded logger info to inside the loop):

                file_name, file_format = await _get_media_meta(_media, _type)
                if _can_download(_type, file_formats, file_format):
                    #if _is_exist(file_name):
                    #    file_name = get_next_name(file_name)
                    #    download_path = await client.download_media(
                    #        message, file_name=file_name
                    #    )
                    #    # pylint: disable = C0301
                    #    download_path = manage_duplicate_file(download_path)  # type: ignore
                    #else:
                    #    download_path = await client.download_media(
                    #        message, file_name=file_name
                    #    )
                    if is_file_undownloaded_or_empty(file_name):
                        download_path = await client.download_media(
                            message, file_name=file_name
                        )
                        if download_path:
                            logger.info("Media downloaded - %s", download_path)
                    else:
                        logger.info("Skipping download of %s", file_name)
		    	
                    DOWNLOADED_IDS.append(message.id)

Now I'm able to resume downloads. With these changes I have disabled the functionality to redownload existing files by appending a number to (see the commented part in the code) it but I think that doesn't really matter as only completely downloaded files get renamed from FILENAME.temp to FILENAME and so we don't need any redownloads.

The final thing I always have to do before relaunching the script is to reset last_read_message_id to 0 as otherwise, downloads don't start for me (weird bug)

@ZoriuqAmsedel
Copy link

This is working for videos but not for photos, it says it already exist but the path is not showing the file name

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants