datalakefileclient.upload_data without overwrite

[kasuteru]

It seems that I cannot use upload_data without setting the overwrite flag to true. Maybe I am doing something wrong:

file1 = my_file_system_client.get_file_client("path/to/NONEXISTING_file.txt")
file1.upload_data(data) # Trying to upload to a non-existing file
# Raises: azure.core.exceptions.ResourceNotFoundError: (PathNotFound) The specified path does not exist.

file1 = my_file_system_client.get_file_client("path/to/NONEXISTING_file.txt")
file1.upload_data(data, overwrite=True) # Trying to upload to a non-existing file, with overwrite
# Works!

file1 = my_file_system_client.get_file_client("path/to/existing.txt")
file1.upload_data(data) # Trying to upload to a existing file, without overwrite
# Raises: azure.storage.filedatalake._generated.models._models_py3.StorageErrorException: Operation returned an invalid status 'The condition specified using HTTP conditional header(s) is not met.'

file1 = my_file_system_client.get_file_client("path/to/existing.txt")
file1.upload_data(data, overwrite=True) # Trying to upload to a existing file, with overwrite
# Works!
 

If this is the case for everyone, I am questioning the use of the overwrite flag... or at least its default value of False?

[ghost]

Thanks for the feedback! We are routing this to the appropriate team for follow-up. cc @sumantmehtams.

Issue Details

---

It seems that I cannot use upload_data without setting the overwrite flag to true. Maybe I am doing something wrong:

file1 = my_file_system_client.get_file_client("path/to/NONEXISTING_file.txt")
file1.upload_data(data) # Trying to upload to a non-existing file
# Raises: azure.core.exceptions.ResourceNotFoundError: (PathNotFound) The specified path does not exist.

file1 = my_file_system_client.get_file_client("path/to/NONEXISTING_file.txt")
file1.upload_data(data, overwrite=True) # Trying to upload to a non-existing file, with overwrite
# Works!

file1 = my_file_system_client.get_file_client("path/to/existing.txt")
file1.upload_data(data) # Trying to upload to a existing file, without overwrite
# Raises: azure.storage.filedatalake._generated.models._models_py3.StorageErrorException: Operation returned an invalid status 'The condition specified using HTTP conditional header(s) is not met.'

file1 = my_file_system_client.get_file_client("path/to/existing.txt")
file1.upload_data(data, overwrite=True) # Trying to upload to a existing file, with overwrite
# Works!
 

If this is the case for everyone, I am questioning the use of the overwrite flag... or at least its default value of False?

Author:	kasuteru
Assignees:	xiafu-msft
Labels:	bug, question, Service Attention, Client, customer-reported, Data Lake Storage Gen2
Milestone:	Backlog

[mahanoor786]

Can someone fix this issue or suggest a workaround? I am also running into same issue with overwrite Flag set to False.

[navba-MSFT]

@kasuteru Apologies for the delayed response. Please refer this article which talks about the overwrite=False being the default parameter value for the upload_data function.

Also if you would like to perform the Append operation you can refer the append_data method. Sample code is here.

I see that this github thread is open for quite sometime now. So wanted to check if you are still looking for assistance on this issue ? Please let us know.

[ghost]

Hi, we're sending this friendly reminder because we haven't heard back from you in a while. We need more information about this issue to help address it. Please be sure to give us your input within the next 7 days. If we don't hear back from you within 14 days of this comment the issue will be automatically closed. Thank you!

[HK-Zhang]

I get the same issue with overwrite Flag set to False.

azure-storage-blob==12.9.0
azure-storage-file-datalake==12.5.0

[isaacnorman82]

is anything happening with this issue? I just ran into it on trying to use azure.storage.filedatalake.

@navba-MSFT I don't see how the documentation you linked to helps in any way. Yes there is a parameter called overwrite, the problem is the API seems to fail if that parameter isn't set to True, even if you're not overwriting. Also none of the errors thrown are suitable for the situation - it should be something that clearly states the cause of the error is trying to overwrite a file that already exists with overwrite set to false (which is not the case here anyway).

[jinlow]

Also, experiencing the exact same issue still with the DataLakeFileClient class.