Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

airlock request failed due to an unknown reason #3705

Closed
basitrehman531997 opened this issue Sep 6, 2023 · 38 comments
Closed

airlock request failed due to an unknown reason #3705

basitrehman531997 opened this issue Sep 6, 2023 · 38 comments
Labels
bug Something isn't working

Comments

@basitrehman531997
Copy link

basitrehman531997 commented Sep 6, 2023

The airlock request encountered an issue and failed after a while of being submitted. Another factor contributing to this problem is that I received an authentication error when executing a CLI command. Additionally, when I attempted to upload a file via the portal, the imported viewer virtual machine displayed an empty folder.

i used base workspace

v 0.14.0

airlock failed
@basitrehman531997 basitrehman531997 added the bug Something isn't working label Sep 6, 2023
@basitrehman531997
Copy link
Author

also awaiting my review section is also empty

@basitrehman531997
Copy link
Author

awaiting my review

@basitrehman531997
Copy link
Author

cliscreenshort

@marrobi
Copy link
Member

marrobi commented Sep 7, 2023

Hi @basitrehman531997 lets deal with the initial issue first.

Have you enabled malware scanning in the config.yaml - we are aware of a potential issue there?

It is could be an issue creating the storage container, or with the function that creates it.

Can you check the airlock function app logs - it is named func-airlock-processor-....

Also can you confirm the TRE version you are using.

Thanks.

@brehmanepicsoft
Copy link

brehmanepicsoft commented Sep 7, 2023

hi @marrobi
Yes, we have implemented malware scanning in the config.yaml file
enable_airlock_malware_scanning: true

but in local.setting.json-sample is
"ENABLE_MALWARE_SCANNING": "false"

@marrobi
Copy link
Member

marrobi commented Sep 7, 2023

Can you try setting it to false, running make tre-deploy, maybe ensure the property has updated in the function app - so will be set to false.

And see if the issue still exists?

@brehmanepicsoft
Copy link

brehmanepicsoft commented Sep 7, 2023

Azure TRE
UI Version:0.5.4
API Version:0.15.9
Cosmos DB:OK
Service Bus:OK
Resource Processor:Ok

is the version ?can tell me how to check the current version of running tre

@brehmanepicsoft
Copy link

@marrobi
I used the command "enable-malware-scanning=false" as you instructed. But when I tried to use the app, I got this error: "We couldn't check if your function app is running, which is needed for log streaming. Please make sure your function host is running."

@marrobi
Copy link
Member

marrobi commented Sep 7, 2023

Yes, that is fine. Are you running from main in this repo or the Deployment repository?

Does the airlock work? If not please send a screenshot of the function app in the portal and verify the malware scanning setting value in the function app configuration?

@brehmanepicsoft
Copy link

brehmanepicsoft commented Sep 7, 2023

@marrobi
yes airlock not working properly give 500 error when i create request for import and the function app status is ruuning

@marrobi
Copy link
Member

marrobi commented Sep 7, 2023

So you no longer get the unknown error? Are there any details with the 500 error?

Also worth checking the Application Insights for any 500 errors logged.

@brehmanepicsoft
Copy link

@marrobi
Copy link
Member

marrobi commented Sep 7, 2023

Ok, you need to check app insights or the function logs as I am not sure what is causing the issue.

Can you double check the review workspace settings are configured correctly.

@brehmanepicsoft
Copy link

brehmanepicsoft commented Sep 7, 2023

this is the activity log of function app

@marrobi
Copy link
Member

marrobi commented Sep 7, 2023

They look to be from a while ago? If you drill down further do you get any more info?

Anything on the front page of the function app overview - can you send a snip of that please?

@brehmanepicsoft
Copy link

brehmanepicsoft commented Sep 7, 2023

oky this is

in application insight section in failures we have fail count include response code 404,and 403

@marrobi
Copy link
Member

marrobi commented Sep 7, 2023

Hmm, there is no runtime version.

@martinpeck is this not similar to what you saw, but was fixed in release v0.14.0?

@brehmanepicsoft under Deployment Center what version is the container image? Thanks.

@brehmanepicsoft
Copy link

brehmanepicsoft commented Sep 7, 2023

@marrobi how to know the version of the container image ?

@marrobi
Copy link
Member

marrobi commented Sep 7, 2023

Go to Deployment Centre in the left menu of the function app.

@brehmanepicsoft
Copy link

brehmanepicsoft commented Sep 7, 2023

in setting of deployment section i seen tag which is 0.6.1

@brehmanepicsoft
Copy link

@marrobi and the other question is that how to know tre version in UI or in devcontainer

@martinpeck
Copy link
Member

That's certainly the behaviour that I'd hoped had been addressed.
Can you please review the firewall application rules that you have?
In your Azure deployment, within the resource group, you should have a Firewall Policy
In that, there are Application Rules
There should be an application rule called functions-runtime that looks something like this (below) and should be allow-listing the functionscdn.azureedge.net endpoint. Can you confirm this is present?

image

@brehmanepicsoft
Copy link

brehmanepicsoft commented Sep 7, 2023

@martinpeck yes this is present in tre firewall.thanks

can i make again enable_airlock_malware_scanning: ture

@marrobi
Copy link
Member

marrobi commented Sep 7, 2023

@brehmanepicsoft leave that off for now, I think there may be two separate issues.

Might be worth checking the firewall logs to see what traffic is being denied form the airlock processor subnet.

@brehmanepicsoft
Copy link

brehmanepicsoft commented Sep 7, 2023

activity log of tre firewall
message : Gets IP prefixes learned by Azure Firewall to not perform SNAT

logs of tre firewall:
HTTPS request from ip_address:port to pypi.org:443. Action: Deny. No rule matched. Proceeding with default action

@martinpeck
Copy link
Member

martinpeck commented Sep 7, 2023

OK, so that issue is different. It shouldn't be preventing the airlock processor from starting up (but...maybe is is? Maybe the airlock processor can't access pypi to pull packages as it starts up?)

0.14.1 fixes this. This is the only fix/change contained in 0.14.1 so can I suggest you upgrade to this and see if that fixes the airlock?

@martinpeck
Copy link
Member

FYI this is the bug that was fixed in 0.14.1...

#3694

@martinpeck
Copy link
Member

If you want, you can manually apply the fix that 0.14.1 includes. If you look at my screenshot of the firewall rules above you'll see that there are two pypi.org related exceptions in the very last rule. You want to allow both pypi.org and *.pypi.org and I suspect you only have the latter,.

You can manually add pypi.org into that rule and see if that fixes things.

0.14.1 applies this fix for you.

@martinpeck
Copy link
Member

Red arrow shows what you want...

image

@brehmanepicsoft
Copy link

I attempted to address the problem, but the issue persists.

@brehmanepicsoft
Copy link

When I send a request to the airlock system, it typically produces a container for file uploads. However, the current situation is that the request no longer results in the creation of a container. Furthermore, when I check the Azure portal, I notice that there is no container defined in the request's SAS URL.

@marrobi
Copy link
Member

marrobi commented Sep 8, 2023

Does the function have a runtime in the portal? That the issue we need to resolve. For some reason the function app isn't starting correctly.

If no runtime, can you try deleting the function and its service plan and try make tre-deploy. Thanks.

@brehmanepicsoft
Copy link

when i access file-system logs of function give me message
Unable to fetch the host status of your function app. To use log streaming, please make sure your function host is running.

@brehmanepicsoft
Copy link

i also update version but in firewall you address me the issue of pypi.org is not added automatically.
i add manually

@brehmanepicsoft
Copy link

brehmanepicsoft commented Sep 8, 2023

@marrobi
there is no version time show only --- in function Overview

@martinpeck
Copy link
Member

martinpeck commented Sep 8, 2023 via email

@marrobi
Copy link
Member

marrobi commented Oct 26, 2023

Looks like is an issue with import requests in certain circumstances:

#3767

@SvenAelterman SvenAelterman changed the title airlock request faield due to an unknow reason airlock request failed due to an unknow reason Oct 26, 2023
@SvenAelterman SvenAelterman changed the title airlock request failed due to an unknow reason airlock request failed due to an unknown reason Oct 26, 2023
@marrobi
Copy link
Member

marrobi commented Dec 7, 2023

Closing as per #3767

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

4 participants