Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Expo returns Bad Gateway 504 errors on sendPushNotification #680

Open
10 tasks
mkraenz opened this issue Sep 7, 2022 · 1 comment · Fixed by mkraenz/you-are-awesome-app-backend#6
Open
10 tasks

Expo returns Bad Gateway 504 errors on sendPushNotification #680

mkraenz opened this issue Sep 7, 2022 · 1 comment · Fixed by mkraenz/you-are-awesome-app-backend#6
Assignees
@mkraenz
Labels
bug Something isn't working

Comments

@mkraenz
Copy link
Owner

mkraenz commented Sep 7, 2022 •
edited
Loading

Observed

12:02:00 pm
START RequestId: 11b8007e-085b-4a84-985d-4d509c2bc53b Version: $LATEST
12:02:00 pm
2022-09-06T10:02:00.409Z 11b8007e-085b-4a84-985d-4d509c2bc53b INFO { msg: 'found subs to send', subs: 5, time: '10:00' }
12:02:00 pm
2022-09-06T10:02:00.500Z 11b8007e-085b-4a84-985d-4d509c2bc53b INFO { msg: 'sending chunk', chunk: 1, maxChunk: 1, time: '10:00', allNotifications: 5 }
12:02:05 pm
2022-09-06T10:02:05.853Z 11b8007e-085b-4a84-985d-4d509c2bc53b ERROR Error: Expo responded with an error with status code 504: <html> <head><title>504 Gateway Time-out</title></head> <body> <center><h1>504 Gateway Time-out</h1></center> <hr><center>nginx</center> </body> </html> at Expo.<anonymous> (/var/task/node_modules/expo-server-sdk/build/ExpoClient.js:234:30) at Generator.next (<anonymous>) at /var/task/node_modules/expo-server-sdk/build/ExpoClient.js:8:71 at new Promise (<anonymous>) at __awaiter (/var/task/node_modules/expo-server-sdk/build/ExpoClient.js:4:12) at Expo.getTextResponseErrorAsync (/var/task/node_modules/expo-server-sdk/build/ExpoClient.js:233:16) at Expo.<anonymous> (/var/task/node_modules/expo-server-sdk/build/ExpoClient.js:222:35) at Generator.next (<anonymous>) at fulfilled (/var/task/node_modules/expo-server-sdk/build/ExpoClient.js:5:58) at processTicksAndRejections (internal/process/task_queues.js:97:5) { statusCode: 504, errorText: '<html>\r\n' + '<head><title>504 Gateway Time-out</title></head>\r\n' + '<body>\r\n' + '<center><h1>504 Gateway Time-out</h1></center>\r\n' + '<hr><center>nginx</center>\r\n' + '</body>\r\n' + '</html>\r\n' }
12:02:05 pm
2022-09-06T10:02:05.853Z 11b8007e-085b-4a84-985d-4d509c2bc53b INFO
12:02:05 pm
2022-09-06T10:02:05.853Z 11b8007e-085b-4a84-985d-4d509c2bc53b ERROR Error: Expo responded with an error with status code 504: <html> <head><title>504 Gateway Time-out</title></head> <body> <center><h1>504 Gateway Time-out</h1></center> <hr><center>nginx</center> </body> </html> at Expo.<anonymous> (/var/task/node_modules/expo-server-sdk/build/ExpoClient.js:234:30) at Generator.next (<anonymous>) at /var/task/node_modules/expo-server-sdk/build/ExpoClient.js:8:71 at new Promise (<anonymous>) at __awaiter (/var/task/node_modules/expo-server-sdk/build/ExpoClient.js:4:12) at Expo.getTextResponseErrorAsync (/var/task/node_modules/expo-server-sdk/build/ExpoClient.js:233:16) at Expo.<anonymous> (/var/task/node_modules/expo-server-sdk/build/ExpoClient.js:222:35) at Generator.next (<anonymous>) at fulfilled (/var/task/node_modules/expo-server-sdk/build/ExpoClient.js:5:58) at processTicksAndRejections (internal/process/task_queues.js:97:5) { statusCode: 504, errorText: '<html>\r\n' + '<head><title>504 Gateway Time-out</title></head>\r\n' + '<body>\r\n' + '<center><h1>504 Gateway Time-out</h1></center>\r\n' + '<hr><center>nginx</center>\r\n' + '</body>\r\n' + '</html>\r\n' }
12:02:05 pm
2022-09-06T10:02:05.855Z 11b8007e-085b-4a84-985d-4d509c2bc53b DEBUG error while getting callsite source context: ENOENT: no such file or directory, open '/var/task/node_modules/expo-server-sdk/src/ExpoClient.ts'
12:02:05 pm
2022-09-06T10:02:05.855Z 11b8007e-085b-4a84-985d-4d509c2bc53b DEBUG error while getting callsite source context: ENOENT: no such file or directory, open '/var/task/node_modules/expo-server-sdk/src/ExpoClient.ts'
12:02:05 pm
2022-09-06T10:02:05.859Z 11b8007e-085b-4a84-985d-4d509c2bc53b ERROR Invoke Error { "errorType": "Error", "errorMessage": "Expo responded with an error with status code 504: <html>\r\n<head><title>504 Gateway Time-out</title></head>\r\n<body>\r\n<center><h1>504 Gateway Time-out</h1></center>\r\n<hr><center>nginx</center>\r\n</body>\r\n</html>\r\n", "statusCode": 504, "errorText": "<html>\r\n<head><title>504 Gateway Time-out</title></head>\r\n<body>\r\n<center><h1>504 Gateway Time-out</h1></center>\r\n<hr><center>nginx</center>\r\n</body>\r\n</html>\r\n", "stack": [ "Error: Expo responded with an error with status code 504: <html>\r", "<head><title>504 Gateway Time-out</title></head>\r", "<body>\r", "<center><h1>504 Gateway Time-out</h1></center>\r", "<hr><center>nginx</center>\r", "</body>\r", "</html>\r", "", " at Expo.<anonymous> (/var/task/node_modules/expo-server-sdk/build/ExpoClient.js:234:30)", " at Generator.next (<anonymous>)", " at /var/task/node_modules/expo-server-sdk/build/ExpoClient.js:8:71", " at new Promise (<anonymous>)", " at __awaiter (/var/task/node_modules/expo-server-sdk/build/ExpoClient.js:4:12)", " at Expo.getTextResponseErrorAsync (/var/task/node_modules/expo-server-sdk/build/ExpoClient.js:233:16)", " at Expo.<anonymous> (/var/task/node_modules/expo-server-sdk/build/ExpoClient.js:222:35)", " at Generator.next (<anonymous>)", " at fulfilled (/var/task/node_modules/expo-server-sdk/build/ExpoClient.js:5:58)", " at processTicksAndRejections (internal/process/task_queues.js:97:5)" ] }
12:02:05 pm
END RequestId: 11b8007e-085b-4a84-985d-4d509c2bc53b
12:02:05 pm
REPORT RequestId: 11b8007e-085b-4a84-985d-4d509c2bc53b Duration: 5478.06 ms Billed Duration: 5479 ms Memory Size: 1024 MB Max Memory Used: 114 MB

Expected

Since after lots of research we have not found a fix for this, ignore these errors and continue sending outstanding push notifications.

DoD

  • Post-mortem identified the issue, possible root cause, and defined actions to mitigate similar issues in the future
  • Post-mortem actions are being worked on with highest priority
  • merged into master
  • emulator app on master still starts
  • emulator app on master contains the update
  • yarn deploy:ota:prod succeeded
  • production app on my phone still starts
  • production app on my phone contains the update
  • production app on my phone tests the changed functionality ad-hoc
  • merged update to build version to master
@mkraenz mkraenz added the bug Something isn't working label Sep 7, 2022
@mkraenz mkraenz self-assigned this Sep 7, 2022
@mkraenz mkraenz mentioned this issue Sep 7, 2022
Merged
@mkraenz
Copy link
Owner Author

mkraenz commented Sep 7, 2022 •
edited
Loading

deployed. mkraenz/you-are-awesome-app-backend#6

Will observe for a few days to see whether issues have been resolved.

@mkraenz mkraenz reopened this Sep 7, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@mkraenz mkraenz

Labels
bug Something isn't working
Projects
None yet
Milestone
No milestone
1 participant
@mkraenz