Skip to content

Commit

Permalink
[warm/fast-reboot] Backup logs from tmpfs to disk during fast/warm sh…
Browse files Browse the repository at this point in the history
…utdown (#2714)

Goal: Preserve logs during TOR upgrades and shutdown

Need:

Below PRs moved logs from disk to tmpfs for specific hwskus.
Due to these changes, shutdown path logs are now lost.
The logs in shutdown path are crucial for debug purposes.

sonic-net/sonic-buildimage#13805
sonic-net/sonic-buildimage#13587
sonic-net/sonic-buildimage#13587

How I did it
Check if logs are on tmpfs. If yes, backup logs from /var/log

How to verify it
Verified on a physical device - logs on tmfs are backed up for past 30 minutes.
  • Loading branch information
vaibhavhd authored and qiluo-msft committed Mar 20, 2023
1 parent 93c7d43 commit cc78747
Showing 1 changed file with 11 additions and 0 deletions.
11 changes: 11 additions & 0 deletions scripts/fast-reboot
Original file line number Diff line number Diff line change
Expand Up @@ -752,6 +752,17 @@ fi
# Reboot: explicitly call Linux native reboot under sbin
debug "Rebooting with ${REBOOT_METHOD} to ${NEXT_SONIC_IMAGE} ..."
LOGS_ON_TMPFS=0
df --output=fstype /var/log* | grep -c 'tmpfs' || LOGS_ON_TMPFS=$?
if [[ LOGS_ON_TMPFS -eq 0 ]]; then
debug "Backup shutdown logs to /host/logs_before_reboot"
mkdir -p /host/logs_before_reboot || /bin/true
# maxdepth 2: find files within 2 nested directories (eg. /var/log/ and /var/log/swss/)
# mmin 30: find files written in past 30 minutes
find /var/log -maxdepth 2 -mmin -30 -type f | xargs -I {} cp {} /host/logs_before_reboot/ || /bin/true
fi
exec ${REBOOT_METHOD}
# Should never reach here
Expand Down

0 comments on commit cc78747

Please sign in to comment.