NeroReflex/mesa
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity

radv/ci: only reboot on hangs for vkcts-navi10-valve

Browse source 
vkcts-navi10-valve has the nasty habit on hanging the GPU, so we
introduced an auto-retry... but for every radv job. Let's stop doing
that, and instead limit the auto-retry to vkcts-navi10-valve only.

Additionally, let's increase the number of attempts to 3 (2 retries),
as sometimes, it may still fail and we don't want to flag it as a
fail in nightly runs.

Let's hope we'll get to the bottom of this hang sooner rather than
later, so that we can remove this hack!

Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Signed-off-by: Martin Roukala (né Peres) <martin.roukala@mupuf.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22596>
This commit is contained in:
Martin Roukala (né Peres) 2023-04-20 15:21:02 +03:00 • committed by Marge Bot
parent a65fe883f4
commit 7af6616030
2 changed files with 6 additions and 2 deletions
Download patch file Download diff file Expand all files Collapse all files

4
.gitlab-ci/test/gitlab-ci.yml
View file

@ -248,9 +248,9 @@ clang-format:
B2C_LOG_LEVEL: 6
B2C_POWEROFF_DELAY: 15
B2C_SESSION_END_REGEX: '^.*It''s now safe to turn off your computer\r$'
B2C_SESSION_REBOOT_REGEX: '(GPU hang detected!|\*ERROR\* ring [^\s]+ timeout(?!, but soft recovered)|The CS has been cancelled because the context is lost)'
B2C_SESSION_REBOOT_REGEX: ''
B2C_TIMEOUT_BOOT_MINUTES: 45
B2C_TIMEOUT_BOOT_RETRIES: 1
B2C_TIMEOUT_BOOT_RETRIES: 0
B2C_TIMEOUT_FIRST_MINUTES: 5
B2C_TIMEOUT_FIRST_RETRIES: 3
B2C_TIMEOUT_MINUTES: 4

4
src/amd/ci/gitlab-ci.yml
View file

@ -355,6 +355,10 @@ vkcts-navi10-valve:
# Use Linux 5.17 to work around hitting __vm_enough_memory then hanging on Linux 6.1
# See https://gitlab.freedesktop.org/mesa/mesa/-/issues/7888 for details
B2C_KERNEL_URL: 'https://gitlab.freedesktop.org/mupuf/valve-infra/-/package_files/144/download' # 5.17.1
# NAVI10 hangs quite often while running vkcts. This is being investigated,
# but in the mean time, retry up to 2 times.
B2C_SESSION_REBOOT_REGEX: '(GPU hang detected!|\*ERROR\* ring [^\s]+ timeout(?!, but soft recovered)|The CS has been cancelled because the context is lost)'
B2C_TIMEOUT_BOOT_RETRIES: 2
vkcts-navi21-valve:
parallel: 3