Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

DATA RACE in the RunawayManager.metricsMap #49351

Closed
Tracked by #41316
hawkingrei opened this issue Dec 12, 2023 · 1 comment · Fixed by #49354
Closed
Tracked by #41316

DATA RACE in the RunawayManager.metricsMap #49351

hawkingrei opened this issue Dec 12, 2023 · 1 comment · Fixed by #49354
Labels
component/pd component/test severity/minor type/bug The issue is confirmed as a bug.

Comments

@hawkingrei
Copy link
Member

Bug Report

Please answer these questions before submitting your issue. Thanks!

1. Minimal reproduce step (Required)

2. What did you expect to see? (Required)

3. What did you see instead (Required)

==================
WARNING: DATA RACE
Read at 0x00c0077e9b60 by goroutine 31921:
  runtime.mapaccess2_faststr()
      GOROOT/src/runtime/map_faststr.go:108 +0x0
  github.com/pingcap/tidb/pkg/domain/resourcegroup.(*RunawayManager).DeriveChecker()
      pkg/domain/resourcegroup/runaway.go:259 +0x41a
  github.com/pingcap/tidb/pkg/executor.(*ExecStmt).Exec()
      pkg/executor/adapter.go:560 +0xda8
  github.com/pingcap/tidb/pkg/session.runStmt()
      pkg/session/session.go:2393 +0x71a
  github.com/pingcap/tidb/pkg/session.(*session).ExecuteStmt()
      pkg/session/session.go:2235 +0x1af8
  github.com/pingcap/tidb/pkg/session.(*session).ExecuteInternal()
      pkg/session/session.go:1599 +0x3b9
  github.com/pingcap/tidb/pkg/ddl/internal/session.(*Session).Execute()
      pkg/ddl/internal/session/session.go:85 +0x27e
  github.com/pingcap/tidb/pkg/ddl.(*ddl).getJob()
      pkg/ddl/job_table.go:117 +0x1bb
  github.com/pingcap/tidb/pkg/ddl.(*ddl).getGeneralJob()
      pkg/ddl/job_table.go:227 +0x84
  github.com/pingcap/tidb/pkg/ddl.(*ddl).getGeneralJob-fm()
      <autogenerated>:1 +0x3d
  github.com/pingcap/tidb/pkg/ddl.(*ddl).loadDDLJobAndRun()
      pkg/ddl/job_table.go:365 +0x408
  github.com/pingcap/tidb/pkg/ddl.(*ddl).startDispatchLoop()
      pkg/ddl/job_table.go:312 +0xdf1
  github.com/pingcap/tidb/pkg/ddl.(*ddl).startDispatchLoop-fm()
      <autogenerated>:1 +0x33
  github.com/pingcap/tidb/pkg/util.(*WaitGroupWrapper).Run.func1()
      pkg/util/wait_group_wrapper.go:157 +0x86
Previous write at 0x00c0077e9b60 by goroutine 46:
  runtime.mapassign_faststr()
      GOROOT/src/runtime/map_faststr.go:203 +0x0
  github.com/pingcap/tidb/pkg/domain/resourcegroup.(*RunawayManager).DeriveChecker()
      pkg/domain/resourcegroup/runaway.go:262 +0x591
  github.com/pingcap/tidb/pkg/executor.(*ExecStmt).Exec()
      pkg/executor/adapter.go:560 +0xda8
  github.com/pingcap/tidb/pkg/session.runStmt()
      pkg/session/session.go:2393 +0x71a
  github.com/pingcap/tidb/pkg/session.(*session).ExecuteStmt()
      pkg/session/session.go:2235 +0x1af8
  github.com/pingcap/tidb/pkg/testkit.(*TestKit).ExecWithContext()
      pkg/testkit/testkit.go:364 +0x889
  github.com/pingcap/tidb/pkg/testkit.(*TestKit).MustQueryWithContext()
      pkg/testkit/testkit.go:175 +0x17b
  github.com/pingcap/tidb/pkg/testkit.(*TestKit).MustQuery()
      pkg/testkit/testkit.go:153 +0x106
  github.com/pingcap/tidb/pkg/executor/internal/querywatch_test.TestQueryWatch()
      pkg/executor/internal/querywatch/query_watch_test.go:93 +0xcce
  github.com/pingcap/tidb/pkg/domain.(*Domain).LoadSysVarCacheLoop()
      pkg/domain/domain.go:1715 +0x93
  github.com/pingcap/tidb/pkg/session.bootstrapSessionImpl()
      pkg/session/session.go:3357 +0x724
  github.com/pingcap/tidb/pkg/domain.(*Domain).GetSessionCache()
      pkg/domain/sysvar_cache.go:62 +0x50
  github.com/pingcap/tidb/pkg/session.(*session).loadCommonGlobalVariablesIfNeeded()
      pkg/session/session.go:3729 +0xf4
  github.com/pingcap/tidb/pkg/session.(*session).ExecuteStmt()
      pkg/session/session.go:2086 +0x184
  github.com/pingcap/tidb/pkg/session.(*session).ExecuteInternal()
      pkg/session/session.go:1599 +0x3b9
  github.com/pingcap/tidb/pkg/domain.(*Domain).LoadPrivilegeLoop()
      pkg/domain/domain.go:1658 +0x108
  github.com/pingcap/tidb/pkg/session.bootstrapSessionImpl()
      pkg/session/session.go:3350 +0x6ca
  github.com/pingcap/tidb/pkg/session.BootstrapSession()
      pkg/session/session.go:3271 +0xad
  github.com/pingcap/tidb/pkg/testkit.bootstrap()
      pkg/testkit/mockstore.go:227 +0x94
  github.com/pingcap/tidb/pkg/testkit.CreateMockStoreAndDomain()
      pkg/testkit/mockstore.go:200 +0xd5
  github.com/pingcap/tidb/pkg/testkit.CreateMockStore()
      pkg/testkit/mockstore.go:68 +0x3d1
  github.com/pingcap/tidb/pkg/executor/internal/querywatch_test.TestQueryWatch()
      pkg/executor/internal/querywatch/query_watch_test.go:29 +0x46
  testing.tRunner()
      GOROOT/src/testing/testing.go:1595 +0x261
  testing.(*T).Run.func1()
      GOROOT/src/testing/testing.go:1648 +0x44
Goroutine 31921 (running) created at:
  github.com/pingcap/tidb/pkg/util.(*WaitGroupWrapper).Run()
      pkg/util/wait_group_wrapper.go:155 +0xe4
  github.com/pingcap/tidb/pkg/ddl.(*ddl).prepareWorkers4ConcurrencyDDL()
      pkg/ddl/ddl.go:745 +0x524
  github.com/pingcap/tidb/pkg/ddl.(*ddl).Start()
      pkg/ddl/ddl.go:771 +0xa64
  github.com/pingcap/tidb/pkg/domain.(*Domain).Init()
      pkg/domain/domain.go:1251 +0x1a85
  github.com/pingcap/tidb/pkg/session.(*domainMap).Get.func1()
      pkg/session/tidb.go:94 +0x6f0
  github.com/pingcap/tidb/pkg/util.RunWithRetry()
      pkg/util/misc.go:69 +0x84
  github.com/pingcap/tidb/pkg/session.(*domainMap).Get()
      pkg/session/tidb.go:80 +0x348
  github.com/pingcap/tidb/pkg/session.createSessionWithOpt()
      pkg/session/session.go:3578 +0x66
  github.com/pingcap/tidb/pkg/session.createSession()
      pkg/session/session.go:3570 +0x35
  github.com/pingcap/tidb/pkg/session.createSessionsImpl()
      pkg/session/session.go:3555 +0x81
  github.com/pingcap/tidb/pkg/session.createSessions()
      pkg/session/session.go:3542 +0x44
  github.com/pingcap/tidb/pkg/session.bootstrapSessionImpl()
      pkg/session/session.go:3317 +0x3a6
  github.com/pingcap/tidb/pkg/session.BootstrapSession()
      pkg/session/session.go:3271 +0xad
  github.com/pingcap/tidb/pkg/testkit.bootstrap()
      pkg/testkit/mockstore.go:227 +0x94
  github.com/pingcap/tidb/pkg/testkit.CreateMockStoreAndDomain()
      pkg/testkit/mockstore.go:200 +0xd5
  github.com/pingcap/tidb/pkg/testkit.CreateMockStore()
      pkg/testkit/mockstore.go:68 +0x3d1
  github.com/pingcap/tidb/pkg/executor/internal/querywatch_test.TestQueryWatch()
      pkg/executor/internal/querywatch/query_watch_test.go:29 +0x46
  testing.tRunner()
      GOROOT/src/testing/testing.go:1595 +0x261
  testing.(*T).Run.func1()
      GOROOT/src/testing/testing.go:1648 +0x44
Goroutine 46 (running) created at:
  testing.(*T).Run()
      GOROOT/src/testing/testing.go:1648 +0x845
  testing.runTests.func1()
      GOROOT/src/testing/testing.go:2054 +0x84
  testing.tRunner()
      GOROOT/src/testing/testing.go:1595 +0x261
  testing.runTests()
      GOROOT/src/testing/testing.go:2052 +0x8ad
  testing.(*M).Run()
      GOROOT/src/testing/testing.go:1925 +0xcd7
  go.uber.org/goleak.VerifyTestMain()
      external/org_uber_go_goleak/testmain.go:53 +0x64
  github.com/pingcap/tidb/pkg/executor/internal/querywatch.TestMain()
      pkg/executor/internal/querywatch/main_test.go:51 +0x996
  main.main()
      bazel-out/k8-fastbuild/bin/pkg/executor/internal/querywatch/querywatch_test_/testmain.go:123 +0x5cb
==================

4. What is your TiDB version? (Required)
@hawkingrei hawkingrei added the type/bug The issue is confirmed as a bug. label Dec 12, 2023
@hawkingrei
Copy link
Member Author

https://prow.hawkingrei.com/view/gs/prowpingcap/logs/tidb_race/1734270265501683712#1:build-log.txt%3A13043

@hawkingrei hawkingrei mentioned this issue Dec 12, 2023
Open
@glorv glorv mentioned this issue Dec 12, 2023
Merged
13 tasks
ti-chi-bot bot pushed a commit that referenced this issue Dec 12, 2023
@glorv
domain: fix data race in runaway DeriveChecker (#49354)
0b39f99 
close #49351
@ti-chi-bot ti-chi-bot mentioned this issue Dec 12, 2023
Closed
13 tasks
@glorv glorv removed the affects-7.5 label Dec 12, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
component/pd component/test severity/minor type/bug The issue is confirmed as a bug.
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

domain: fix data race in runaway DeriveChecker glorv/tidb
3 participants
@hawkingrei @jebter @glorv