Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

DEVPROD-11847 remove create version route #8368

Merged
merged 5 commits into from
Oct 6, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 0 additions & 1 deletion cmd/evergreen/evergreen.go
Original file line number Diff line number Diff line change
Expand Up @@ -70,7 +70,6 @@ func buildApp() *cli.App {
operations.PatchRemoveModule(),
operations.PatchFinalize(),
operations.PatchCancel(),
operations.CreateVersion(),
}

userHome, err := homedir.Dir()
Expand Down
2 changes: 1 addition & 1 deletion config.go
Original file line number Diff line number Diff line change
Expand Up @@ -34,7 +34,7 @@ var (

// ClientVersion is the commandline version string used to control updating
// the CLI. The format is the calendar date (YYYY-MM-DD).
ClientVersion = "2024-09-27"
ClientVersion = "2024-10-04"

// Agent version to control agent rollover. The format is the calendar date
// (YYYY-MM-DD).
Expand Down
8 changes: 4 additions & 4 deletions docs/Project-Configuration/Evergreen-Data-for-Analytics.md
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,7 @@ Daily aggregated statistics for task executions run in Evergreen. Tasks are aggr
| project\_id | VARCHAR | Unique project identifier.
| variant | VARCHAR | Name of the build variant on which the tasks ran.
| task\_name | VARCHAR | Display name of the tasks.
| request\_type | VARCHAR | Name of the trigger that requested the task executions. Will always be one of: `patch_request`, `github_pull_request`, `gitter_request` (mainline), `trigger_request`, `github_merge_request` (GitHub merge queue), `merge_test` (Evergreen commit queue, deprecated), `create_version_endpoint` (REST API) or `ad_hoc` (periodic build).
| request\_type | VARCHAR | Name of the trigger that requested the task executions. Will always be one of: `patch_request`, `github_pull_request`, `gitter_request` (mainline), `trigger_request`, `github_merge_request` (GitHub merge queue), `merge_test` (Evergreen commit queue, deprecated), or `ad_hoc` (periodic build).
| finish\_date | VARCHAR | Date, in ISO format `YYYY-MM-DD`, on which the tasks ran.
| num\_success | BIGINT | Number of successful task executions in the group.
| num\_failed | BIGINT | Number of failed task executions in the group.
Expand All @@ -53,7 +53,7 @@ Daily aggregated statistics for test executions run in Evergreen. Test stats are
| variant | VARCHAR | Name of the build variant on which the tests ran.
| task\_name | VARCHAR | Name of the task that the test ran under. This is the display task name for tasks that are part of a display task.
| test\_name | VARCHAR | Display name of the tests.
| request\_type | VARCHAR | Name of the trigger that requested the task execution. Will always be one of: `patch_request`, `github_pull_request`, `gitter_request` (mainline), `trigger_request`, `github_merge_request` (GitHub merge queue), `merge_test` (Evergreen commit queue, deprecated), `create_version_endpoint` (REST API) or `ad_hoc` (periodic build).
| request\_type | VARCHAR | Name of the trigger that requested the task execution. Will always be one of: `patch_request`, `github_pull_request`, `gitter_request` (mainline), `trigger_request`, `github_merge_request` (GitHub merge queue), `merge_test` (Evergreen commit queue, deprecated), or `ad_hoc` (periodic build).
| num\_pass | BIGINT | Number of passing tests.
| num\_fail | BIGINT | Number of failing tests.
| num\_status\_swaps | BIGINT | Number of test status changes between subsequent runs. Only available for tests that ran after 2024-09-15.
Expand All @@ -78,7 +78,7 @@ Finished Evergreen tasks, partitioned by the project and date (in ISO format). F
| build\_id | VARCHAR | ID of the build containing the task.
| order | BIGINT | Order number of the task's version. For patches this is the user's current patch submission count. For mainline versions this is the number of versions for that repository so far.
| revision | VARCHAR | Git commit SHA.
| requester | VARCHAR | Name of the trigger that requested the task execution. Will always be one of: `patch_request`, `github_pull_request`, `gitter_request` (mainline), `trigger_request`, `github_merge_request` (GitHub merge queue), `merge_test` (Evergreen commit queue, deprecated), `create_version_endpoint` (REST API) or `ad_hoc` (periodic build).
| requester | VARCHAR | Name of the trigger that requested the task execution. Will always be one of: `patch_request`, `github_pull_request`, `gitter_request` (mainline), `trigger_request`, `github_merge_request` (GitHub merge queue), `merge_test` (Evergreen commit queue, deprecated), or `ad_hoc` (periodic build).
| tags | ARRAY(VARCHAR) | Array of task tags from project configuration.
| priority | BIGINT | Scheduling priority of the task.
| task\_group | VARCHAR | Name of the task group that contains this task.
Expand Down Expand Up @@ -174,7 +174,7 @@ Note that this is an estimated cost based solely on compute (`BoxUsage`) usage p
| display\_name | VARCHAR | Display name of the task, will be the parent task's display name if part of a display task.
| build\_variant | VARCHAR | Name of the task's build variant.
| build\_variant\_display\_name | VARCHAR | Display name of the task's build variant.
| requester | VARCHAR | Name of the trigger that requested the task execution. Will always be one of: `patch_request`, `github_pull_request`, `gitter_request` (mainline), `trigger_request`, `github_merge_request` (GitHub merge queue), `merge_test` (Evergreen commit queue, deprecated), `create_version_endpoint` (REST API) or `ad_hoc` (periodic build).
| requester | VARCHAR | Name of the trigger that requested the task execution. Will always be one of: `patch_request`, `github_pull_request`, `gitter_request` (mainline), `trigger_request`, `github_merge_request` (GitHub merge queue), `merge_test` (Evergreen commit queue, deprecated), or `ad_hoc` (periodic build).
| activated\_by | VARCHAR | User or service that activated this task.
| tags | ARRAY(VARCHAR) | Array of task tags from project configuration.
| host\_id | VARCHAR | ID of the host that ran this task.
Expand Down
3 changes: 1 addition & 2 deletions docs/Project-Configuration/Project-Configuration-Files.md
Original file line number Diff line number Diff line change
Expand Up @@ -634,7 +634,6 @@ The valid requester values are:
- `commit`: mainline commits.
- `trigger`: downstream trigger versions.
- `ad_hoc`: periodic build versions.
- `create_version_endpoint`: versions created via API.
- `github_merge_queue`: GitHub's merge queue.

By default, if no `allowed_requesters` are explicitly specified, then a task can
Expand Down Expand Up @@ -799,7 +798,7 @@ Every task has some expansions available by default:
- `${is_commit_queue}` is the string "true" if this is a merge
queue task
- `${requester}` is what triggered the task: `patch`, `github_pr`,
`github_tag`, `commit`, `trigger`, `github_merge_queue`, `create_version_endpoint` or `ad_hoc`
`github_tag`, `commit`, `trigger`, `github_merge_queue`, or `ad_hoc`
- `${otel_collector_endpoint}` is the gRPC endpoint for Evergreen's
OTel collector. Tasks can send traces to this endpoint.
- `${otel_trace_id}` is the OTel trace ID this task is running under.
Expand Down
45 changes: 19 additions & 26 deletions globals.go
Original file line number Diff line number Diff line change
Expand Up @@ -642,25 +642,25 @@ const (
DefaultAmboyDatabaseURL = "mongodb://localhost:27017"

// version requester types
PatchVersionRequester = "patch_request"
GithubPRRequester = "github_pull_request"
GitTagRequester = "git_tag_request"
RepotrackerVersionRequester = "gitter_request"
TriggerRequester = "trigger_request"
MergeTestRequester = "merge_test" // Evergreen commit queue
AdHocRequester = "ad_hoc" // periodic build
CreateVersionEndpointRequester = "create_version_endpoint" // create version endpoint
GithubMergeRequester = "github_merge_request" // GitHub merge queue
PatchVersionRequester = "patch_request"
GithubPRRequester = "github_pull_request"
GitTagRequester = "git_tag_request"
RepotrackerVersionRequester = "gitter_request"
TriggerRequester = "trigger_request"
MergeTestRequester = "merge_test" // Evergreen commit queue
AdHocRequester = "ad_hoc" // periodic build
GithubMergeRequester = "github_merge_request" // GitHub merge queue
)

// Constants related to requester types.

var (
// SystemVersionRequesterTypes contain non-patch requesters that are created by the Evergreen system, i.e. configs and patch files are unchanged by author.
SystemVersionRequesterTypes = []string{
RepotrackerVersionRequester,
TriggerRequester,
GitTagRequester,
AdHocRequester,
CreateVersionEndpointRequester,
}
AllRequesterTypes = []string{
PatchVersionRequester,
Expand All @@ -670,7 +670,6 @@ var (
TriggerRequester,
MergeTestRequester,
AdHocRequester,
CreateVersionEndpointRequester,
GithubMergeRequester,
}
)
Expand All @@ -692,15 +691,14 @@ const (
// user-facing functionality such as YAML configuration and expansions and
// should be translated into the true internal requester types so they're
// actually usable.
PatchVersionUserRequester UserRequester = "patch"
GithubPRUserRequester UserRequester = "github_pr"
GitTagUserRequester UserRequester = "github_tag"
RepotrackerVersionUserRequester UserRequester = "commit"
TriggerUserRequester UserRequester = "trigger"
MergeTestUserRequester UserRequester = "commit_queue"
AdHocUserRequester UserRequester = "ad_hoc"
CreateVersionEndpointUserRequester UserRequester = "create_version_endpoint"
GithubMergeUserRequester UserRequester = "github_merge_queue"
PatchVersionUserRequester UserRequester = "patch"
GithubPRUserRequester UserRequester = "github_pr"
GitTagUserRequester UserRequester = "github_tag"
RepotrackerVersionUserRequester UserRequester = "commit"
TriggerUserRequester UserRequester = "trigger"
MergeTestUserRequester UserRequester = "commit_queue"
AdHocUserRequester UserRequester = "ad_hoc"
GithubMergeUserRequester UserRequester = "github_merge_queue"
)

var AllUserRequesterTypes = []UserRequester{
Expand All @@ -711,7 +709,6 @@ var AllUserRequesterTypes = []UserRequester{
TriggerUserRequester,
MergeTestUserRequester,
AdHocUserRequester,
CreateVersionEndpointUserRequester,
GithubMergeUserRequester,
}

Expand All @@ -733,8 +730,6 @@ func InternalRequesterToUserRequester(requester string) UserRequester {
return MergeTestUserRequester
case AdHocRequester:
return AdHocUserRequester
case CreateVersionEndpointRequester:
return CreateVersionEndpointUserRequester
case GithubMergeRequester:
return GithubMergeUserRequester
default:
Expand All @@ -760,8 +755,6 @@ func UserRequesterToInternalRequester(requester UserRequester) string {
return MergeTestRequester
case AdHocUserRequester:
return AdHocRequester
case CreateVersionEndpointUserRequester:
return CreateVersionEndpointRequester
case GithubMergeUserRequester:
return GithubMergeRequester
default:
Expand Down Expand Up @@ -1119,7 +1112,7 @@ func IsGithubMergeQueueRequester(requester string) bool {
}

func ShouldConsiderBatchtime(requester string) bool {
return !IsPatchRequester(requester) && requester != AdHocRequester && requester != GitTagRequester && requester != CreateVersionEndpointRequester
return !IsPatchRequester(requester) && requester != AdHocRequester && requester != GitTagRequester
}

func PermissionsDisabledForTests() bool {
Expand Down
2 changes: 1 addition & 1 deletion model/lifecycle.go
Original file line number Diff line number Diff line change
Expand Up @@ -588,7 +588,7 @@ func CreateBuildFromVersionNoInsert(ctx context.Context, creationInfo TaskCreati
rev = fmt.Sprintf("patch_%s_%s", creationInfo.Version.Revision, creationInfo.Version.Id)
} else if creationInfo.Version.Requester == evergreen.TriggerRequester {
rev = fmt.Sprintf("%s_%s", creationInfo.SourceRev, creationInfo.DefinitionID)
} else if creationInfo.Version.Requester == evergreen.AdHocRequester || creationInfo.Version.Requester == evergreen.CreateVersionEndpointRequester {
} else if creationInfo.Version.Requester == evergreen.AdHocRequester {
rev = creationInfo.Version.Id
} else if creationInfo.Version.Requester == evergreen.GitTagRequester {
rev = fmt.Sprintf("%s_%s", creationInfo.SourceRev, creationInfo.Version.TriggeredByGitTag.Tag)
Expand Down
2 changes: 1 addition & 1 deletion model/project.go
Original file line number Diff line number Diff line change
Expand Up @@ -870,7 +870,7 @@ func NewTaskIdConfigForRepotrackerVersion(p *Project, v *Version, pairsToCreate
rev = fmt.Sprintf("patch_%s_%s", v.Revision, v.Id)
} else if v.Requester == evergreen.TriggerRequester {
rev = fmt.Sprintf("%s_%s", sourceRev, defID)
} else if v.Requester == evergreen.AdHocRequester || v.Requester == evergreen.CreateVersionEndpointRequester {
} else if v.Requester == evergreen.AdHocRequester {
rev = v.Id
} else if v.Requester == evergreen.GitTagRequester {
rev = fmt.Sprintf("%s_%s", sourceRev, v.TriggeredByGitTag.Tag)
Expand Down
6 changes: 4 additions & 2 deletions model/project_vars.go
Original file line number Diff line number Diff line change
Expand Up @@ -322,7 +322,7 @@ func (projectVars *ProjectVars) FindAndModify(varsToDelete []string) (*adb.Chang

func (projectVars *ProjectVars) GetVars(t *task.Task) map[string]string {
vars := map[string]string{}
isAdmin := projectVars.ShouldGetAdminOnlyVars(t)
isAdmin := shouldGetAdminOnlyVars(t)
for k, v := range projectVars.Vars {
if !projectVars.AdminOnlyVars[k] || isAdmin {
vars[k] = v
Expand All @@ -331,7 +331,9 @@ func (projectVars *ProjectVars) GetVars(t *task.Task) map[string]string {
return vars
}

func (projectVars *ProjectVars) ShouldGetAdminOnlyVars(t *task.Task) bool {
// shouldGetAdminOnlyVars returns true if the task is part of a version that can't be modified by users,
// or if the task was activated by a project admin.
func shouldGetAdminOnlyVars(t *task.Task) bool {
if utility.StringSliceContains(evergreen.SystemVersionRequesterTypes, t.Requester) {
return true
} else if t.ActivatedBy == "" {
Expand Down
121 changes: 121 additions & 0 deletions model/project_vars_test.go
Original file line number Diff line number Diff line change
@@ -1,9 +1,14 @@
package model

import (
"fmt"
"testing"

"github.com/evergreen-ci/evergreen"
"github.com/evergreen-ci/evergreen/db"
"github.com/evergreen-ci/evergreen/model/task"
"github.com/evergreen-ci/evergreen/model/user"
"github.com/evergreen-ci/gimlet"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
Expand Down Expand Up @@ -390,3 +395,119 @@ func TestGetParamNameForVar(t *testing.T) {
assert.Zero(t, paramName)
})
}

func TestShouldGetAdminOnlyVars(t *testing.T) {
type testCase struct {
requester string
usrId string
shouldGetAdminVars bool
}

usrId := "not_admin"
adminUsrId := "admin"
testCases := map[string]testCase{
"repotrackerShouldSucceed": {
requester: evergreen.RepotrackerVersionRequester,
usrId: usrId,
shouldGetAdminVars: true,
},
"triggerShouldSucceed": {
requester: evergreen.TriggerRequester,
usrId: usrId,
shouldGetAdminVars: true,
},
"gitTagShouldSucceed": {
requester: evergreen.GitTagRequester,
usrId: usrId,
shouldGetAdminVars: true,
},
"adHocShouldSucceed": {
requester: evergreen.AdHocRequester,
usrId: usrId,
shouldGetAdminVars: true,
},
"patchVersionShouldFail": {
requester: evergreen.PatchVersionRequester,
usrId: usrId,
shouldGetAdminVars: false,
},
"githubPRShouldFail": {
requester: evergreen.GithubPRRequester,
usrId: usrId,
shouldGetAdminVars: false,
},
"mergeTestShouldFail": {
requester: evergreen.MergeTestRequester,
usrId: usrId,
shouldGetAdminVars: false,
},
"mergeRequestShouldFail": {
requester: evergreen.GithubMergeRequester,
usrId: usrId,
shouldGetAdminVars: false,
},
"githubPRWithAdminShouldSucceed": {
requester: evergreen.GithubPRRequester,
usrId: adminUsrId,
shouldGetAdminVars: true,
},
"patchVersionWithAdminShouldSucceed": {
requester: evergreen.PatchVersionRequester,
usrId: adminUsrId,
shouldGetAdminVars: true,
},
}

for name, testCase := range testCases {
assert.NoError(t, db.ClearCollections(user.Collection, evergreen.ScopeCollection, evergreen.RoleCollection))

usr := user.DBUser{
Id: usrId,
}

adminUsr := user.DBUser{
Id: adminUsrId,
}
assert.NoError(t, usr.Insert())
assert.NoError(t, adminUsr.Insert())
env := evergreen.GetEnvironment()
roleManager := env.RoleManager()
projectScope := gimlet.Scope{
ID: "projectScopeID",
Type: evergreen.ProjectResourceType,
Resources: []string{"myProject"},
}
require.NoError(t, roleManager.AddScope(projectScope))

role := gimlet.Role{
ID: "admin_role",
Scope: projectScope.ID,
Permissions: gimlet.Permissions{evergreen.PermissionProjectSettings: evergreen.ProjectSettingsEdit.Value},
}
require.NoError(t, roleManager.UpdateRole(role))
require.NoError(t, adminUsr.AddRole(role.ID))
tsk := &task.Task{
Id: "t1",
Project: "myProject",
}

t.Run(name, func(t *testing.T) {
tsk.Requester = testCase.requester
tsk.ActivatedBy = testCase.usrId

assert.Equal(t, testCase.shouldGetAdminVars, shouldGetAdminOnlyVars(tsk))
})
}

// Verify that all requesters are tested on the non-admin.
for _, requester := range evergreen.AllRequesterTypes {
tested := false
for _, testCase := range testCases {
if testCase.usrId == usrId && requester == testCase.requester {
tested = true
break
}
}
assert.True(t, tested, fmt.Sprintf("requester '%s' not tested with non-admin", requester))
}
}
Loading
Loading