Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Is spawing more then 1 slave broke or am i just doing it wrong? #72

Open
woodshop2300 opened this issue Dec 9, 2020 · 1 comment
Open

Comments

@woodshop2300
Copy link

Excess workload of 1, provisioning new Jenkins worker on Nomad cluster
Dec 09, 2020 8:24:23 PM INFO hudson.slaves.NodeProvisioner$StandardStrategyImpl apply
Started provisioning jenkins-aa81f58e6281 from ENG Cluster with 1 executors. Remaining excess workload: 0
Dec 09, 2020 8:24:23 PM INFO org.jenkinsci.plugins.nomad.NomadCloud$ProvisioningCallback call
Asking Nomad to schedule new Jenkins worker
Dec 09, 2020 8:24:23 PM INFO org.jenkinsci.plugins.nomad.NomadCloud$ProvisioningCallback lambda$call$0
Worker scheduled, waiting for connection
Dec 09, 2020 8:24:32 PM INFO hudson.TcpSlaveAgentListener$ConnectionHandler run
Connection #15 failed: java.io.EOFException
Dec 09, 2020 8:24:32 PM INFO hudson.TcpSlaveAgentListener$ConnectionHandler run
Accepted JNLP4-connect connection #16 from XXXXXXXXXXXXXXXXXXXXXXXXXX
Dec 09, 2020 8:24:37 PM INFO org.jenkinsci.plugins.nomad.NomadCloud$ProvisioningCallback call
Connection established
Dec 09, 2020 8:24:37 PM INFO org.jenkinsci.plugins.nomad.NomadComputer taskAccepted
 Computer jenkins-aa81f58e6281 (worker: org.jenkinsci.plugins.nomad.NomadWorker[jenkins-aa81f58e6281]): task accepted
Dec 09, 2020 8:24:43 PM INFO hudson.slaves.NodeProvisioner lambda$update$6
jenkins-aa81f58e6281 provisioning successfully completed. We have now 2 computer(s)
Dec 09, 2020 8:25:44 PM INFO org.jenkinsci.plugins.nomad.NomadComputer taskCompleted
 Computer jenkins-aa81f58e6281 (worker: org.jenkinsci.plugins.nomad.NomadWorker[jenkins-aa81f58e6281]): task completed
Dec 09, 2020 8:25:45 PM INFO org.jenkinsci.plugins.nomad.NomadComputer taskAccepted
 Computer jenkins-aa81f58e6281 (worker: org.jenkinsci.plugins.nomad.NomadWorker[jenkins-aa81f58e6281]): task accepted

I have been working my through getting this all up and running. I have things at the point that Jenkins can and does spawn my docker build slave when i queue a build. When the build is done, the slave is tore down after a time. So looks well and good.

I next added parameters to my Jenkins build so i could queue a bunch of those builds w/o Jenkins pruning them since they would otherwise be identical and queued up 20 builds.

My builds are very simple, its a alpine base docker with openjdk8 and my build command is sleep 60, so this is all pure testing, there isn't even version management checkouts involved in the process.

What happens is the above log snippet, 1 build slave gets spawned, and it slowly churns through all 20 builds and gets destroyed. Based on the other issues here, NomadProvisioningStrategy.java, and my own expectations i would think mutiple build slaves should have been spawned.

Reading the log snippet, the Excess workload log line would suggest reading the queue depth is somehow broken or otherwise not returning that there is 19 other builds it needs to work though.

Jenkins Nomad plugin version 0.7.1

Jenkins Version 2.263.1

@j3t
Copy link
Member

j3t commented Jul 23, 2021

I have experienced a similar behavior but this was related to my test setup.

If you have no workers running and you are starting a bunch of job execution in parallel then only one worker gets started and all jobs are waiting for the same executer.

I guess this behavior is related to Jenkins and how jobs are scheduled because when the jobs are not scheduled all at once then a new worker gets started if necessary.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants