Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update test matrix to run faster & add 18.04 #107

Merged
merged 13 commits into from
Dec 18, 2018
Merged

Update test matrix to run faster & add 18.04 #107

merged 13 commits into from
Dec 18, 2018

Conversation

onlyhavecans
Copy link
Member

@onlyhavecans onlyhavecans commented Dec 13, 2018

Travis numbers:
Before: Ran for 1 hr 29 min 42 sec
After: Ran for 12 min 12 sec

Using travis' concurrency with separated kitchen instances makes finding errors, parsing the logs, and all general troubleshooting better too.

Signed-off-by: Amy Aronsohn <WagThatTail@Me.com>
Signed-off-by: Amy Aronsohn <WagThatTail@Me.com>
@onlyhavecans
Copy link
Member Author

On both suites it's not the authoritive that is failing now but Postgresql test suite, they are timing out. I did not have this issue when I ran all the 18.04 tests locally but I will run the full suite and see if I can replicate

Signed-off-by: Amy Aronsohn <WagThatTail@Me.com>
PoswerDNS did not make a 409 repo for Bionic

Signed-off-by: Amy Aronsohn <WagThatTail@Me.com>
@onlyhavecans
Copy link
Member Author

The problem is we cannot test installing 40 on 18.04, there is no repo for it. The tests were hiding the problem because they have allow failure on, so things broke later.

---- Begin output of ["apt-get", "-q", "update"] ----                                                                 
STDOUT: Hit:1 http://archive.ubuntu.com/ubuntu bionic InRelease                                                       
Hit:2 http://security.ubuntu.com/ubuntu bionic-security InRelease                                                     
Ign:3 http://repo.powerdns.com/ubuntu bionic-auth-40 InRelease                                                        
Hit:4 http://archive.ubuntu.com/ubuntu bionic-updates InRelease                                                       
Err:5 http://repo.powerdns.com/ubuntu bionic-auth-40 Release                                                          
  404  Not Found [IP: 188.166.116.224 80]                                                                             
Hit:6 http://archive.ubuntu.com/ubuntu bionic-backports InRelease                                                     
Hit:7 https://download.postgresql.org/pub/repos/apt bionic-pgdg InRelease                                             
Reading package lists...                                                                                              
STDERR: E: The repository 'http://repo.powerdns.com/ubuntu bionic-auth-40 Release' does not have a Release file.      
---- End output of ["apt-get", "-q", "update"] ----                                                                   
Ran ["apt-get", "-q", "update"] returned 100; ignore_failure is set, continuing                                       
                                                                                                                      
        ================================================================================                              
        Error executing action `update` on resource 'apt_update[powerdns-authoritative]'                              
        ================================================================================                              
                                                                                                                      
        Mixlib::ShellOut::ShellCommandFailed                                                                          
        ------------------------------------                                                                          
        execute[apt-get -q update] (/opt/chef/embedded/lib/ruby/gems/2.5.0/gems/chef-14.8.12/lib/chef/provider/apt_upd
ate.rb line 70) had an error: Mixlib::ShellOut::ShellCommandFailed: Expected process to exit with [0], but received '1
00'                                                                                                                   
        ---- Begin output of ["apt-get", "-q", "update"] ----                                                         
        STDOUT: Hit:1 http://archive.ubuntu.com/ubuntu bionic InRelease                                               
        Hit:2 http://security.ubuntu.com/ubuntu bionic-security InRelease                                             
        Ign:3 http://repo.powerdns.com/ubuntu bionic-auth-40 InRelease                                                
        Hit:4 http://archive.ubuntu.com/ubuntu bionic-updates InRelease                                               
        Err:5 http://repo.powerdns.com/ubuntu bionic-auth-40 Release                                                  
          404  Not Found [IP: 188.166.116.224 80]                                                                     
        Hit:6 http://archive.ubuntu.com/ubuntu bionic-backports InRelease                                             
        Hit:7 https://download.postgresql.org/pub/repos/apt bionic-pgdg InRelease                                     
        Reading package lists...                                                                                      
        STDERR: E: The repository 'http://repo.powerdns.com/ubuntu bionic-auth-40 Release' does not have a Release fil
e.                                                                                                                    
        ---- End output of ["apt-get", "-q", "update"] ----                                                           
        Ran ["apt-get", "-q", "update"] returned 100                                                                  
                                                                                                                      
        Resource Declaration:                                                                                         
        ---------------------                                                                                         
        # In /opt/chef/embedded/lib/ruby/gems/2.5.0/gems/chef-14.8.12/lib/chef/provider/apt_repository.rb             
                                                                                                                      
         57:         declare_resource(:apt_update, new_resource.name) do                                              
         58:           ignore_failure true                                                                            
         59:           action :nothing                                                                                
         60:         end                                                                                              
         61:                                                                                                          

We install a specific minor now
Signed-off-by: Amy Aronsohn <WagThatTail@Me.com>
Signed-off-by: Amy Aronsohn <WagThatTail@Me.com>
Signed-off-by: Amy Aronsohn <WagThatTail@Me.com>
@onlyhavecans
Copy link
Member Author

I'm passing all locally now so just waiting for tests to finish in Travis

Signed-off-by: Amelia Aronsohn <WagThatTail@Me.com>
right now tracing down what test failed in a huge suite is problimatic.
Lets break this up to make it easier to troubleshoot.

I stole all of this from chef-cookbooks/chef-client

Signed-off-by: Amelia Aronsohn <WagThatTail@Me.com>
Signed-off-by: Amelia Aronsohn <WagThatTail@Me.com>
Signed-off-by: Amelia Aronsohn <WagThatTail@Me.com>
@onlyhavecans
Copy link
Member Author

Trying to make the test suite faster I enabled concurrent builds for this repo, there is not any resources it needs that should require them. I also broke out all instances so they are easier to see where the failures are. This gives us much more concurrency than the two we originally had

Signed-off-by: Amelia Aronsohn <WagThatTail@Me.com>
@onlyhavecans onlyhavecans changed the title Add 18.04 support to test matrix Update test matrix to run faster & add 18.04 support Dec 14, 2018
@onlyhavecans onlyhavecans changed the title Update test matrix to run faster & add 18.04 support Update test matrix to run faster & add 18.04 Dec 14, 2018
@onlyhavecans
Copy link
Member Author

While we still have the one test failing and timing out at least it runs way faster

Before: Ran for 1 hr 29 min 42 sec
After: Ran for 21 min 1 sec

@martinisoft
Copy link
Contributor

If we're ok with the 18.04 PG test not completing we can either remove it or comment it out with a note.

Signed-off-by: Amelia Aronsohn <WagThatTail@Me.com>
@onlyhavecans
Copy link
Member Author

I've only removed it in the travis matrix, as the test runs locally without issue

Copy link
Contributor

@martinisoft martinisoft left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fantastic work! Sooooooo much faster too, I love it!

@onlyhavecans onlyhavecans merged commit 9259cc0 into master Dec 18, 2018
@onlyhavecans onlyhavecans deleted the 1804 branch December 18, 2018 20:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants