Skip to content

Commit

Permalink
Merge remote-tracking branch 'upstream/master'
Browse files Browse the repository at this point in the history 
* upstream/master: (35 commits)
  Prep for release
  Added setting `:magento_deploy_jobs` to support configuring number of parallel static content deployment tasks
  Improved error reporting on static content deployment failure
  Updated uses of bin/magento where output is parsed to include `--no-ansi` to eliminate potential failures
  Fixed issue where ./update dir may exist without a composer.json file, causing deployment failure
  Version bump
  Fixed Magento version check failing on some servers due to ansi output in non-interactive shells
  Cleaned up add auth credential settings and added information to README
  Updated pending change log hook
  README corrections
  Fixed spelling error in README (lame spell checker, lame)
  Updated CL and version number
  Updates to README for issue davidalger#58
  Updated messaging to include host name on all pending change log heading messages
  Added full-featured pending changes logging
  Set :capistrano_pending_role default to match :magento_deploy_setup_role to avoid potential incompatibilities
  Fixed inability to set PATH in capistrano configuration vs `.bashrc` file
  Remove terminal-notifier dependency
  Prepare for version 0.5.9 release
  Add composer magento auth config
  ...
  • Loading branch information
@tunght13488
tunght13488 committed Apr 12, 2017
2 parents f1399ce + 45915fe commit d7ffd22
Show file tree
Hide file tree
Showing 9 changed files with 297 additions and 102 deletions.
45 changes: 44 additions & 1 deletion CHANGELOG.md
View file
Original file line number Diff line number Diff line change
@@ -1,9 +1,52 @@
# Capistrano::Magento2 Change Log

0.6.2
==========

* Added setting `:magento_deploy_jobs` to support configuring number of parallel static content deployment tasks
* Fixed issue where ./update dir may exist without a composer.json file, causing deployment failure
* Updated uses of bin/magento where output is parsed to include `--no-ansi` to eliminate potential failures
* Improved error reporting on static content deployment failure

0.6.1
==========

* Fixed Magento version check failing on some servers due to ansi output in non-interactive shells (issue #64)
* Added ability to configure Magento's composer authentication keys. See README for details (PR #56)
* Changed pending change log to hook into before `deploy:check` (previously hooked before `deploy`)

0.6.0
==========

* Added full-featured pending change logging functionality. See README for details (issue #58)
* Fixed inability to set PATH in capistrano configuration vs `.bashrc` file (issue #62)
* Updated README to reflect removing the `terminal-notifier` gem as a hard dependency (issue #19)
* Removed `capistrano-pending` as a dependency (issue #58)

0.5.9
==========

* Updated README with Capistrano 3.7 setup information
* Updated `linked_dirs` to link `pub/sitemaps` by default in similar fashion to the Magento1 deployment gem
* Updated README with guidance on adding a path to the list of `linked_dirs` without copying the entire configuration forward
* Fixed bug causing pipefail option to persist after `Capistrano::Magento2::Setup.static_content_deploy` is called

0.5.8
==========

* Fixed critical failure due to command map being broken in v0.5.7 updates (issue #50, issue #51)

0.5.7
==========
_Note: This release was yanked from RubyGems due to a critical failure in the deploy routine._

* Fixed failing deploys for Magento 2.1.0 caused by improper version checks on flags added in version 2.1.1 (issue #45)
* Fixed failure to detect error codes Magento 2.1.1 returns on a failed static-content deploy job (issue #44)

0.5.6
==========

* Fixed issue where setup:di:compile failing to return an exit code caused DI compilation failures to be masked
* Fixed issue where setup:di:compile failing to return an exit code caused DI compilation failures to be masked (issue #41)

0.5.5
==========
Expand Down
92 changes: 79 additions & 13 deletions README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -43,6 +43,11 @@ _Note: By default, Capistrano creates "staging" and "production" stages. If you
# Load Magento deployment tasks
require 'capistrano/magento2/deploy'
require 'capistrano/magento2/pending'
# Load Git plugin
require "capistrano/scm/git"
install_plugin Capistrano::SCM::Git
# Load custom tasks from `lib/capistrano/tasks` if you have any defined
Dir.glob('lib/capistrano/tasks/*.rake').each { |r| import r }
Expand Down Expand Up @@ -114,7 +119,8 @@ Before you can use Capistrano to deploy, you must configure the `config/deploy.r
| `:magento_deploy_setup_role` | `:all` | Role from which primary host is chosen to run things like setup:upgrade on
| `:magento_deploy_cache_shared` | `true` | If true, cache operations are restricted to the primary node in setup role
| `:magento_deploy_languages` | `['en_US']` | Array of languages passed to static content deploy routine
| `:magento_deploy_themes` | `[]` | Array of themes passed to static content deploy routine (Magento 2.1+ only)
| `:magento_deploy_themes` | `[]` | Array of themes passed to static content deploy (Magento 2.1.1 and later)
| `:magento_deploy_jobs` | `4` | Number of threads to use for static content deploy (Magento 2.1.1 and later)
| `:magento_deploy_composer` | `true` | Enables composer install behaviour in the built-in deploy routine
| `:magento_deploy_production` | `true` | Enables production specific DI compilation and static content generation
| `:magento_deploy_maintenance` | `true` | Enables use of maintenance mode while magento:setup:upgrade runs
Expand Down Expand Up @@ -143,12 +149,12 @@ For the sake of simplicity in new project setups `:linked_dirs` and `:linked_fil
set :linked_files, [
'app/etc/env.php',
'var/.setup_cronjob_status',
'var/.update_cronjob_status',
'pub/sitemap.xml'
'var/.update_cronjob_status'
]
set :linked_dirs, [
'pub/media',
'pub/media',
'pub/sitemaps',
'var/backups',
'var/composer_home',
'var/importexport',
Expand All @@ -159,7 +165,24 @@ set :linked_dirs, [
]
```
If you would like to customize the linked files or directories for your project, you can copy either one or both of the above arrays into the `config/deploy.rb` or `config/deploy/*.rb` files and tweak them to fit your project's needs.
If you would like to customize the linked files or directories for your project, you can copy either one or both of the above arrays into the `config/deploy.rb` or `config/deploy/*.rb` files and tweak them to fit your project's needs. Alternatively, you can add a single linked dir (or file) using `append` like this:

```ruby
append :linked_dirs, 'path/to/link'
```

### Composer Auth Credentials

Magento 2's composer repository requires auth credentials to install. These can be set on target servers in a global composer `auth.json` file, the project's `composer.json` or by setting them in your deployment configuration using the following two settings:

```ruby
set :magento_auth_public_key, '<your_public_key_here>'
set :magento_auth_private_key, '<your_prviate_key_here>'
```

To obtain these credentials, reference the official documentation on DevDocs: [Get your authentication keys](http://devdocs.magento.com/guides/v2.0/install-gde/prereq/connect-auth.html)

**Caution:** When using these settings, the values will be logged to the `log/capistrano.log` file by SSHKit. They will not, however, be included in the general command output by default.

### Magento 2 Deploy Routine

Expand All @@ -171,7 +194,7 @@ To see what process the built-in routine runs, take a look at the included rake

### Web Server Root Path

Before deploying with Capistrano, you must update each of your web servers to point to a `current` directory inside of the `:deploy_to` directory. For example: `/var/www/html/current` Refer to the [Capistrano Structure](http://capistranorb.com/documentation/getting-started/structure/) to learn more about Capistrano's folder structure.
Before deploying with Capistrano, you must update each of your web servers to point to the `current` directory inside of the configured `:deploy_to` directory. For example: `/var/www/html/current/pub` Refer to the [Capistrano Structure](http://capistranorb.com/documentation/getting-started/structure/) to learn more about Capistrano's folder structure.
## Magento Specific Tasks
Expand Down Expand Up @@ -200,21 +223,64 @@ All Magento 2 tasks used by the built-in `deploy.rake` file as well as some addi
| magento:setup:static-content:deploy | Deploys static view files |
| magento:setup:upgrade | Run the Magento upgrade process |
## Terminal Notifier on OS X
This gem specifies [terminal-notifier](https://rubygems.org/gems/terminal-notifier) as a dependency in order to support notifications on OS X via an optional include. To use the built-in notifications, add the following line to your `Capfile`:
## Pending Changes Support
When the line `require 'capistrano/magento2/pending'` is included in your `Capfile` per the recommended configuration above, this gem will report changes pending deployment in an abbreviated git log style format. Here is an example:
```
00:00 deploy:pending:log
01 git fetch origin
✔ 01 dalger@localhost 1.241s
✔ 01 dalger@localhost 1.259s
Changes pending deployment on web1 (tags/2.1.2 -> 2.1):
f511288 Thu Feb 23 12:19:20 2017 -0600 David Alger (HEAD -> 2.1, tag: 2.1.4, origin/2.1) Magento 2.1.4
7fb219c Thu Feb 23 12:17:11 2017 -0600 David Alger (tag: 2.1.3) Magento 2.1.3
570c9b3 Thu Feb 23 12:12:43 2017 -0600 David Alger Updated capistrano configuration
No changes to deploy on web2 (from and to are the same: 2.1 -> 2.1)
```
When there are no changes due for deployment to any host, a warning requiring confirmation will be emitted by default:
```
No changes to deploy on web1 (from and to are the same: 2.1 -> 2.1)
No changes to deploy on web2 (from and to are the same: 2.1 -> 2.1)
Are you sure you want to continue? [y/n]
```
This confirmational warning can be disabled by including the following in your project's configuration:

```ruby
require 'capistrano/magento2/notifier'
set :magento_deploy_pending_warn, false
```

## Pending Changes
### Pending Changes Configuration

This gem specifies [capistrano-pending](https://rubygems.org/gems/capistrano-pending) as a dependency and adds some (optional) custom functionality on top of that gem: Any time the `deploy` command is run, a one line summary of git commits that will be deployed will be displayed. If the server(s) you are deploying to already have the latest changes, you will be warned of this and a prompt will appear confirming that you want to continue deploying.
| setting | what it does
| -------------------------------- | ------- | ---
| `:magento_deploy_pending_role` | Role to check for pending changes on; defaults to `:all`
| `:magento_deploy_pending_warn` | Set this to `false` to disable confirmational warning on zero-change deployments
| `:magento_deploy_pending_format` | Can be used to set a custom change log format; refer to `defaults.rb` for example

### Pending Changes Tasks

| cap command | what it does |
| ------------------------------------- | -------------------------------------------------- |
| deploy:pending | Displays a summary of commits pending deployment |

Note: For more details including screenshots of what this functionality does, reference [this post](https://github.com/davidalger/capistrano-magento2/issues/58#issuecomment-282404477).

## Terminal Notifier on OS X

This gem includes an optional configuration file include which adds notification support via the [terminal-notifier](https://rubygems.org/gems/terminal-notifier) gem. To configure notifications, simply add the following line to your `Capfile`:

```ruby
require 'capistrano/magento2/notifier'
```

To add the `capistrano-pending` gem and additional functionality to you project, add the following line to your `Capfile`:
**Notice:** The `terminal-notifier` gem is currently macOS specific and thus can not be used on generic *nix environments. Because this gem has been known to cause ruby stability issues on certain non-macOS environments, it is not specified as a hard requirement in this gem's gemspec. When using this functionality, it is expected the gem either be already present on your working environment or be added to your project's `Gemfile`:

```ruby
require 'capistrano/magento2/pending'
gem 'terminal-notifier'
```

## Development
Expand Down
4 changes: 0 additions & 4 deletions capistrano-magento2.gemspec
View file
Original file line number Diff line number Diff line change
Expand Up @@ -29,10 +29,6 @@ Gem::Specification.new do |spec|

spec.add_dependency 'capistrano', '~> 3.1'

# TODO explore removing these from gemspec per issue #19
spec.add_dependency 'terminal-notifier', '~> 1.6'
spec.add_dependency 'capistrano-pending', '~> 0.1'

spec.add_development_dependency 'bundler', '~> 1.7'
spec.add_development_dependency 'rake', '~> 10.0'
end
29 changes: 23 additions & 6 deletions lib/capistrano/magento2.rb
View file
Original file line number Diff line number Diff line change
Expand Up @@ -13,11 +13,11 @@ module Capistrano
module Magento2
module Helpers
def magento_version
return (capture "/usr/bin/env php -f #{release_path}/bin/magento -- -V").split(' ').pop.to_f
return Gem::Version::new((capture :php, "-f #{release_path}/bin/magento -- -V --no-ansi").split(' ').pop)
end

def disabled_modules
output = capture :magento, 'module:status'
output = capture :magento, 'module:status --no-ansi'
output = output.split("disabled modules:\n", 2)[1]

if output == nil or output.strip == "None"
Expand All @@ -29,21 +29,38 @@ def disabled_modules
def cache_hosts
return fetch(:magento_deploy_cache_shared) ? (primary fetch :magento_deploy_setup_role) : (release_roles :all)
end

# Set pipefail allowing console exit codes in Magento 2.1.1 and later to halt execution when using pipes
def Helpers.set_pipefail
if not SSHKit.config.command_map[:magento].include? 'set -o pipefail' # avoids trouble on multi-host deploys
@@pipefail_less = SSHKit.config.command_map[:magento].dup
SSHKit.config.command_map[:magento] = "set -o pipefail; #{@@pipefail_less}"
end
end

# Reset the command map without prefix, removing pipefail option so it won't affect other commands
def Helpers.unset_pipefail
SSHKit.config.command_map[:magento] = @@pipefail_less
end
end

module Setup
def static_content_deploy params
Helpers.set_pipefail
output = capture :magento,
"setup:static-content:deploy #{params} | stdbuf -o0 tr -d .",
"setup:static-content:deploy --no-ansi #{params} | stdbuf -o0 tr -d .",
verbosity: Logger::INFO
Helpers.unset_pipefail

# String based error checking is here to catch errors in Magento 2.1.0 and earlier; later versions will exit
# immediately when a console exit code is retruned, never evaluating this code.
if not output.to_s.include? 'New version of deployed files'
raise Exception, "\e[0;31mFailed to compile static assets\e[0m"
raise Exception, "\e[0;31mFailed to compile static assets. No new version found in command output!\e[0m"
end

output.to_s.each_line { |line|
if line.split('errors: ', 2).pop.to_i > 0
raise Exception, "\e[0;31mFailed to compile static assets\e[0m"
raise Exception, "\e[0;31mFailed to compile static assets. Errors found in command output: #{line}\e[0m"
end
}
end
Expand Down
23 changes: 18 additions & 5 deletions lib/capistrano/magento2/defaults.rb
View file
Original file line number Diff line number Diff line change
Expand Up @@ -10,19 +10,18 @@
set :linked_files, fetch(:linked_files, []).push(
'app/etc/env.php',
'var/.setup_cronjob_status',
'var/.update_cronjob_status',
'pub/sitemap.xml'
'var/.update_cronjob_status'
)

set :linked_files_touch, fetch(:linked_files_touch, []).push(
'app/etc/env.php',
'var/.setup_cronjob_status',
'var/.update_cronjob_status',
'pub/sitemap.xml'
'var/.update_cronjob_status'
)

set :linked_dirs, fetch(:linked_dirs, []).push(
'pub/media',
'pub/media',
'pub/sitemaps',
'var/backups',
'var/composer_home',
'var/importexport',
Expand All @@ -32,6 +31,11 @@
'var/tmp'
)

# magento composer repository auth credentials
set :magento_auth_repo_name, fetch(:magento_auth_repo_name, 'http-basic.repo.magento.com')
set :magento_auth_public_key, fetch(:magento_auth_public_key, false)
set :magento_auth_private_key, fetch(:magento_auth_private_key, false)

# deploy permissions defaults
set :magento_deploy_chmod_d, fetch(:magento_deploy_chmod_d, '2770')
set :magento_deploy_chmod_f, fetch(:magento_deploy_chmod_f, '0660')
Expand All @@ -44,7 +48,16 @@
set :magento_deploy_maintenance, fetch(:magento_deploy_maintenance, true)
set :magento_deploy_production, fetch(:magento_deploy_production, true)
set :magento_deploy_themes, fetch(:magento_deploy_themes, [])
set :magento_deploy_jobs, fetch(:magento_deploy_jobs, nil) # this defaults to 4 when supported by bin/magento

# deploy targetting defaults
set :magento_deploy_setup_role, fetch(:magento_deploy_setup_role, :all)
set :magento_deploy_cache_shared, fetch(:magento_deploy_cache_shared, true)

# pending deploy check defaults
set :magento_deploy_pending_role, fetch(:magento_deploy_pending_role, :all)
set :magento_deploy_pending_warn, fetch(:magento_deploy_pending_warn, true)
set :magento_deploy_pending_format, fetch(
:magento_deploy_pending_format,
'--pretty="format:%C(yellow)%h %Cblue%>(12)%ad %Cgreen%<(7)%aN%Cred%d %Creset%s"'
)
68 changes: 32 additions & 36 deletions lib/capistrano/magento2/pending.rb
View file
Original file line number Diff line number Diff line change
Expand Up @@ -8,46 +8,42 @@
##

require 'capistrano/deploy'
require 'capistrano/pending/scm/base'
require 'capistrano/magento2'

module Capistrano
module Pending
module SCM
class Git < Base

# Enhance capistrano-pending gem's native deploy:pending:log command by updating repository and then
# showing the actual changes that will be deployed. Also changes log output to oneline for easy reading
#
# Params:
# +from+ - commit-ish to compare from
# +to+ - commit-ish to compare to
# +returnOutput+ - whether to return or print the output
#
def log(from, to, returnOutput=false)
module Magento2
module Pending
def ensure_revision inform_user = false
if test "[ -f #{current_path}/REVISION ]"
yield
else
warn "\e[0;31mSkipping pending changes check on #{host} (no REVISION file found)\e[0m" if inform_user
return false
end
return true
end

def from_rev
within current_path do
current_revision = capture(:cat, "REVISION")

run_locally do
execute :git, :fetch, :origin # update repository to ensure accuracy of pending changes report

# Since the :branch to deploy from may be behind the upstream branch, get name of upstream branch
# and use it for comparison. We are using the test command in case the :branch is set to a specific
# commit hash, in which case there is no upstream branch.

if test(:git, 'rev-parse', '--abbrev-ref', '--symbolic-full-name', to + '@{u}')
to = capture(:git, 'rev-parse', '--abbrev-ref', '--symbolic-full-name', to + '@{u}')
end

output = capture(
:git,
:log,
"#{from}..#{to}",
'--pretty="format:%C(yellow)%h %Cblue%>(12)%ad %Cgreen%<(7)%aN%Cred%d %Creset%s"'
)

if returnOutput
return output
else
puts output
end
return capture(:git, "name-rev --always --name-only #{current_revision}") # find symbolic name for ref
end
end
end

def to_rev
run_locally do
to = fetch(:branch)

# get target branch upstream if there is one
if test(:git, "rev-parse --abbrev-ref --symbolic-full-name #{to}@{u}")
to = capture(:git, "rev-parse --abbrev-ref --symbolic-full-name #{to}@{u}")
end

# find symbolic name for revision
to = capture(:git, "name-rev --always --name-only #{to}")
end
end
end
Expand Down
Loading

0 comments on commit d7ffd22

Please sign in to comment.