Skip to content

Commit

Permalink
Merge pull request #6 from cbrainerd/feat_public_dev
Browse files Browse the repository at this point in the history
Move development to the public repo
  • Loading branch information
dmoxon committed May 20, 2016
2 parents 6102e10 + 1430a82 commit b0b42ed
Show file tree
Hide file tree
Showing 35 changed files with 1,174 additions and 59,799 deletions.
11 changes: 11 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
.DS_Store
*.pyc
dist
env
.coverage
.cache
stat_key_browser/data/key_cats.json
stat_key_browser/data/key_tags.json
web_app/js/keys.js
web_app/index.html
isilon_stat_browser_*.zip
19 changes: 19 additions & 0 deletions .travis.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
language: python
python:
- "2.7"
- "3.3"
- "3.4"
- "3.5"

virtualenv:
system_site_packages: false

install:
- pip install -r requirements-dev.txt
- git clone https://github.com/Isilon/isilon_sdk_7_2_python.git
- cd isilon_sdk_7_2_python
- python setup.py install
- cd ..

script:
- make travis-ci
47 changes: 47 additions & 0 deletions Makefile
Original file line number Diff line number Diff line change
@@ -0,0 +1,47 @@
PYTHON=`which python`
PIP=`which pip`
PYLINT=`which pylint`
HEXAPARSE=./hexaparse.py

DIST_DIR=dist

clean:
-rm -rf $(DIST_DIR)
-rm -rf test_results
-rm stat_key_browser/data/key_tags.json
-rm stat_key_browser/data/key_cats.json
-rm web_app/js/keys.js
-rm web_app/index.html
-rm isilon_stat_browser_v*.zip

tags:
$(HEXAPARSE) stat_key_browser/data/key_tags.hexa > stat_key_browser/data/key_tags.json
$(HEXAPARSE) stat_key_browser/data/key_cats.hexa > stat_key_browser/data/key_cats.json

lint:
$(PYLINT) -E -f colorized -r n stat_key_browser bin/ tests/

unittests: lint
$(PYTHON) -m pytest -v tests/unit/ *.py

coverage: lint
$(PYTHON) -m pytest -v --cov=stat_key_browser --cov-report term-missing --cov-config tests/unit/.coveragerc tests/unit/ *.py

travis-ci: tags lint
$(PYTHON) -m pytest -v tests/unit/ hexaparse.py

check_cluster:
if [ -z $$BUILD_BROWSER_ARGS ]; then echo BUILD_BROWSER_ARGS not set, builder will pause for input; fi

functional_tests: check_cluster clean unittests tags
$(PYTHON) -m pytest -v tests/functional/

dist: check_cluster clean unittests tags
./build_stat_browser.py -x $(BUILD_BROWSER_ARGS)
mkdir -p $(DIST_DIR)
cp build_stat_browser.py $(DIST_DIR)
cp requirements.txt $(DIST_DIR)
cp README.md $(DIST_DIR)
cp -r stat_key_browser $(DIST_DIR)
cp -r web_app $(DIST_DIR)
zip -r isilon_stat_browser_$$(git describe --exact-match --abbrev=0).zip dist/*
133 changes: 133 additions & 0 deletions README-dev.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,133 @@
[![Master Build Status](https://travis-ci.org/Isilon/isilon_stat_browser.svg?branch=master)](https://travis-ci.org/Isilon/isilon_stat_browser)
[![Average time to resolve an issue](http://isitmaintained.com/badge/resolution/isilon/isilon_stat_browser.svg)](http://isitmaintained.com/project/isilon/isilon_stat_browser "Average time to resolve an issue")
[![Percentage of issues still open](http://isitmaintained.com/badge/open/isilon/isilon_stat_browser.svg)](http://isitmaintained.com/project/isilon/isilon_stat_browser "Percentage of issues still open")

#Statistics Key Browser Development

A single page web app that provides a browsable, searchable view of Isilon statistics keys. A Python script generates the html by querying an Isilon cluster for the list of statistics keys, then organizing and categorizing the keys before outputting the html app to web_app.

##Requirements
Python: 2.7, 3.3, 3.4, 3.5

Dependencies listed in requirements-dev.txt

Isilon SDK Python language bindings

[`https://github.com/isilon/`](https://github.com/isilon)

##Development getting started

###Clone this repo:

`git clone <repo>`

`cd isilon_stat_browser`

### Install the dependencies:

`pip install -r requirements-dev.txt`

### Run unit tests:

`make unittests`

### Check unit test coverage:

`make coverage`

### Run functional tests:
The functional tests are only a stub currently.

`make functional_tests`

###Run the page building tool

`./build_stat_browser.py -c <cluster IP>`

### Generate a distributable zip file:
The build will pause to request cluster IP, username and password.

`make dist`

### Generate a distributable zip file non-interactively:

Supply the cluster IP, username and password when building via automation.

`make dist BUILD_BROWSER_ARGS='-c <cluster IP> -u <username> -p <password>'`

# Release process

**Note:**
This is a temporary manual process to be used by repo owners to cut a release until automated build/release is implemented.


Once the master branch is in a state ready for a release, tag the current commit
with a version number.

`git tag -a v0.0.1 -m 'version 0.0.1'`

Push the tag to git

`git push origin v0.0.1`

This creates a release in [isilon\_stat\_browser/releases](../../releases)

Create the distribution for release

`make dist BUILD_BROWSER_ARGS='-c <cluster IP> -u <username> -p <password>'`

This creates a .zip file in the top-level project directory. The file will be
automatically named with the version specified in the tag. If no version number
appears in the file name something has gone wrong.

Go to [isilon\_stat\_browser/releases](../../releases) and draft a new release.
Enter the tag into the tag version box and the tag should be recognized as an
existing tag.

Enter any needed release notes

Attach the zipped release distribution to the release.

Publish the release.

# Files

* `README-dev.md`

The developer facing readme that you are reading now.

* `README.md`

The user-facing readme that gets packaged into the distributable zip.

* `stat_key_browser/data/key_cats.hexa`

A human written and readable file that defines which categories and subcategories are to be applied to lists of key names. Parsed into json by hexaparse.py during `make tags`.

* `stat_key_browser/data/key_cats.json`:

The automatically generated JSON that results when hexaparse.py parses key\_cats.hexa during the build process. This file is referenced by build\_stat\_browser.py to categorize stat keys received from PAPI.

* `stat_key_browser/data/key_tags.hexa`

A human written and readable file that defines which tags are to be applied to lists of key names. Parsed into json by hexaparse.py during the build process.

* `stat_key_browser/data/key_tags.json`:

The automatically generated JSON that results when hexaparse.py parsed key\_tags.hexa during the build process. This file is referenced by build\_stat\_browser.py to tag stat keys received from PAPI and is part of the distributable zip.

* `keys.js`

JSON formatted stat keys with tags and categories attached. This is created by build\_stat\_browser.py and read by the html app's JavaScript.

* `stat_key_browser/templates/app_template.html`

The main template for the app. Contains the main html plus categories.

* `stat_key_browser/template/key_template.html`

Contains a template representing a single key and all of its info, including its extra info.

* `web_app/index.html`

This is the file opened by the user to view the stat browser. This file is rendered by build_stat_browser.py from the templates, the definitions in key_tags.json and from the PAPI supplied list of keys.
45 changes: 34 additions & 11 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,25 +1,46 @@
[![Master Build Status](https://travis-ci.org/Isilon/isilon_stat_browser.svg?branch=master)](https://travis-ci.org/Isilon/isilon_stat_browser)
[![Average time to resolve an issue](http://isitmaintained.com/badge/resolution/isilon/isilon_stat_browser.svg)](http://isitmaintained.com/project/isilon/isilon_stat_browser "Average time to resolve an issue")
[![Percentage of issues still open](http://isitmaintained.com/badge/open/isilon/isilon_stat_browser.svg)](http://isitmaintained.com/project/isilon/isilon_stat_browser "Percentage of issues still open")

#Statistics Key Browser

This repository is part of the Isilon SDK, which is an evolving package of documents and files. This README describes how to use the statistics key browser (stat browser) to browse the statistics keys that an Isilon cluster exposes. The stat browser is a Python script-generated web browser that provides a searchable list of OneFS statistics keys, organized by functional categories.

##Requirements
Python 2.7 or 3.5

A compatible web browser:

* Google Chrome
* Firefox
* Safari
* Internet Explorer 11
* Microsoft Edge

##Installation
You can obtain the files for the stat browser by cloning this Github repository:
You can obtain the files for the stat browser by downloading a release zip file from this Github repository:

`git clone git@github.com/isilon/isilon_stat_browser.git`
[isilon\_stat\_browser/releases](../../releases)

or by downloading the isilon_stat_browser zip file through the "Download ZIP" link and extracting the files.
## Development

Developers can view the development readme for information about stat browser development.

[isilon\_stat\_browser/README-dev.md](README-dev.md)

`unzip <branch-name>.zip`

##View statistics keys with the stat browser
Once the files and directories in the zip file are extracted, you will see a stat_key_browser directory. Open the web_app/index.html file in a web browser to browse the statistics keys available in the distribution package.

##Generate the stat browser

You can also generate the stat browser from your OneFS cluster by running the build_stat_browser.py Python script. You may want to perform this step on your version of OneFS because available statistics keys can vary between OneFS versions, so generating the keys will provide the most accurate list for your implementation.
You can also generate the stat browser from your OneFS cluster by running the build\_stat\_browser.py Python script. You may want to perform this step on your version of OneFS because available statistics keys can vary between OneFS versions, so generating the keys will provide the most accurate list for your implementation. During the generation process, your cluster will be queried for its list of statistc keys to generate the stat browser.

### Requirements
Python: 2.7, 3.3, 3.4, 3.5

Isilon SDK Python Language Bindings installed

### Generating the stat browser

First unpack the zip file as described above and then install the requirements.

Expand All @@ -29,16 +50,19 @@ Next run the page builder:

`./build_stat_browser.py --cluster <hostname-or-ip-address>`

For help on the build_stat_browser.py usage, run:
For help on build\_stat\_browser.py usage, run:

`./build_stat_browser.py --help`

The script will prompt you for a username and password, and will create the browsable web_app/index.html page and key_cats.json Java script which contains the statistics keys categories.
The script will prompt you for a username and password, and will create the browsable web\_app/index.html page and key\_cats.json Java script which contains the statistics keys categories.

##Install SDK Python language bindings

You can use the SDK Python language bindings to automate the configuration, maintenance, and monitoring of your Isilon cluster. For information on how to install the Python language bindings and write Python scripts to access the OneFS API, refer to the following Github site:
[`https://github.west.isilon.com/eng-axp/isi_sdk_python`](https://github.west.isilon.com/eng-axp/isi_sdk_python)
You can use the SDK Python language bindings to automate the configuration, maintenance, and monitoring of your Isilon cluster. For information on how to install the Python language bindings and write Python scripts to access the OneFS API, refer to the following Github sites:

[`https://github.com/Isilon/isilon_sdk_7_2_python`](https://github.com/Isilon/isilon_sdk_7_2_python)
[`https://github.com/Isilon/isilon_sdk_8_0_python`](https://github.com/Isilon/isilon_sdk_8_0_python)


Copyright (c) 2016 EMC Corporation

Expand All @@ -59,4 +83,3 @@ AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.

3 changes: 1 addition & 2 deletions build_stat_browser.py
Original file line number Diff line number Diff line change
Expand Up @@ -38,8 +38,7 @@ def build_browser():
(opts, args) = parser.parse_args()

if opts.cluster is None:
print('--cluster <IP> is required')
sys.exit(1)
opts.cluster = input('Cluster IP or hostname: ')

if opts.debug:
log_lvl = logging.DEBUG
Expand Down
Loading

0 comments on commit b0b42ed

Please sign in to comment.