Skip to content
This repository has been archived by the owner on Feb 3, 2021. It is now read-only.

Feature: getting started script #475

Merged
merged 37 commits into from
Apr 11, 2018
Merged
Show file tree
Hide file tree
Changes from 28 commits
Commits
Show all changes
37 commits
Select commit Hold shift + click to select a range
ca499b2
initial changes for getting started scripts
paselem Mar 23, 2018
62c1518
add temp error handling
paselem Mar 26, 2018
4754ca3
rename file - fix typo
paselem Mar 26, 2018
3f05898
add debug strings
paselem Mar 26, 2018
9bac57f
add handling for existing user
paselem Mar 26, 2018
9ab8810
WIP: wait for subprocess to complete to get exit code
paselem Mar 26, 2018
a7076bf
WIP: handle existing user and refactor code
paselem Mar 26, 2018
d93e73e
WIP: add missing return statements
paselem Mar 26, 2018
8c80ba0
WIP: fix typo
paselem Mar 26, 2018
e6b7d53
start sdk refactor
jafreck Mar 29, 2018
23ff3e2
mostly working create
jafreck Mar 30, 2018
2576393
working happy create path
jafreck Mar 30, 2018
010b8c1
handle errors for vnet, aad application
jafreck Apr 3, 2018
e33e0c8
make account setup interactive
jafreck Apr 3, 2018
746e7d0
add prompt
jafreck Apr 3, 2018
51997e6
add docs
jafreck Apr 3, 2018
d34b05a
rename account_setup_refac to account_setup
jafreck Apr 3, 2018
742dc6b
Merge branch 'master' into feature/getting_started_scripts
jafreck Apr 3, 2018
d4c261a
add some logging
jafreck Apr 3, 2018
c7cf1dc
Merge branch 'feature/getting_started_scripts' of github.com:azure/az…
jafreck Apr 3, 2018
1049419
pip install msrest, azure-cli-core, import issues
jafreck Apr 4, 2018
b59848b
remove in script pip, add shell wrapper program
jafreck Apr 4, 2018
96dd518
ellipsis to period
jafreck Apr 4, 2018
29b4111
update branch name for account_setup.sh
jafreck Apr 4, 2018
35c5ee9
docstring
jafreck Apr 4, 2018
3851dd4
retry resource group creation
jafreck Apr 5, 2018
56f1919
fix typo, update retry
jafreck Apr 5, 2018
3616f0a
explicitly set output location
jafreck Apr 5, 2018
a6cbd56
wget overwrite flag, docs update
jafreck Apr 5, 2018
84a4bc2
add prompt for multi tenants
jafreck Apr 5, 2018
4ebc4f3
fix bug with batch account creation
jafreck Apr 5, 2018
75b039e
Merge branch 'master' into feature/getting_started_scripts
jafreck Apr 6, 2018
60bdc18
Merge branch 'master' into feature/getting_started_scripts
jafreck Apr 6, 2018
30fb1ae
Merge branch 'master' into feature/getting_started_scripts
jafreck Apr 9, 2018
76e95f5
add spinner, print statements, fix formatting bug
jafreck Apr 10, 2018
d0c7a41
Merge branch 'master' into feature/getting_started_scripts
jafreck Apr 10, 2018
27e45d8
fix param bug
jafreck Apr 10, 2018
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
22 changes: 13 additions & 9 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,28 +16,32 @@ This toolkit is built on top of Azure Batch but does not require any Azure Batch
- Ability to run _spark submit_ directly from your local machine's CLI

## Setup

1. Clone the repo
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are we comfortable enough to just pip install? Technically it should be the same thing, right??

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

technically, yes. But I elected not to make that change here until our release on pip is "official". I can do it now if you'd prefer.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it just seems so much easier now... even if the version shows up as '0.6.1b' for example...

```bash
git clone -b stable https://www.github.com/azure/aztk

# You can also clone directly from master to get the latest bits
git clone https://www.github.com/azure/aztk
```
2. Use pip to install required packages (requires python 3.5+ and pip 9.0.1+)
```bash
pip install -r requirements.txt
```
3. Use setuptools:

3. Install `aztk`:
```bash
pip install -e .
```
4. Initialize the project in a directory [This will automatically create a *.aztk* folder with config files in your working directory]:
4. Initialize the project in a directory. This will automatically create a *.aztk* folder with config files in your working directory:
```bash
aztk spark init
```
5. Fill in the fields for your Batch account and Storage account in your *.aztk/secrets.yaml* file. (We'd also recommend that you enter SSH key info in this file)
5. Login or register for an [Azure Account](https://azure.microsoft.com), navigate to [Azure Cloud Shell](https://shell.azure.com), and run:
```sh
wget -q https://raw.githubusercontent.com/Azure/aztk/master/account_setup.sh &&
chmod 755 account_setup.sh &&
/bin/bash account_setup.sh
```

6. Follow the on screen prompts to create the necessary Azure resources and copy the output into your `.aztk/secrets.yaml` file. For more infomration see [Getting Started Scripts](./01-Getting-Started-Script).

This package is built on top of two core Azure services, [Azure Batch](https://azure.microsoft.com/en-us/services/batch/) and [Azure Storage](https://azure.microsoft.com/en-us/services/storage/). Create those resources via the portal (see [Getting Started](./docs/00-getting-started.md)).

## Quickstart Guide

Expand All @@ -55,7 +59,7 @@ aztk spark cluster list
aztk spark cluster delete
```
```sh
# login and submit jobs to your cluster
# login and submit applications to your cluster
aztk spark cluster ssh
aztk spark cluster submit
```
Expand Down
Loading