-
Notifications
You must be signed in to change notification settings - Fork 0
/
setup.txt
96 lines (76 loc) · 3.65 KB
/
setup.txt
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
pip package requirments:
- pyxml
- boto
- boto3
Stage1:
-------
Requirements: To create python script which produce ansible script converted from given bash script with some basic commands
Implementation:
Code parse line by line of script and check for given commands, create ansible compatible yaml with corresponding module, write to single file ansible.yml
Usage:
Stage 1 is tested using static inventory to reach to hosts using ip,
1. Put bash_history in existing directory, and execute ansible-gen.py
2. python ansible-gen.py
3. It will generate ansible.yml in existing directory, which can be execute by ansible
4. Need to change ip in hosts file in repo directory
5. Use ansible_execute_stage1.sh to execute playbook, (change key name and its location accordingly)
Stage 1 Basic commands:
-----------------------
1. apt-get
a) install: implemented mutiple packages support like: apt-get install -y pkg1 pkg2 pkg3 pkg4
b) apt-get update
c) apt-get dist-upgrade
2. pip
a) pip install -r requirements.txt
Note: requirements.txt will pick from current working directory at remote after 'cd'.
3. echo
a) echo /dev/xvdp /media/scraped/ ext4 auto 0 1 >> /etc/fstab
Implemented using shell module
4. cd
a) cd path/to/some_dir
Implemented using 'cwd' fact variable, all related task will use this to find current working directory
5. chown:
a) chown ubuntu:ubuntu /media/scraped
Implemented using chown ansible module, support -R recursive
6. mkdir:
a) mkdir /media/scraped
Support multiple directories creation (using ansible loop)
Support -p option, to create tree
7. mount:
a) mount /media/scraped
Currently support only single argument command, i.e should be in fstab already
8. rm:
a) rm -rf html logs output_csv
Support file and recursive(directory) delete, also support multiple dir deletion, using ansible loop
Stage2:
-------
Part A: Save existing instance config:
-------------------------------------
ec2_config_fetch.py script is use to save existing ec2 instance config to yaml file, which can further use by ansible to create similar instance. Config parameters can be modify according to requirements
Usage:
1. Define aws access/secret keys in ~/.aws/credentials, boto3 will read from that.
2. python ec2_config_fetch.py --name=webserver
It will generate in ec2_config.yml in existing directory, which will further be use by ec2.yml
Part B: Execute ansible with dynamic inventory:
-----------------------------------------------
Dynamic inventory script ec2.py and ec2.ini are added in repo
1. Make sure following has been added to ~/.ansible.cfg
[defaults]
host_key_checking = False
2. Put aws access/secret key in ~/.aws/credential
3. change config and key name and path in ec2_config.yml
4. Use ansible_execute_stage2.sh to execute playbook, change key name/location accordingly.
It will create instance (based on config saved in ec2_config.yml generate by ec2_config_fetch.py) and execute ansible.yml script (generated by ansible-gen.py) on created instance
Part C: Additional command:
---------------------------
9: git:
a) git clone git@github.com:mgajda/scraper-python.git
Implemented using 'shell' right, because I didn;t know exact cases right now. Can use ansible module as well
All operation are execute in current working directory on remote (applicable for only 'clone' case)
10: scp:
a) scp file.txt hostname:/some_path
Simple file copy case on remote absolute path
b) scp -r keys_dir hostname:~/.ssh/
Directory copy case on remote relative path (somewhere in home directory of user)
c) scp -r dir/* hostname:~/target
Wildcard, copy every things inside dir to remote target