New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Feature/infra proxy pipeline #6617

Closed

GorillaGigabytes wants to merge 140 commits into feature/infra_user_management from feature/infra_proxy_pipeline

Contributor

GorillaGigabytes commented Jan 25, 2022 •

edited

Loading

🔩 Description: What code changed, and why?

Create a Demo pipeline for extraction and parsing of zip file and saving all the required info into the db.

⛓️ Related Resources

Migration Data Service, upload of zip file

👍 Definition of Done

Gat a demo pipeline ready which communicates using channels, from extraction of a zip file to storage all the info to the database.

👟 How to Build and Test the Change

rebuild components/infra-proxy-service/

Only Unzip routine is working flawlessly, and rest of the function, routines needs to be updated with actual data to test and run.

✅ Checklist

All PRs must tick these:

I have read the CONTRIBUTING document.
All commits signed-off for the Developer Certification of Origin.

With occasional exceptions, all PRs from Progress employees must tick these:

Is the code clear? (complicated code or lots of comments--subdivide and use well-named methods, meaningful variable names, etc.)
Consistency checked? (user notifications, user prompts, visual patterns, code patterns, variable names)
Repeated code blocks eliminated? (adapt and reuse existing components, blocks, functions, etc.)
Spelling, grammar, typos checked? (at a minimum use make spell in any component directory)
Code well-formatted? (indents, line breaks, etc. improve rather than hinder readability)

All PRs from Progress employees should tick these if appropriate:

Tests added/updated? (all new code needs new tests)
Docs added/updated? (all customer-facing changes)

Please add a note next to any checkbox above if you are NOT ticking it.

📷 Screenshots, if applicable

GorillaGigabytes added automate-infra-proxy-api infra-user-mgnt labels

GorillaGigabytes requested review from kalroy, vinay033 and sonali523

January 25, 2022 06:49

GorillaGigabytes self-assigned this

GorillaGigabytes force-pushed the feature/infra_proxy_pipeline branch from b6126a6 to b52bf0e Compare

January 25, 2022 10:49

sonali523 and others added 23 commits

January 27, 2022 12:34


          Script added for table 'users' (#5722)

c3c282f

* Script added for table 'users'

Signed-off-by: root <swale@msystechnologies.com>

* updated down script

Signed-off-by: root <swale@msystechnologies.com>

* updated up script

Signed-off-by: root <swale@msystechnologies.com>

* updated up script

Signed-off-by: root <swale@msystechnologies.com>


          iam added on roles tab

f5d2180

Signed-off-by: Abdul-Az <aazeez@progress.com>


          add fallback and bldr channel env vars to envrc and set back to dev w…

36bcc3d

…hen doing builds (#5741)

Signed-off-by: Rick Marry <rmarry@chef.io>


          Script added for table 'org_users' (#5738)

ed1da75

Signed-off-by: root <swale@msystechnologies.com>


          added edit policy

452be0d

Signed-off-by: Abdul-Az <aazeez@progress.com>


          Storage functions for users table (#5731)

404b274

* Storage functions added for users table

Signed-off-by: root <swale@msystechnologies.com>

* Updated with review comments

Signed-off-by: root <swale@msystechnologies.com>


          Chaitali/add server slider (#5728)

* converted server modal into slider

Signed-off-by: chaitali-mane <cmane@progress.com>

* Updated test cases

Signed-off-by: chaitali-mane <cmane@progress.com>


          environments create, delete policy added

Signed-off-by: Abdul-Az <aazeez@progress.com>


          edit env policy added

c766927

Signed-off-by: Abdul-Az <aazeez@progress.com>


          added data bags policy

50c0b36

Signed-off-by: Abdul-Az <aazeez@progress.com>


          Chaitali/add server slider (#5728)

f0e5902

* converted server modal into slider

Signed-off-by: chaitali-mane <cmane@progress.com>

* Updated test cases

Signed-off-by: chaitali-mane <cmane@progress.com>


          ui three dots fix

417cf07

Signed-off-by: Abdul-Az <aazeez@progress.com>


          reverted iam actions

9f592ec

Signed-off-by: Abdul-Az <aazeez@progress.com>


          Add changes for the create server slider (#5755)

3c79c3d

* added some changes for the create server slider

Signed-off-by: Vinay Sharma <vsharma@chef.io>

* added some minor changes

Signed-off-by: Vinay Sharma <vsharma@chef.io>


          removed roles iam

ed49fb8

Signed-off-by: Abdul-Az <aazeez@progress.com>


          environments authz reverted

8a0a0a9

Signed-off-by: Abdul-Az <aazeez@progress.com>


          databags authz reverted

cbe2ad1

Signed-off-by: Abdul-Az <aazeez@progress.com>


          Added API to get server users list from backend DB (#5760)

69bdb37

Signed-off-by: root <swale@msystechnologies.com>


          Added users tab in server details page (#5802)

2aea513

* Added Users tab UI

Signed-off-by: chaitali-mane <cmane@progress.com>

* Added enitites api integration

Signed-off-by: chaitali-mane <cmane@progress.com>

* minor changes

Signed-off-by: chaitali-mane <cmane@progress.com>

* Updated minor change

Signed-off-by: chaitali-mane <cmane@progress.com>


          Added css change for notification overlap slider. (#5781)

8704e44

Signed-off-by: Himanshi Chhabra <hchhabra@progress.com>


          Command to add the sample automate infra server users (#5786)

4c79374

* Shell script added to add the sample infra server users

Signed-off-by: root <swale@msystechnologies.com>

* Minor changes added

Signed-off-by: root <swale@msystechnologies.com>


          Renaming the sql files and modifying to incorporate WebUI key

e34f440

Signed-off-by: Kallol Roy <karoy@progress.com>


          infra_server_actions tests

ac9165f

Signed-off-by: Abdul-Az <aazeez@progress.com>

sonali523 and others added 19 commits

January 27, 2022 12:34


          Infra proxy API to get the organisation of chef server (#6363)

146e139

* Get organisation of infra server from chef server API added

Signed-off-by: sonali wale <swale@msystechnologies.com>

* minor changes added

Signed-off-by: sonali wale <swale@msystechnologies.com>

* Test cases added

Signed-off-by: sonali wale <swale@msystechnologies.com>

* Changes added in test cases

Signed-off-by: sonali wale <swale@msystechnologies.com>

* Added some changes in test cases

Signed-off-by: sonali wale <swale@msystechnologies.com>

* Removed test cases to check the existing one

Signed-off-by: sonali wale <swale@msystechnologies.com>

* added test cases changes

Signed-off-by: sonali wale <swale@msystechnologies.com>

* incorporating review comments

Signed-off-by: sonali wale <swale@msystechnologies.com>

* Minor changes added in test cases

Signed-off-by: sonali wale <swale@msystechnologies.com>

* added some logs

Signed-off-by: sonali wale <swale@msystechnologies.com>

* changes added in test cases

Signed-off-by: sonali wale <swale@msystechnologies.com>

* test cases changes added

Signed-off-by: sonali wale <swale@msystechnologies.com>

* Fixes added for repo health status

Signed-off-by: sonali wale <swale@msystechnologies.com>

* Fix added for repo health failure

Signed-off-by: sonali wale <swale@msystechnologies.com>

* Changes added in bash script of add users to remove credential id from users table

Signed-off-by: sonali wale <swale@msystechnologies.com>

* Conflicts resolved

Signed-off-by: sonali wale <swale@msystechnologies.com>


          Server details page - UI should have the option to update the WebUI k…

ae92fba

…ey (#6440)

* added changes to update the webui key on server details page

Signed-off-by: Vinay Sharma <vsharma@chef.io>

* added some minor changes

Signed-off-by: Vinay Sharma <vsharma@chef.io>


          Show the validation status of the WebUI key on the Chef Server Metada…

b1b2b5b

…ta (#6435)

* changes for the validate webui key on server details page

Signed-off-by: Vinay Sharma <vsharma@chef.io>

* added chanegs for the validate webui key

Signed-off-by: Vinay Sharma <vsharma@chef.io>

* added changes for the specs

Signed-off-by: Vinay Sharma <vsharma@chef.io>

* added changes for the validate webui key error effect

Signed-off-by: Vinay Sharma <vsharma@chef.io>


          API to update the web ui key (#6433)

9a613a7

* Update web ui key API added

Signed-off-by: sonali wale <swale@msystechnologies.com>

* Removed extra files

Signed-off-by: sonali wale <swale@msystechnologies.com>

* Read secret changes added

Signed-off-by: sonali wale <swale@msystechnologies.com>

* Conflicts resolved

Signed-off-by: sonali wale <swale@msystechnologies.com>

* Test cases added

Signed-off-by: sonali wale <swale@msystechnologies.com>

* Commented validate web ui key test cases

Signed-off-by: sonali wale <swale@msystechnologies.com>

* Changes added in mock file

Signed-off-by: sonali wale <swale@msystechnologies.com>

* uncomment validate web ui key test cases

Signed-off-by: sonali wale <swale@msystechnologies.com>


          infra_viewer_actions added (#6436)

53c1033

Signed-off-by: Abdul-Az <aazeez@progress.com>


          Abdul/infra editor actions (#6453)

6a3441f

* editor actions added

Signed-off-by: Abdul-Az <aazeez@progress.com>

* infra_editor test cases

Signed-off-by: Abdul-Az <aazeez@progress.com>

* test cases fix

Signed-off-by: Abdul-Az <aazeez@progress.com>


          Changes to use web ui key in all the Infra Proxy APIs for Cookbooks, …

c227718

…Policy Files, Policy Groups, Clients (#6457)

* Changes added to use the web ui key

Signed-off-by: sonali wale <swale@msystechnologies.com>

* added minor UI change

Signed-off-by: Chaitali Mane <cmane@progress.com>

Co-authored-by: Chaitali Mane <cmane@progress.com>


          Database script for migration tables (#6500)

5c7fdd6

* Script added for migration tables

Signed-off-by: sonali wale <sonali.wale@progress.com>

* Insert scripts added for migration type and status

Signed-off-by: sonali wale <sonali.wale@progress.com>

* Server id column added in migration table

Signed-off-by: sonali wale <sonali.wale@progress.com>

* Index added on server_id column in migration table

Signed-off-by: sonali wale <sonali.wale@progress.com>


          Abdul/infra editor actions (#6508)

59aeff4

* editor actions added

Signed-off-by: Abdul-Az <aazeez@progress.com>

* infra_editor test cases

Signed-off-by: Abdul-Az <aazeez@progress.com>

* test cases fix

Signed-off-by: Abdul-Az <aazeez@progress.com>

* infra editor actions update

Signed-off-by: Abdul-Az <aazeez@progress.com>

* editor role fix

Signed-off-by: Abdul-Az <aazeez@progress.com>

* test cases fix

Signed-off-by: Abdul-Az <aazeez@progress.com>


          Abdul/infra project owner (#6512)

be3dd15

* project owner wip

Signed-off-by: Abdul-Az <aazeez@progress.com>

* project owner actions

Signed-off-by: Abdul-Az <aazeez@progress.com>

* fix

Signed-off-by: Abdul-Az <aazeez@progress.com>

* doc update

Signed-off-by: Abdul-Az <aazeez@progress.com>

* pipeline fix

Signed-off-by: Abdul-Az <aazeez@progress.com>


          Updated Infra Proxy cypress tests for web UI key (#6511)

56ef9ef

* Updated cypress tests for web ui key

Signed-off-by: Chaitali Mane <cmane@progress.com>

* Key in uppercase

Signed-off-by: Chaitali Mane <cmane@progress.com>

* Update for minor

Signed-off-by: Chaitali Mane <cmane@progress.com>

* removed unused lines

Signed-off-by: Chaitali Mane <cmane@progress.com>

* removed error unit tests

Signed-off-by: Chaitali Mane <cmane@progress.com>

* removed iam lint error

Signed-off-by: Chaitali Mane <cmane@progress.com>

* Adding Cypress env variable for WebUI key

Signed-off-by: Kallol Roy <karoy@progress.com>

* Updated variable

Signed-off-by: Chaitali Mane <cmane@progress.com>

* removed replace

Signed-off-by: Chaitali Mane <cmane@progress.com>

* Adding variable

Signed-off-by: Chaitali Mane <cmane@progress.com>

* minor change

Signed-off-by: Chaitali Mane <cmane@progress.com>

* removed hard coded values

Signed-off-by: Chaitali Mane <cmane@progress.com>

* Removed hard coded values

Signed-off-by: Chaitali Mane <cmane@progress.com>

* Updated variables

Signed-off-by: Chaitali Mane <cmane@progress.com>

Co-authored-by: Kallol Roy <karoy@progress.com>


          New version of protoc and added migrations proto files (#6578)

4b6991b

Signed-off-by: Yashvi Jain <Yashvi.jain@progress.com>


          Added sync organization and user button (#6592)

98639c7

* updated docs

Signed-off-by: Abdul-Az <aazeez@progress.com>

* cypress error fix

Signed-off-by: Abdul-Az <aazeez@progress.com>

* lint fixes

Signed-off-by: Abdul-Az <aazeez@progress.com>

* updated docs

Signed-off-by: Abdul-Az <aazeez@progress.com>

* cypress error fix

Signed-off-by: Abdul-Az <aazeez@progress.com>

* Added button as per new design

Signed-off-by: Chaitali Mane <cmane@progress.com>

* update change

Signed-off-by: Chaitali Mane <cmane@progress.com>

Co-authored-by: Abdul-Az <aazeez@progress.com>


          Creating a new package migrations (#6595)

80ff2ba

Signed-off-by: Yashvi Jain <Yashvi.jain@progress.com>


          Storage functions for the migration phases (#6529)

7a002d5

* Get organisation API changes added

Signed-off-by: sonali wale <sonali.wale@progress.com>

* minor changes added

Signed-off-by: sonali wale <sonali.wale@progress.com>

* Remove command to create an orgs

Signed-off-by: sonali wale <sonali.wale@progress.com>

* Client Mock changes

Signed-off-by: Yashvi Jain <Yashvi.jain@progress.com>

* Storage function for all the phases of migration

Signed-off-by: sonali wale <sonali.wale@progress.com>

* Minor changes added

Signed-off-by: sonali wale <sonali.wale@progress.com>

* Incorporated review comments

Signed-off-by: sonali wale <sonali.wale@progress.com>

* Minor changes added

Signed-off-by: sonali wale <sonali.wale@progress.com>

* Changes for review comments

Signed-off-by: sonali wale <sonali.wale@progress.com>

* Comments removed

Signed-off-by: sonali wale <sonali.wale@progress.com>

Co-authored-by: Yashvi Jain <Yashvi.jain@progress.com>


          Migration stage script and storage functions added (#6619)

20ccc72

Signed-off-by: sonali wale <sonali.wale@progress.com>


          Addition of migration id and migration status in get server api (#6613)

4dbdfa8

* resolving conflicts and rebasing it

Signed-off-by: Yashvi Jain <Yashvi.jain@progress.com>

* Removing the loggers

Signed-off-by: Yashvi Jain <Yashvi.jain@progress.com>

* Query changes for lastes migration status

Signed-off-by: Yashvi Jain <Yashvi.jain@progress.com>

* changing the response varibale

Signed-off-by: Yashvi Jain <Yashvi.jain@progress.com>


          Changes to audit the log for infra proxy (#6622)

c5978ad

* added changes for infra proxy audit log

* added reflect codes

Signed-off-by: Kallol Roy <karoy@progress.com>

* added changes for the audit logs for infra

Signed-off-by: Vinay Sharma <vsharma@chef.io>

* removed commented code

Signed-off-by: Vinay Sharma <vsharma@chef.io>

* added some minor changes

Signed-off-by: Vinay Sharma <vsharma@chef.io>

Co-authored-by: Kallol Roy <karoy@progress.com>


          Migration Data service creation and created a rpc method for upload z…

7fda7f0

…ip file (#6612)

* Changes for stream api for upload

Signed-off-by: Yashvi Jain <Yashvi.jain@progress.com>

* changes

Signed-off-by: Yashvi Jain <Yashvi.jain@progress.com>

* added some changes for the protofile

Signed-off-by: Vinay Sharma <vsharma@chef.io>

* added function changes

* added changes for the proto

* added changes to devide the chunks in bytes

* added changes for the validation

Signed-off-by: Vinay Sharma <vsharma@chef.io>

* adding status in database

Signed-off-by: Yashvi Jain <Yashvi.jain@progress.com>

* adding the request for server id and filename

Signed-off-by: Yashvi Jain <Yashvi.jain@progress.com>

* Adding  loggers and saving the failed migration phase

Signed-off-by: Yashvi Jain <Yashvi.jain@progress.com>

* optimising the code

Signed-off-by: Yashvi Jain <Yashvi.jain@progress.com>

* Changes for content-type

Signed-off-by: Yashvi Jain <Yashvi.jain@progress.com>

* added changes for the validation

Signed-off-by: Vinay Sharma <vsharma@chef.io>

* Upload to the hab svc data folder

Signed-off-by: Kallol Roy <karoy@progress.com>

* Enhance log and modify migration phases

Signed-off-by: Kallol Roy <karoy@progress.com>

* Granularize the migration stages

Signed-off-by: Kallol Roy <karoy@progress.com>

* Granularize the migration stages

Signed-off-by: Kallol Roy <karoy@progress.com>

Co-authored-by: Vinay Sharma <vsharma@chef.io>
Co-authored-by: Kallol Roy <karoy@progress.com>

vinay033 force-pushed the feature/infra_user_management branch from a3618d2 to 7fda7f0 Compare

January 27, 2022 07:04

Pappu Kumar added 2 commits

January 28, 2022 13:38


          Gotten demo pipeline ready

804b88a

Signed-off-by: Pappu Kumar <pappu.kumar@progress.com>


          Data pipeline architecture

c1ba310

Signed-off-by: Pappu Kumar <pappu.kumar@progress.com>

GorillaGigabytes force-pushed the feature/infra_proxy_pipeline branch from a4cda88 to c1ba310 Compare

January 28, 2022 08:09

Pappu Kumar and others added 2 commits

January 28, 2022 13:41


          Removed a file

8d07dc1

Signed-off-by: Pappu Kumar <pappu.kumar@progress.com>


          Brainstorming ideas

dceab3a

Signed-off-by: Kallol Roy <karoy@progress.com>

sonarcloud bot commented Jan 28, 2022

Kudos, SonarCloud Quality Gate passed!

0 Bugs
0 Vulnerabilities
0 Security Hotspots
1 Code Smell

0.0% Coverage
0.0% Duplication

vinay033 force-pushed the feature/infra_user_management branch from 19df40a to 15dafca Compare

January 31, 2022 17:54

Contributor Author

GorillaGigabytes commented Feb 9, 2022

I have raised a different PR hence closing this one

GorillaGigabytes closed this

GorillaGigabytes deleted the feature/infra_proxy_pipeline branch

February 16, 2022 08:57

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

automate-infra-proxy-api infra-user-mgnt