Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Migrate to pydantic #214

Merged
merged 99 commits into from
Oct 8, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
99 commits
Select commit Hold shift + click to select a range
368a6aa
Initial pydantic testing.
terjekv Jan 29, 2024
faeb23c
Implement `host find`.
terjekv Apr 9, 2024
9718859
Clean up authentication mess. (#206)
terjekv Jan 30, 2024
51528cd
Fix `host find -comment` metavar (#207)
pederhan Feb 7, 2024
8d0b673
Allow oneshot commands from the terminal. (#208)
terjekv Feb 12, 2024
6c66d5f
Improve connection errors. (#209)
terjekv Feb 19, 2024
1964597
Fix a bug where not having any default URL breaks application start. …
terjekv Feb 20, 2024
6215a6c
Use corrolation_id for logging across commands. (#213)
terjekv Feb 21, 2024
3838470
Make `host remove` a bit safer to use. (#211)
terjekv Feb 29, 2024
683379a
Merge branch 'master' into migrate_to_pydantic
terjekv Apr 9, 2024
5457b3d
Add Endpoint enum, migrate add_user, refactor MacAddress.
terjekv Apr 9, 2024
28c696d
Make api.get_host generic.
terjekv Apr 15, 2024
181e811
Refactoring for readability.
terjekv Apr 15, 2024
100fdc8
Remove redudant docstring.
terjekv Apr 15, 2024
ad8a341
Rename HostModel to Host.
terjekv Apr 15, 2024
071e69f
Refactoring and cleaning up host processing.
terjekv Apr 17, 2024
ce243bd
Typing, Zones and some more RRs.
terjekv Apr 18, 2024
dd6f21c
Use class methods for lookups, object methods for API operations.
terjekv Apr 20, 2024
d6708f6
Fix host comparison.
terjekv Apr 20, 2024
684ace3
Type fixes.
terjekv Apr 20, 2024
0cc061d
Move knowledge of endpoint ID mapping peculiarities to Endpoints only…
terjekv Apr 20, 2024
a66a256
Allow for manually setting ip or pass network to host add.
terjekv Apr 22, 2024
ccd6f6c
Look up field names by alias
pederhan Apr 22, 2024
b8fbce5
Use dict.get() with no default
pederhan Apr 22, 2024
eb37282
Make pylance happy with explicit type
terjekv Apr 22, 2024
f0eb55c
Split the models.py into smaller files.
terjekv Apr 23, 2024
2f2cd6c
Add a run script for pyinstaller.
terjekv Apr 23, 2024
001b980
Either raise an exception *or* use cli_{warning,error}.
terjekv Apr 23, 2024
1cf92ae
Fix host add (again).
terjekv Apr 23, 2024
c1eee08
Upgrade models.py to Pydantic V2 semantics (#215)
pederhan Apr 23, 2024
1066e98
Migrate to python 3.11.
terjekv Apr 23, 2024
0fc7df5
Require setuptools due to python 3.12+
terjekv Apr 23, 2024
b753df4
Require a newer version of setuptools.
terjekv Apr 23, 2024
af5d22e
Newer image, force upgrade of pip, and drop caches,
terjekv Apr 23, 2024
8c1ca58
Use tox to test pyinstaller for 3.11 and 3.12.
terjekv Apr 23, 2024
0af325a
Add `from __future__ import annotations` lint rule (#216)
pederhan Apr 24, 2024
3fb1b1a
Finish up the last bits of core host commands...
terjekv Apr 24, 2024
58edfe1
Add Host.get_by_any_means_or_raise and use it wisely. :)
terjekv Apr 24, 2024
8080aff
Acutally support multi-host lookups with info and fix bacnet display.
terjekv Apr 24, 2024
f95ff73
Add group lookup support to host info.
terjekv Apr 24, 2024
0c558b1
Refactor hostgroup parent parsing.
terjekv Apr 24, 2024
102c345
Refactor and add group output support.
terjekv Apr 25, 2024
4073a62
Refactor away the function-based API and migrated a/aaaa commands.
terjekv Apr 26, 2024
a1d4f6c
host cname commands migration.
terjekv Apr 27, 2024
3523122
Hinfo support.
terjekv Apr 27, 2024
9f892b6
Well, that was effin' awful.
terjekv Apr 27, 2024
9e73353
Add RR MX support.
terjekv Apr 29, 2024
13eae87
Add `get_typed()` utility function
pederhan May 2, 2024
aed9364
Add NAPTR and PTR support.
terjekv May 3, 2024
2914d45
Add `Atom` & `Label` models + base Host Policy model (#217)
pederhan May 3, 2024
be67d3e
Add SRV support.
terjekv May 3, 2024
fdf42cc
Finalize RR support.
terjekv May 3, 2024
38f7353
Add inline comments for `HostPolicy` validator
pederhan May 5, 2024
34f0bdc
Add `WithName` mixin (#218)
pederhan May 7, 2024
ad4e1b5
Better exception modelling. (#219)
terjekv May 8, 2024
71dddd1
Use `Self` type annotation for `APIMixin` (#222)
pederhan May 8, 2024
98ea140
dhcp commands and exception usage. (#223)
terjekv May 10, 2024
7123311
Uniform exception names (#225)
terjekv May 12, 2024
da3335a
Add policy commands (#220)
pederhan May 13, 2024
8d711f9
Pydantic group commands (#224)
terjekv May 13, 2024
b7c7370
Labels and some permission work. (#226)
terjekv May 14, 2024
b79a1b0
Permission commands migrated. (#228)
terjekv May 21, 2024
4877e83
Zone commands (#227)
pederhan May 21, 2024
dd51e64
Fix incorrect method used in `host sshfp_remove` (#233)
pederhan May 21, 2024
f08aaca
Support segmenting tokens per user and url. (#229)
terjekv May 21, 2024
d482bde
Set prompt to default to the server one is connected to. (#234)
terjekv May 21, 2024
4a6f262
Remove `use_json` (#235)
pederhan May 21, 2024
29d6d81
Fix/standardize some patch commands (#236)
pederhan May 22, 2024
cd156d1
Add `WithHistory` + fetching history for deleted objects (#237)
pederhan May 22, 2024
a2fb43b
Network commands (#238)
pederhan May 24, 2024
2de44b6
Add pagination support for `get_typed` (#240)
pederhan May 27, 2024
bf8c1d3
Define metadata and requirements in pyproject.toml (#239)
pederhan May 27, 2024
248a19e
Fix broken command(s) in CI (#243)
pederhan May 27, 2024
db17b84
Fix missing pyinstaller dev dependency (#246)
pederhan May 28, 2024
07cb471
Fix DHCP commands (#245)
pederhan May 28, 2024
7bbc3da
Add period when finding subzones (#248)
pederhan May 29, 2024
e1c333e
Validate GET responses with Pydantic (#241)
pederhan May 29, 2024
b5fac79
Fix typo
pederhan May 29, 2024
0c198c5
Strip None values when sending JSON (#249)
pederhan May 29, 2024
d234c36
Require SOCKS dependencies (#244)
pederhan May 30, 2024
4a35efe
Add `--version` option (#252)
pederhan May 30, 2024
b922b50
Show git commit in `--version` (#253)
pederhan May 31, 2024
02681f0
Copy policies from one host to another. (#257)
terjekv Jun 4, 2024
5dabdac
Support multiple destinations in host_copy. (#259)
terjekv Jun 6, 2024
54a3c09
Instantiate JSON mapping validator at top level (#260)
pederhan Jun 10, 2024
bb27ad1
Import host_submodules explicitly. (#262)
pederhan Jun 10, 2024
98cf819
Add publishing workflow and contribution instructions (#254)
pederhan Jun 12, 2024
7485ec1
Add authors to pyproject.toml (#263)
pederhan Jun 12, 2024
c5f46ca
Make testsuite work again (#255)
oyvindhagberg Sep 17, 2024
2729de6
Build Linux PyInstaller binary in CentOS 8 container (#293)
pederhan Sep 17, 2024
84aa6b8
Change publish tag pattern (#294)
pederhan Sep 17, 2024
0ba648c
Update contributing instructions
pederhan Sep 17, 2024
2853ce7
Add `getattr` default in `APIMixin.refetch` (#295)
pederhan Oct 2, 2024
b65ad63
Don't pass params for pagination requests (#296)
pederhan Oct 2, 2024
c36ad96
Reorder overloads (#297)
pederhan Oct 2, 2024
8d23297
Fix cli.py type checking errors (#299)
pederhan Oct 4, 2024
4e7b19a
Fix unquoted regex error handling (#298)
pederhan Oct 4, 2024
95356b2
Add error msg builder with underlined suggestions (#300)
pederhan Oct 7, 2024
b7d1a6c
Add defaults for url and domain config options (#301)
pederhan Oct 8, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
127 changes: 127 additions & 0 deletions .github/workflows/publish.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,127 @@
name: build mreg-cli

on:
push:
tags:
- '[0-9]+.[0-9]+.[0-9]+**'

concurrency:
group: build-mreg-cli-${{ github.head_ref }}

jobs:
build_pypi:
name: Build wheels and source distribution
runs-on: ubuntu-latest

steps:
- uses: actions/checkout@v4

- name: Exit if not on master branch
if: github.ref_name != 'master'
run: exit -1

- name: Install uv
uses: astral-sh/setup-uv@v2

- name: Set up Python 3.12
run: uv python install 3.12

- name: Install build dependencies
run: |
uv venv
uv pip install --upgrade build

- name: Build source distribution
run: uv run python -m build

- uses: actions/upload-artifact@v4
with:
name: pypi_artifacts
path: dist/*
if-no-files-found: error

build_pyinstaller:
name: Build pyinstaller binary
strategy:
matrix:
os: [ubuntu-latest, windows-latest, macos-latest]
python-version:
- '3.12'
include:
- os: ubuntu-latest
container: centos:8
runs-on: ${{ matrix.os }}
container: ${{ matrix.container }}

steps:
- uses: actions/checkout@v4

- name: Install uv
uses: astral-sh/setup-uv@v2

- name: Set up Python ${{ matrix.python-version }}
run: uv python install ${{ matrix.python-version }}

- name: Run PyInstaller with Tox
run: |
uv venv
uv pip install tox-uv tox-gh-actions
tox

- name: Rename binary
run: |
mv dist/mreg-cli${{ contains(matrix.os, 'windows') && '.exe' || '' }} dist/mreg-cli-${{ matrix.os }}-${{ matrix.python-version }}${{ contains(matrix.os, 'windows') && '.exe' || '' }}

- uses: actions/upload-artifact@v4
with:
name: mreg-cli-${{ matrix.os }}-${{ matrix.python-version }}${{ contains(matrix.os, 'windows') && '.exe' || '' }}
path: dist/mreg-cli-${{ matrix.os }}-${{ matrix.python-version }}${{ contains(matrix.os, 'windows') && '.exe' || '' }}
if-no-files-found: error

publish_pypi:
name: Publish PyPI release
needs:
- build_pypi
runs-on: ubuntu-latest
permissions:
id-token: write
steps:
- uses: actions/download-artifact@v4
with:
name: pypi_artifacts
path: dist

- name: Push build artifacts to PyPI
uses: pypa/gh-action-pypi-publish@v1.8.14

publish_github:
name: Publish GitHub release
needs:
- build_pypi
- build_pyinstaller
runs-on: ubuntu-latest

steps:
- name: Download PyInstaller binaries
uses: actions/download-artifact@v4
with:
pattern: mreg-cli-*
path: dist
merge-multiple: true

- name: Download wheel and source distributions
uses: actions/download-artifact@v4
with:
pattern: pypi_artifacts
path: dist
merge-multiple: true

- name: Create GitHub release
uses: softprops/action-gh-release@v2
with:
files: dist/*
body: |
Release ${{ github.ref_name }}
draft: false
prerelease: false

58 changes: 22 additions & 36 deletions .github/workflows/test.yml
Original file line number Diff line number Diff line change
@@ -1,42 +1,33 @@
on: [push, pull_request]


name: CI
jobs:
test:
name: Integration
runs-on: ${{ matrix.os }}
strategy:
matrix:
os: [ubuntu-20.04] # For python3.6
os: [ubuntu-22.04]
python-version:
- '3.6'
- '3.7'
- '3.8'
- '3.9'
- '3.10'
- '3.11'
- '3.12'
env:
# Disable colors and formatting in Rich console output
TERM: dumb
steps:
- name: Checkout
uses: actions/checkout@v3
- name: Cache pip
uses: actions/cache@v3
with:
path: ~/.cache/pip
key: v1-pip-${{ runner.os }}-${{ matrix.python-version }}-${{ hashFiles('requirements-*.txt') }}
restore-keys: |
v1-pip-${{ runner.os }}-${{ matrix.python-version }}
v1-pip-${{ runner.os }}
v1-pip-
- name: Setup Python
uses: actions/setup-python@v4
with:
python-version: ${{ matrix.python-version }}
uses: actions/checkout@v4
- name: Install uv
uses: astral-sh/setup-uv@v2
- name: Set up Python ${{ matrix.python-version }}
run: uv python install ${{ matrix.python-version }}
- name: Install mreg-cli
run: |
pip install -r requirements.txt
pip install -e .
uv venv
uv pip install -e .[dev]
- name: Test and compare api calls
run: ci/run_testsuite_and_record.sh
run: uv run ci/run_testsuite_and_record.sh

tox:
name: tox
Expand All @@ -45,24 +36,19 @@ jobs:
fail-fast: false
matrix:
python-version:
- "3.7"
- "3.8"
- "3.9"
- "3.10"
- "3.11"
- "3.12"

steps:
- uses: actions/checkout@v3
- uses: actions/checkout@v4
- name: Install uv
uses: astral-sh/setup-uv@v2
- name: Set up Python ${{ matrix.python-version }}
uses: actions/setup-python@v4
with:
python-version: ${{ matrix.python-version }}
run: uv python install ${{ matrix.python-version }}
- name: Install dependencies
run: |
python -m pip install --upgrade pip
python -m pip install tox tox-gh-actions
python -m pip install -r requirements.txt
python -m pip install -r requirements-dev.txt
uv venv
uv pip install tox-uv tox-gh-actions
- name: Test with tox
run: tox r
run: uv run tox r

1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,7 @@ wheels/
.installed.cfg
*.egg
MANIFEST
mreg_cli/_version.py

# PyInstaller
# Usually these files are written by a python script from a template
Expand Down
12 changes: 12 additions & 0 deletions AUTHORS
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
Dmytro Karpenko
Fredrik Larsen
Magnus Hirth
Marius Bakke
Nico
Nils Hiorth
Paal Braathen
Peder Hovdan Andresen
Safet Amedov
Terje Kvernes
Øyvind Hagberg
Øyvind Kolbu
36 changes: 36 additions & 0 deletions CHANGELOG
Original file line number Diff line number Diff line change
@@ -0,0 +1,36 @@
# Changelog

All notable changes to this project will be documented in this file.

The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).

## [Unreleased]

The big Pydantic update. The entire codebase has been rewritten to use Pydantic for request and response validation. This brings with it a huge improvement to the development experience and the robustness of the code.

### Added

- `--version` option to display current application version and exit.
Version number is in the form of `major.minor.patch` if installed from a published version.
Shows a version number including the commit hash if installed from a git repository in the form of `major.minor.patch.dev123+gabc1234`.
See [Default versioning scheme](https://setuptools-scm.readthedocs.io/en/latest/usage/#default-versioning-scheme) in the [setuptools_scm](https://github.com/pypa/setuptools_scm/) documentation for more information.
- The version can be accessed programmatically with `mreg_cli.__version__`.
- `label set_description` command to set the description of a label.
- `network list_excluded_ranges` command to list the excluded ranges of a network.
- Application can now store tokens for multiple servers and will pick the correct one based on the server URL.
- Building binaries for Windows, Linux and MacOS, and publishing the package to PyPI on each GitHub release.

### Changed

- The application now uses Pydantic internally to validate request and response data. This should make the code more robust and easier to maintain.
- Application now attempts to send JSON for every request. This should improve the consistency of the API responses.
- Version now follows Semantic Versioning 2.0.0 and is automatically determined based on the most recent git tag (`mreg-cli-v*`). As part of this change, the verison has been bumped from 0.9.10 to 1.0.0. See the Added section for more information on how version numbers are accessed.

### Removed

- `label rename -desc` option. Description modification is now done through the new `label set_description` command.

### Fixed

- Hopefully more than we broke.
38 changes: 38 additions & 0 deletions CONTRIBUTING
Original file line number Diff line number Diff line change
@@ -0,0 +1,38 @@
# Contributing

If you would like to contribute to this project, please follow these steps:

1. Fork the repository
2. Create a new branch (`git checkout -b feature/branch-name`)
3. Make changes
4. Commit your changes (`git commit -am 'Add some feature'`)
5. Push to the branch (`git push origin feature/branch-name`)
6. Create a Pull Request

## Publishing

Publishing new versions is automatically handled by GitHub actions for versions tagged with `x.y.z[.prerelease]` (`1.2.3`, `1.2.3.rc1`, etc.). If you are a maintainer, you can create a new release by following these steps:

1. Switch to the `master` branch:

```bash
git checkout master
```

2. Create a new tag:

```bash
git tag 1.2.3
```

3. Push the tag to the upstream repository:

```bash
git push upstream 1.2.3
```

To push a tag together with a new commit:

```bash
git push upstream master --tags
```
Loading