Update to clap 4.x #817

hitenkoku · 2022-11-19T07:11:47Z

What Changed

updated to clap v4

Note:

Colored output is currently not supported in clap v4.

However, it appears that v4 will be developed with high priority, so another pull-request will be made once colored display is supported.

ref: clap-rs/clap#4132 (comment)

Evidence

clap-rs/clap#4132 (comment)

main branch (216f9a0) result

 >./main.exe -d ..\hayabusa-sample-evtx\ -o main.csv -q

...

Analyzing event files: 581
Total file size: 148.5 MB

Loading detections rules. Please wait.

Excluded rules: 14
Noisy rules: 5 (Disabled)

Experimental rules: 1892 (58.89%)
Stable rules: 213 (6.63%)
Test rules: 1108 (34.48%)

Hayabusa rules: 141
Sigma rules: 3072
Total enabled detection rules: 3213

...

Results Summary:

Events with hits / Total events: 19,565 / 76,967 (Data reduction: 57,402 events (74.58%))

Total | Unique detections: 32,742 | 587
Total | Unique critical detections: 46 (0.14%) | 18 (3.07%)
Total | Unique high detections: 6,237 (19.05%) | 265 (45.14%)
Total | Unique medium detections: 1,585 (4.84%) | 176 (29.98%)
Total | Unique low detections: 6,626 (20.24%) | 75 (12.78%)
Total | Unique informational detections: 18,248 (55.73%) | 53 (9.03%)

Dates with most total detections:
critical: 2019-07-19 (15), high: 2016-09-20 (3,664), medium: 2021-04-22 (186), low: 2016-09-20 (3,777), informational: 2016-08-19 (2,105)

Top 5 computers with most unique detections:
critical: MSEDGEWIN10 (6), IEWIN7 (3), rootdc1.offsec.lan (2), srvdefender01.offsec.lan (2), FS03.offsec.lan (2)
high: MSEDGEWIN10 (119), IEWIN7 (72), FS03.offsec.lan (34), fs03vuln.offsec.lan (29), IE10Win7 (24)
medium: MSEDGEWIN10 (67), IEWIN7 (40), FS03.offsec.lan (17), IE10Win7 (15), PC01.example.corp (15)
low: MSEDGEWIN10 (33), IEWIN7 (16), FS03.offsec.lan (16), fs03vuln.offsec.lan (13), fs01.offsec.lan (11)
informational: MSEDGEWIN10 (18), IEWIN7 (17), fs01.offsec.lan (15), PC01.example.corp (13), FS03.offsec.lan (12)

╭───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╮
│ Top critical alerts:                                              Top high alerts:
                  │
├╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┤
│ Sticky Key Like Backdoor Usage (10)                               Metasploit SMB Authentication (3,562)               │
│ Active Directory Replication from Non Machine Account (6)         Malicious Svc Possibly Installed (271)              │
│ Meterpreter or Cobalt Strike Getsystem Service Installation (6)   Susp Svc Installed (257)
                  │
│ Defender Alert (Severe) (4)                                       PowerShell Scripts Installed as Services (253)      │
│ WannaCry Ransomware (4)                                           Suspicious Service Installation Script (250)        │
├╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┤
│ Top medium alerts:                                                Top low alerts:
                  │
├╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┤
│ Potentially Malicious PwSh (235)                                  Logon Failure (Wrong Password) (3,564)              │
│ Proc Injection (104)                                              Susp CmdLine (Possible LOLBIN) (1,418)              │
│ Reg Key Value Set (Sysmon Alert) (103)                            Non Interactive PowerShell (325)
                  │
│ Suspicious Remote Thread Target (93)                              Rare Service Installations (321)
                  │
│ Wscript Execution from Non C Drive (61)                           Windows Processes Suspicious Parent Directory (282) │
├╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┤
│ Top informational alerts:
                  │
├╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┤
│ Proc Exec (11,173)                                                Explicit Logon (342)
                  │
│ NetShare File Access (2,564)                                      Svc Installed (331)
                  │
│ PwSh Scriptblock (789)                                            New Non-USB PnP Device (268)                        │
│ PwSh Pipeline Exec (680)                                          Logon (Type 3 Network) (228)                        │
│ NetShare Access (433)                                             File Created (210)
                  │
╰─────────────────────────────────────────────────────────────────╌─────────────────────────────────────────────────────╯

Elapsed time: 00:00:10.736
Saved file: main.csv (16.3 MB)

this pull request results

>.\725.exe -d ..\hayabusa-sample-evtx\ -o 725.csv -q
...

Analyzing event files: 581
Total file size: 148.5 MB

Loading detections rules. Please wait.

Excluded rules: 14
Noisy rules: 5 (Disabled)

Experimental rules: 1892 (58.89%)
Stable rules: 213 (6.63%)
Test rules: 1108 (34.48%)

Hayabusa rules: 141
Sigma rules: 3072
Total enabled detection rules: 3213

...

Results Summary:

Events with hits / Total events: 19,565 / 76,967 (Data reduction: 57,402 events (74.58%))

Total | Unique detections: 32,742 | 587
Total | Unique critical detections: 46 (0.14%) | 18 (3.07%)
Total | Unique high detections: 6,237 (19.05%) | 265 (45.14%)
Total | Unique medium detections: 1,585 (4.84%) | 176 (29.98%)
Total | Unique low detections: 6,626 (20.24%) | 75 (12.78%)
Total | Unique informational detections: 18,248 (55.73%) | 53 (9.03%)

Dates with most total detections:
critical: 2019-07-19 (15), high: 2016-09-20 (3,664), medium: 2021-04-22 (186), low: 2016-09-20 (3,777), informational: 2016-08-19 (2,105)

Top 5 computers with most unique detections:
critical: MSEDGEWIN10 (6), IEWIN7 (3), srvdefender01.offsec.lan (2), rootdc1.offsec.lan (2), FS03.offsec.lan (2)
high: MSEDGEWIN10 (119), IEWIN7 (72), FS03.offsec.lan (34), fs03vuln.offsec.lan (29), IE10Win7 (24)
medium: MSEDGEWIN10 (67), IEWIN7 (40), FS03.offsec.lan (17), PC01.example.corp (15), IE10Win7 (15)
low: MSEDGEWIN10 (33), IEWIN7 (16), FS03.offsec.lan (16), fs03vuln.offsec.lan (13), fs01.offsec.lan (11)
informational: MSEDGEWIN10 (18), IEWIN7 (17), fs01.offsec.lan (15), PC01.example.corp (13), IE8Win7 (12)

╭───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╮
│ Top critical alerts:                                              Top high alerts:
                  │
├╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┤
│ Sticky Key Like Backdoor Usage (10)                               Metasploit SMB Authentication (3,562)               │
│ Active Directory Replication from Non Machine Account (6)         Malicious Svc Possibly Installed (271)              │
│ Meterpreter or Cobalt Strike Getsystem Service Installation (6)   Susp Svc Installed (257)
                  │
│ Defender Alert (Severe) (4)                                       PowerShell Scripts Installed as Services (253)      │
│ WannaCry Ransomware (4)                                           Suspicious Service Installation Script (250)        │
├╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┤
│ Top medium alerts:                                                Top low alerts:
                  │
├╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┤
│ Potentially Malicious PwSh (235)                                  Logon Failure (Wrong Password) (3,564)              │
│ Proc Injection (104)                                              Susp CmdLine (Possible LOLBIN) (1,418)              │
│ Reg Key Value Set (Sysmon Alert) (103)                            Non Interactive PowerShell (325)
                  │
│ Suspicious Remote Thread Target (93)                              Rare Service Installations (321)
                  │
│ Wscript Execution from Non C Drive (61)                           Windows Processes Suspicious Parent Directory (282) │
├╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┤
│ Top informational alerts:
                  │
├╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┤
│ Proc Exec (11,173)                                                Explicit Logon (342)
                  │
│ NetShare File Access (2,564)                                      Svc Installed (331)
                  │
│ PwSh Scriptblock (789)                                            New Non-USB PnP Device (268)                        │
│ PwSh Pipeline Exec (680)                                          Logon (Type 3 Network) (228)                        │
│ NetShare Access (433)                                             File Created (210)
                  │
╰─────────────────────────────────────────────────────────────────╌─────────────────────────────────────────────────────╯

Elapsed time: 00:00:09.413
Saved file: 725.csv (16.3 MB)

I would appreciate it if you could review.

YamatoSecurity · 2022-11-19T13:06:24Z

まだカラーに対応していないのは少し残念ですが、デフォルトでカテゴリごとにオプションが出力されて良いですね！
--target-file-ext <EVTX_FILE_EXT> Specify additional target file extensions (ex: evtx_data) (ex: evtx1 evtx2)等の複数可能のオプションはCargo 3までは分かるように<EVTX_FILE_EXT>...だったと思いますが、Clap 4以上では...が無くなる感じですか？
(複数可能かどうか区別できた方が良さそうです）

hitenkoku · 2022-11-20T10:16:00Z

確認してみます。Conflictも発生しているのでそこも直しておきます。

YamatoSecurity · 2022-11-21T10:02:53Z

複数を設定してみたら、エラーが出たので、複数を指定できないっぽいです。

hitenkoku · 2022-11-21T16:06:11Z

確認ありがとうございます。他の指定方法がないか確認してみます

YamatoSecurity · 2022-11-22T12:56:42Z

以前のようにmultiple_values = true等を指定してみたのですが、コンパイルエラーが出ていたので、Clap 4では使えなくなったかな？バグかな？
コンマで区切った方が良さそうなので、use_value_delimiter = true, value_delimiter = ','を追加してみました。
...が表示されないのは残念だけど、ヘルプメニューを見たら分かるので、...が無くても大丈夫だと思います。
Clap 4ではUSAGE等のタイトルがUsageに変わっているので、それも合わせました。
ご確認ください。

hitenkoku · 2022-11-22T15:01:47Z

Vecを指定することで複数の値を取得することは可能のようですので少々お待ちください

hitenkoku · 2022-11-22T15:51:57Z

@YamatoSecurity arg_numというのが新しく作られてそちらにまとめられたようです。
arg_numを指定することで以前のように以下のコマンドで問題なく動作することを確認しました。

どちらを採用するのか決めて頂けますでしょうか。現在はそちらが利用しているカンマ区切りのソースコードとしています

>./725.exe -d ..\hayabusa-sample-evtx\ --target-file-ext evtx2  evtx3

...
Analyzing event files: 582
Total file size: 148.5 MB

Loading detections rules. Please wait.

Excluded rules: 14
Noisy rules: 5 (Disabled)

Experimental rules: 1898 (58.96%)
Stable rules: 213 (6.62%)
Test rules: 1108 (34.42%)

Hayabusa rules: 141
Sigma rules: 3078

…urity/hayabusa into 725-upgrade-to-clap-4

YamatoSecurity

対応ありがとうございます！
一つだけ：
オプション指定なし、-u等々は毎回メモリのデバッグ情報が出力されています。
確認して頂けますか？

hitenkoku · 2022-11-23T03:32:05Z

@YamatoSecurity 対応完了しました。ご確認下さい

YamatoSecurity

LGTM!
ありがとうございました！

hitenkoku added 7 commits November 19, 2022 14:08

updated clap version 4 #725

7f4125f

updated changelog #725

3676fbe

added color feature and replaced clap attribte with arg and command #725

12d0555

reorder Config struct fields

982bb53

formatting help menu

e7306af

updated readme #725

d3642cd

updated rules

401d9ab

hitenkoku added the enhancement New feature or request label Nov 19, 2022

hitenkoku requested a review from YamatoSecurity November 19, 2022 07:11

hitenkoku self-assigned this Nov 19, 2022

hitenkoku linked an issue Nov 19, 2022 that may be closed by this pull request

Upgrade to Clap 4 #725

Closed

fukusuket and others added 2 commits November 22, 2022 20:26

add mimalloc stats print feature

99eb146

update help menu titles and logo spacing

834f556

hitenkoku added 2 commits November 23, 2022 00:14

WIP: ccd855b fixed wrong if condition

2b8e6a9

Merge branch 'main' into 725-upgrade-to-clap-4

5c00a06

hitenkoku and others added 6 commits November 23, 2022 00:56

fixed compile and adjust multi value get

10f360d

Merge branch '725-upgrade-to-clap-4' of https://github.com/Yamato-Sec…

45a870e

…urity/hayabusa into 725-upgrade-to-clap-4

fixed merge error

4154100

change title to Display Settings

7a12e58

update readme

c27245c

updated changelog

17ef9fc

YamatoSecurity requested changes Nov 23, 2022

View reviewed changes

cargo fmt

49dca2b

removed unnecessary memory stat display and updated changelog #725

fbd6d4c

hitenkoku requested a review from YamatoSecurity November 23, 2022 03:32

YamatoSecurity approved these changes Nov 23, 2022

View reviewed changes

YamatoSecurity merged commit 91b0ca5 into main Nov 23, 2022

hitenkoku deleted the 725-upgrade-to-clap-4 branch November 23, 2022 04:05

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Update to clap 4.x #817

Update to clap 4.x #817

hitenkoku commented Nov 19, 2022 •

edited

Loading

YamatoSecurity commented Nov 19, 2022

hitenkoku commented Nov 20, 2022

YamatoSecurity commented Nov 21, 2022

hitenkoku commented Nov 21, 2022

YamatoSecurity commented Nov 22, 2022

hitenkoku commented Nov 22, 2022

hitenkoku commented Nov 22, 2022 •

edited

Loading

YamatoSecurity left a comment

hitenkoku commented Nov 23, 2022

YamatoSecurity left a comment

Update to clap 4.x #817

Update to clap 4.x #817

Conversation

hitenkoku commented Nov 19, 2022 • edited Loading

What Changed

Evidence

YamatoSecurity commented Nov 19, 2022

hitenkoku commented Nov 20, 2022

YamatoSecurity commented Nov 21, 2022

hitenkoku commented Nov 21, 2022

YamatoSecurity commented Nov 22, 2022

hitenkoku commented Nov 22, 2022

hitenkoku commented Nov 22, 2022 • edited Loading

YamatoSecurity left a comment

Choose a reason for hiding this comment

hitenkoku commented Nov 23, 2022

YamatoSecurity left a comment

Choose a reason for hiding this comment

hitenkoku commented Nov 19, 2022 •

edited

Loading

hitenkoku commented Nov 22, 2022 •

edited

Loading