Add PCAP recognition

[rakovskij-stanislav]

Match first four bytes to detect pcap files

[rakovskij-stanislav]

Also probably you should allow karton-yaramatcher to proceed unrecognized files with further changing status as "recognized" if there is any rule that has meta field "extension" and match this file. It will help to recognize some filetypes without changing code of karton-classifier. Or just add yara to karton-classifier and allow user to write rules that will be used as last resolve to detect file type :)

[nazywam]

Hey, thanks for the PR!
What do you think about recognizing the PCAP files by looking at the output produced by file? We primarily use this method for all other file formats so it should fit in nicely.
Something like

if magic.startswith("pcap capture file"):
    sample_class.update(
        {
            "kind": "pcap",
        }
    )
    return sample_class

should work just fine

[chivay]

Maybe we should also add support for handling pcapng files?

[rakovskij-stanislav]

Hi!
@nazywam, it looks fine

@chivay,
file command on pcapng prints: pcap-ng capture file - version 1.0

So yes, I'll renew this merge request right now, stay tuned)

[chivay]

It seems that you're using an old version of libmagic. In my case the output for pcapng has the following output: pcapng capture file - is seems that the change was made around 2019.
Please us this value in your patch. After this change LGTM.

We know that it's suboptimal to rely on values that aren't stable between environments. To alleviate this problem we'd like to stop depending on system-installed version of libmagic database and ship another one with the classifier. You can track this issue here: #26.

[rakovskij-stanislav]

We know that it's suboptimal to rely on values that aren't stable between environments. To alleviate this problem we'd like to stop depending on system-installed version of libmagic database and ship another one with the classifier. You can track this issue here: #26.

Fmm, for current case with pcapng there is a solution, i'll make a new commit...

[rakovskij-stanislav]

Also probably you should allow karton-yaramatcher to proceed unrecognized files with further changing status as "recognized" if there is any rule that has meta field "extension" and match this file. It will help to recognize some filetypes without changing code of karton-classifier. Or just add yara to karton-classifier and allow user to write rules that will be used as last resolve to detect file type :)

I would like to see this change, it's very useful thing BUT it changes the file processing pipeline, this way it need to be discussed with karton developers.

[rakovskij-stanislav]

Thank you. Please look at other issues in karton-core.

[rakovskij-stanislav]

Unmerged commits...

[rakovskij-stanislav]

Only those with write access to this repository can merge pull requests.

Understood :)

[chivay]

Thanks!