Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix Aplib decompression #64

Merged
merged 12 commits into from
Aug 10, 2021
Merged

Fix Aplib decompression #64

merged 12 commits into from
Aug 10, 2021

Conversation

sisoma2
Copy link
Contributor

@sisoma2 sisoma2 commented Aug 9, 2021

Using the function malduck.aplib() returned a None object.

We can see that running the tests:
image

After the fixes in this PR the tests on compression are passed:
image

@nazywam nazywam self-requested a review August 10, 2021 07:51
Copy link
Member

@nazywam nazywam left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks! Left a few comments

malduck/compression/components/aplib.py Outdated Show resolved Hide resolved
malduck/compression/components/aplib.py Outdated Show resolved Hide resolved
malduck/compression/components/aplib.py Outdated Show resolved Hide resolved
malduck/compression/components/aplib.py Outdated Show resolved Hide resolved
malduck/compression/components/aplib.py Outdated Show resolved Hide resolved
sisoma2 and others added 5 commits August 10, 2021 11:03
Co-authored-by: Michał Praszmo <nazywam@gmail.com>
Co-authored-by: Michał Praszmo <nazywam@gmail.com>
Co-authored-by: Michał Praszmo <nazywam@gmail.com>
Co-authored-by: Michał Praszmo <nazywam@gmail.com>
@sisoma2
Copy link
Contributor Author

sisoma2 commented Aug 10, 2021

Thanks for the review! All the suggested changes are fixed and committed 👍

@nazywam
Copy link
Member

nazywam commented Aug 10, 2021

This diff fixes the pipeline for me:

diff --git a/malduck/compression/aplib.py b/malduck/compression/aplib.py
index 8e75303..39efe4b 100644
--- a/malduck/compression/aplib.py
+++ b/malduck/compression/aplib.py
@@ -1,4 +1,5 @@
 from typing import Optional
+from binascii import crc32
 
 from .components.aplib import APLib
 
@@ -37,24 +38,30 @@ class aPLib:
         orig_crc = None
         strict = not headerless
 
-        if buf.startswith(b'AP32') and len(buf) >= 24:
+        if buf.startswith(b"AP32") and len(buf) >= 24:
             # buf has an aPLib header
-            header_size, packed_size, packed_crc, orig_size, orig_crc = struct.unpack_from('=IIIII', buf, 4)
+            (
+                header_size,
+                packed_size,
+                packed_crc,
+                orig_size,
+                orig_crc,
+            ) = struct.unpack_from("=IIIII", buf, 4)
             buf = buf[header_size : header_size + packed_size]
 
         if strict:
             if packed_size is not None and packed_size != len(buf):
-                raise RuntimeError('Packed buf size is incorrect')
+                raise RuntimeError("Packed buf size is incorrect")
             if packed_crc is not None and packed_crc != crc32(buf):
-                raise RuntimeError('Packed buf checksum is incorrect')
+                raise RuntimeError("Packed buf checksum is incorrect")
 
         result = APLib(buf, strict=strict).depack()
 
         if strict:
             if orig_size is not None and orig_size != len(result):
-                raise RuntimeError('Unpacked buf size is incorrect')
+                raise RuntimeError("Unpacked buf size is incorrect")
             if orig_crc is not None and orig_crc != crc32(result):
-                raise RuntimeError('Unpacked buf checksum is incorrect')
+                raise RuntimeError("Unpacked buf checksum is incorrect")
 
         return result
 
diff --git a/malduck/compression/components/aplib.py b/malduck/compression/components/aplib.py
index 267bbb0..9c010e6 100644
--- a/malduck/compression/components/aplib.py
+++ b/malduck/compression/components/aplib.py
@@ -5,18 +5,16 @@ Adapted from the original C source code from http://ibsensoftware.com/files/aPLi
 Approximately 20 times faster than other Python implementations.
 Compatible with both Python 2 and 3.
 """
-import struct
-from binascii import crc32
 from io import BytesIO
 
-__all__ = ['APLib', 'decompress']
-__version__ = '0.6'
-__author__ = 'Sandor Nemes'
+__all__ = ["APLib"]
+__version__ = "0.6"
+__author__ = "Sandor Nemes"
 
 
 class APLib(object):
 
-    __slots__ = 'source', 'destination', 'tag', 'bitcount', 'strict'
+    __slots__ = "source", "destination", "tag", "bitcount", "strict"
 
     def __init__(self, source: bytes, strict: bool = True) -> None:
         self.source = BytesIO(source)
@@ -126,18 +124,9 @@ class APLib(object):
 
         except (TypeError, IndexError):
             if self.strict:
-                raise RuntimeError('aPLib decompression error')
+                raise RuntimeError("aPLib decompression error")
 
         return bytes(self.destination)
 
     def pack(self):
         raise NotImplementedError
-
-def main():
-    # self-test
-    data = b'T\x00he quick\xecb\x0erown\xcef\xaex\x80jumps\xed\xe4veur`t?lazy\xead\xfeg\xc0\x00'
-    assert decompress(data) == b'The quick brown fox jumps over the lazy dog'
-
-
-if __name__ == '__main__':
-    main()
diff --git a/tests/test_compression.py b/tests/test_compression.py
index f256264..573203f 100644
--- a/tests/test_compression.py
+++ b/tests/test_compression.py
@@ -22,8 +22,6 @@ QacB19//yAF9ff/8hwHX3//IAX19//yHAdff/8gBfX3//IcB19//yAF9ff/
 8hwHX3//IAX19//yH\nAdff/8gBXXf/2QqAAA==
 """)) == b"A"*1024*1024 + b"\n"
 
-    assert aplib(b"helloworld") is None
-    assert aplib(b"") is None
     assert (
         aplib(b'T\x00he quick\xecb\x0erown\xcef\xaex\x80jumps\xed\xe4veur`t?lazy\xead\xfeg\xc0\x00') ==
         b'The quick brown fox jumps over the lazy dog')

Copy link
Member

@nazywam nazywam left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good, thanks! Sorry about the hassle with CI

@nazywam nazywam merged commit 0f3c774 into CERT-Polska:master Aug 10, 2021
@sisoma2
Copy link
Contributor Author

sisoma2 commented Aug 10, 2021

No problem Haha Thanks to you for the quick review and merge. Do you plan to publish a new Pypi release for the other projects that use the package?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants