Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature: named structures #169

Merged
merged 17 commits into from
Jul 14, 2017
Merged

Feature: named structures #169

merged 17 commits into from
Jul 14, 2017

Conversation

cgranade
Copy link
Contributor

In developing tests for #167, I finally got tired of packing and unpacking opaque struct.Struct format strings and wrote a helper class to fix that. This helper class allows for referring to named fields of C-style structures, instead of having to remember the order of fields in a Struct format string. Though this new class doesn't yet support arrays other than char[], it should already be useful for applications such as the HW info response from APT devices.

@coveralls
Copy link

coveralls commented Apr 20, 2017

Coverage Status

Coverage decreased (-0.01%) to 84.632% when pulling fc177be on feature-named-struct into bc4fdbb on master.

@cgranade
Copy link
Contributor Author

One question, pylint seems to be unable to ensure that Field.__len__ returns a non-negative integer, even though it always does so or raises an exception. Is there a best practice you'd suggest for addressing this? Thanks!

@scasagrande scasagrande changed the base branch from master to develop April 20, 2017 14:16

# pylint: disable=no-member,protected-access,blacklisted-name,missing-docstring

def test_named_struct_roundtrip():
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These might be a good candidate to use Hypothesis on, where we can try a wide variety of input values. I'll give it a try and see what we get.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh, that's a great idea, I had forgotten about Hypothesis. Agreed, though, seems like a really good candidate for finding test cases.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As a sneak peek:

from hypothesis import given
import hypothesis.strategies as st

@given(st.integers(min_value=0, max_value=0x7FFF*2+1), st.integers(min_value=0, max_value=0xFF))
def test_named_struct_roundtrip(var1, var2):
    class Foo(NamedStruct):
        a = Field('H')
        padding = Padding(12)
        b = Field('B')

    foo = Foo(a=var1, b=var2)
    assert Foo.unpack(foo.pack()) == foo

Seems to work pretty well!

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's awesome, love that it works out for this case. That may help us write other unit tests in the future, too. For this PR, maybe it makes sense to use a string strategy for StringField instances?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I believe so; I'm going to give it a try and see how it works.

@coveralls
Copy link

Coverage Status

Coverage decreased (-0.02%) to 84.907% when pulling 6dd744e on feature-named-struct into 065d3d2 on develop.

2 similar comments
@coveralls
Copy link

Coverage Status

Coverage decreased (-0.02%) to 84.907% when pulling 6dd744e on feature-named-struct into 065d3d2 on develop.

@coveralls
Copy link

coveralls commented Apr 22, 2017

Coverage Status

Coverage decreased (-0.02%) to 84.907% when pulling 6dd744e on feature-named-struct into 065d3d2 on develop.

@scasagrande
Copy link
Contributor

I wonder if we should pull this out into its own project. I can handle that if we want. Thoughts?

@crazy4pi314
Copy link

I think that could be a good idea, there are quite a few instruments that i have now that require their own parser/unpack-er for the results. Could be nice to collect these all in one place...

@scasagrande
Copy link
Contributor

Yeah, the code in this PR is pretty generic, and I imagine its useful for all the different instrument's that you're working on. Heck, maybe even useful to other people 😲

@cgranade
Copy link
Contributor Author

In particular, the named struct pattern has come up before in parsing data returned by instruments. Apparently, using HDF5 is just too straightforward and useful...

@scasagrande
Copy link
Contributor

That would just make too much sense! And after dealing with things like tcubes...

@cgranade
Copy link
Contributor Author

Revisiting this, would it make sense to include the feature in IK directly for now, possibly splitting it out into its own project later? The Python packaging infrastructure isn't especially great for micropackages (this might be a good thing, given left-pad), such that unless we're splitting out more than just named structures, the cost of a new package doesn't seem justified yet. Anyway, that's just my 2¢...

@coveralls
Copy link

Coverage Status

Coverage decreased (-0.06%) to 84.969% when pulling 9f72365 on feature-named-struct into 215a6d0 on develop.

@cgranade
Copy link
Contributor Author

Seems as though the build is still getting stuck on #174. I'll lock requirements.txt as @scasagrande suggested in closing the issue, see if that helps resolve the CI builds.

@coveralls
Copy link

coveralls commented Jul 12, 2017

Coverage Status

Coverage decreased (-0.06%) to 84.969% when pulling a897b8c on feature-named-struct into 215a6d0 on develop.

@cgranade
Copy link
Contributor Author

I think the coverage decrease is erroneous, since it's concentrated on files outside the scope of this PR, but I'll investigate further.

@coveralls
Copy link

coveralls commented Jul 12, 2017

Coverage Status

Coverage decreased (-0.03%) to 84.993% when pulling 50733ef on feature-named-struct into 215a6d0 on develop.

@coveralls
Copy link

coveralls commented Jul 12, 2017

Coverage Status

Coverage increased (+0.05%) to 85.072% when pulling 165a7dd on feature-named-struct into 215a6d0 on develop.

@cgranade
Copy link
Contributor Author

I'm pretty happy with this PR now. Is there anything you'd like me to add, @scasagrande? Thanks!

@cgranade cgranade mentioned this pull request Jul 12, 2017
@scasagrande
Copy link
Contributor

Locking numpy to ==1.12.1 doesn't exactly make me happy. I suppose for now we can specify <1.13.0. Some people that run projects dependent on pyquantities have been working at updating it to work with numpy 1.13, but it seems that the main dev doesn't have time for the project or something.

I still feel that this should at some point probably be its own repo, but I'll worry about that on my own time in the future.

Otherwise I'm good with moving forward with this.

@scasagrande
Copy link
Contributor

Also, I need to fix my github notifications...

@cgranade
Copy link
Contributor Author

That's a good point, I'll change the version to <1.13.0 as you suggest. Thanks!

@coveralls
Copy link

coveralls commented Jul 14, 2017

Coverage Status

Coverage increased (+0.05%) to 85.072% when pulling 8c066fb on feature-named-struct into 215a6d0 on develop.

@scasagrande scasagrande merged commit 12e9a15 into develop Jul 14, 2017
@scasagrande scasagrande deleted the feature-named-struct branch July 14, 2017 22:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants