Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add a Float generation method #154

Closed
rlouf opened this issue Jun 21, 2023 · 2 comments · Fixed by #175
Closed

Add a Float generation method #154

rlouf opened this issue Jun 21, 2023 · 2 comments · Fixed by #175
Labels
enhancement text Linked to text generation

Comments

@rlouf
Copy link
Member

rlouf commented Jun 21, 2023

We can steer the generation process so we only output floats. I propose to implement a Float subclass of Sequence that uses masking to restrict the generated tokens to floats. The mask is a function of the tokens that have already been generated: if we sequence generated so far contains a period we can only generate integers. We will need to add a create_proposal method to Sequence that applies the mask to the logits generated by the model.

We can also add constraints on the generated floats:

  • Enforce a given precision
  • Add an upper or lower bound on the value of the float

We will probably need SMC sampling to implement the constraints.

@rlouf rlouf added text Linked to text generation enhancement transformers Linked to the `transformers` integration and removed transformers Linked to the `transformers` integration labels Jun 21, 2023
@brandonwillard
Copy link
Member

If we're talking about regex-driven float parsing, we can also use the vocabulary pre-processing approach described in #131 and this Gist. That could possibly turn the process of determining valid next tokens (and/or the respective indices to be masked) into something closer to a simple dict look-up.

@brandonwillard
Copy link
Member

Looks like the current float masking occasionally produces strings like ".801.4" in test_hf_transformers.test_type_float: see the CI failure in a run of #172 here.

The FSM-based pre-processing approach mentioned above and utilized in #166 should fix that.

@rlouf rlouf linked a pull request Jul 11, 2023 that will close this issue
4 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement text Linked to text generation
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants