-
Notifications
You must be signed in to change notification settings - Fork 10
Issues: InftyAI/llmaz
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Author
Label
Projects
Milestones
Assignee
Sort
Issues list
Is there any early proposal or document about integrating with Gateway API ?
feature
Categorizes issue or PR as related to a new feature.
needs-priority
Indicates a PR lacks a label and requires one.
needs-triage
Indicates an issue or PR lacks a label and requires one.
#165
opened Sep 15, 2024 by
caozhuozi
[ModelLoader] Some huggingface models may contain duplicated weights
feature
Categorizes issue or PR as related to a new feature.
needs-priority
Indicates a PR lacks a label and requires one.
needs-triage
Indicates an issue or PR lacks a label and requires one.
#163
opened Sep 14, 2024 by
kerthcet
3 tasks
[WebUI] Add support for webui
feature
Categorizes issue or PR as related to a new feature.
help wanted
Extra attention is needed
needs-kind
Indicates a PR lacks a label and requires one.
needs-priority
Indicates a PR lacks a label and requires one.
needs-triage
Indicates an issue or PR lacks a label and requires one.
#160
opened Sep 13, 2024 by
kerthcet
3 tasks
[Umbrella] Improve test coverages
cleanup
Categorizes issue or PR as related to cleaning up code, process, or technical debt.
good first issue
Good for newcomers
needs-priority
Indicates a PR lacks a label and requires one.
needs-triage
Indicates an issue or PR lacks a label and requires one.
#156
opened Sep 12, 2024 by
kerthcet
Customized flags for backendRuntimes
feature
Categorizes issue or PR as related to a new feature.
needs-priority
Indicates a PR lacks a label and requires one.
needs-triage
Indicates an issue or PR lacks a label and requires one.
#140
opened Sep 11, 2024 by
kerthcet
3 tasks done
Support traditional models
feature
Categorizes issue or PR as related to a new feature.
needs-priority
Indicates a PR lacks a label and requires one.
needs-triage
Indicates an issue or PR lacks a label and requires one.
#133
opened Sep 9, 2024 by
kerthcet
3 tasks done
Loading model weights more efficiently
feature
Categorizes issue or PR as related to a new feature.
needs-priority
Indicates a PR lacks a label and requires one.
needs-triage
Indicates an issue or PR lacks a label and requires one.
Support scaling with Spot instances for cost saving
needs-priority
Indicates a PR lacks a label and requires one.
needs-triage
Indicates an issue or PR lacks a label and requires one.
question
Further information is requested
#106
opened Aug 27, 2024 by
kerthcet
3 tasks
Accelerate model loading
feature
Categorizes issue or PR as related to a new feature.
important-soon
Must be staffed and worked on either currently, or very soon, ideally in time for the next release.
needs-triage
Indicates an issue or PR lacks a label and requires one.
#103
opened Aug 21, 2024 by
kerthcet
3 tasks
Support filesystems
feature
Categorizes issue or PR as related to a new feature.
needs-priority
Indicates a PR lacks a label and requires one.
needs-triage
Indicates an issue or PR lacks a label and requires one.
#100
opened Aug 19, 2024 by
kerthcet
3 tasks done
Model aware scheduling
feature
Categorizes issue or PR as related to a new feature.
needs-priority
Indicates a PR lacks a label and requires one.
needs-triage
Indicates an issue or PR lacks a label and requires one.
#96
opened Aug 19, 2024 by
kerthcet
2 of 3 tasks
ollama support
feature
Categorizes issue or PR as related to a new feature.
needs-priority
Indicates a PR lacks a label and requires one.
needs-triage
Indicates an issue or PR lacks a label and requires one.
#91
opened Aug 17, 2024 by
kerthcet
1 of 3 tasks
Prompts managements
feature
Categorizes issue or PR as related to a new feature.
needs-priority
Indicates a PR lacks a label and requires one.
needs-triage
Indicates an issue or PR lacks a label and requires one.
#90
opened Aug 17, 2024 by
kerthcet
3 tasks done
Failover policy for various backends
important-longterm
Important over the long term, but may not be staffed and/or may need multiple releases to complete.
needs-kind
Indicates a PR lacks a label and requires one.
needs-triage
Indicates an issue or PR lacks a label and requires one.
#86
opened Aug 13, 2024 by
kerthcet
2 of 3 tasks
Parallel model serving
important-longterm
Important over the long term, but may not be staffed and/or may need multiple releases to complete.
needs-kind
Indicates a PR lacks a label and requires one.
needs-triage
Indicates an issue or PR lacks a label and requires one.
#85
opened Aug 13, 2024 by
kerthcet
2 of 3 tasks
Lack the flexibility to express deploy primitives
needs-priority
Indicates a PR lacks a label and requires one.
needs-triage
Indicates an issue or PR lacks a label and requires one.
question
Further information is requested
#81
opened Aug 12, 2024 by
kerthcet
Integrate with Kueue for fungibility capacity
feature
Categorizes issue or PR as related to a new feature.
needs-priority
Indicates a PR lacks a label and requires one.
needs-triage
Indicates an issue or PR lacks a label and requires one.
#74
opened Aug 8, 2024 by
kerthcet
2 of 3 tasks
Mount /dev/shm for shared memory files
feature
Categorizes issue or PR as related to a new feature.
needs-priority
Indicates a PR lacks a label and requires one.
needs-triage
Indicates an issue or PR lacks a label and requires one.
#73
opened Aug 8, 2024 by
kerthcet
3 tasks
Support TGI as another alternative backend
feature
Categorizes issue or PR as related to a new feature.
help wanted
Extra attention is needed
needs-priority
Indicates a PR lacks a label and requires one.
needs-triage
Indicates an issue or PR lacks a label and requires one.
Benchmark toolkit support
feature
Categorizes issue or PR as related to a new feature.
needs-priority
Indicates a PR lacks a label and requires one.
needs-triage
Indicates an issue or PR lacks a label and requires one.
#66
opened Aug 6, 2024 by
kerthcet
1 of 3 tasks
Milestone v0.1.0
needs-kind
Indicates a PR lacks a label and requires one.
needs-priority
Indicates a PR lacks a label and requires one.
needs-triage
Indicates an issue or PR lacks a label and requires one.
#63
opened Aug 5, 2024 by
kerthcet
Support different GPU accelerators for fungibility
feature
Categorizes issue or PR as related to a new feature.
needs-priority
Indicates a PR lacks a label and requires one.
needs-triage
Indicates an issue or PR lacks a label and requires one.
An an example for multi-host inference with Service
cleanup
Categorizes issue or PR as related to cleaning up code, process, or technical debt.
needs-priority
Indicates a PR lacks a label and requires one.
needs-triage
Indicates an issue or PR lacks a label and requires one.
Model version management
feature
Categorizes issue or PR as related to a new feature.
important-longterm
Important over the long term, but may not be staffed and/or may need multiple releases to complete.
needs-triage
Indicates an issue or PR lacks a label and requires one.
#58
opened Aug 4, 2024 by
kerthcet
3 tasks done
Concurrently download the main container image when downloading weights
feature
Categorizes issue or PR as related to a new feature.
important-longterm
Important over the long term, but may not be staffed and/or may need multiple releases to complete.
needs-triage
Indicates an issue or PR lacks a label and requires one.
#50
opened Aug 2, 2024 by
kerthcet
Previous Next
ProTip!
Adding no:label will show everything without a label.