Skip to content

Issues: flashinfer-ai/flashinfer

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

27 Open 75 Closed
27 Open 75 Closed
Author
Filter by author
Loading
Label
Filter by label
Loading
Use alt + click/return to exclude labels
or ⇧ + click/return for logical OR
Projects
Filter by project
Loading
Milestones
Filter by milestone
Loading
Assignee
Filter by who’s assigned
Sort
Sort by
Newest Oldest Most commented Least commented Recently updated Least recently updated Best match
Most reactions
👍 👎 😄 🎉 😕 ❤️ 🚀 👀

Issues list

Does Flashinfer support 8-bit attention calculation?
#502 opened Sep 19, 2024 by Juelianqvq
confused about get_lse usage in single_decode with large kv length
#499 opened Sep 14, 2024 by JamesLim-sy
bug: partial unit tests failed
#479 opened Aug 28, 2024 by zhyncs
@zhyncs
2
failed to dispatch head_dim 96
#455 opened Aug 20, 2024 by ZX-ModelCloud
@yzh119
SingleDecodeWithKVCache meets illegal memory access when setting input tensors to cuda:1 bug Something isn't working
#452 opened Aug 17, 2024 by jason-huang03
7
apply_rope_inplace will cause graphbreak due to mutated inputs
#403 opened Jul 28, 2024 by jianc99
4
[FEAT REQ][CUDA GRAPH] Allow explicit control flag to force enable/disable split KV
#397 opened Jul 26, 2024 by AgrawalAmey
2
pytorch 2.4 support
#395 opened Jul 25, 2024 by yzh119
1 of 2 tasks
Chunked prefill support
#392 opened Jul 24, 2024 by Juelianqvq
5
Feature: Flash Attention 3
#369 opened Jul 12, 2024 by zhyncs
@yzh119
3
CUDA Error: no kernel image is available for execution on the device (209) /tmp/build-via-sdist-nl8se4dx/flashinfer-0.0.4+cu118torch2.2/include/flashinfer/attention/decode.cuh: line 871 at function cudaFuncSetAttribute(kernel, cudaFuncAttributeMaxDynamicSharedMemorySize, smem_size)
#249 opened May 16, 2024 by lucasjinreal
2
Circular import error when importing built-from-source flashinfer
#248 opened May 15, 2024 by vedantroy
1
multiple definition of `cuda::__3::pipeline...
#245 opened May 14, 2024 by jpf888
Support MLA (Multi-Head Latent Attention) in DeepSeek-v2
#237 opened May 7, 2024 by yzh119
4
[LoRA] Roadmap of LoRA operators
#199 opened Apr 8, 2024 by yzh119
3 tasks
1
Shared-prefix rope issue
#194 opened Apr 1, 2024 by lkc1997
1
JIT compilation priority: high
#170 opened Mar 11, 2024 by yzh119
1
stack smashing detected in begin_forward when compiling directly from the repo
#166 opened Mar 8, 2024 by mkrima
1
[Feature request] Interleaved ROPE support
#151 opened Mar 4, 2024 by guocuimi
@yzh119
Still looking forward to an e2e example!
#149 opened Mar 4, 2024 by ZSL98
@yzh119
1
[Feature Request] Versatile head dimension
#142 opened Feb 29, 2024 by yzh119
1
4
Can I only profile dense layer or attention layer in flashinfer rather than the whole kernel?
#139 opened Feb 27, 2024 by yintao-he
@yzh119
How to use low-bit KV Cache in flashinfer? enhancement New feature or request
#125 opened Feb 18, 2024 by zhaoyang-star
7
Basic inference example for LLama/Mistral
#108 opened Feb 5, 2024 by vgoklani
3
[Tracking Issue] Documentation and Examples
#67 opened Jan 10, 2024 by yzh119
3 of 4 tasks
ProTip! Exclude everything labeled bug with -label:bug.