GGML Shape & Strides question #1111
akapoor3518
started this conversation in
General
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
Hi,
I was looking GGML code it has ne & nb array under tensor structure. ne is used for shape & nb is used for stride. I have following questions:
1 For Llama.cpp application how we are tuning shape & stride for tensor for particular GGML operation. The reason i asked Base on this information we will general Kernel code for our custom hardware which will depend on ne & nb value.
2 Do we have any recommendation for shape & stride for Matrix Multiplication , does it defer for every model?
3 Can backend can control ne & nb value for tensor or it can be only controlled o trough llama.cpp application?
4 Our Customer hardware kernel are fixed size of ne & nb, once we generate the custom kernel, ne & nb cant change hence we would like to know what is recommendation for ne & nb to model run using llama.cpp or ollama application
Thanks,
Anoop
Beta Was this translation helpful? Give feedback.
All reactions