Discussions

• 1 You must be logged in to vote

  💡

  Self-frankenmerge support?

  marcingomulkiewicz started Apr 30, 2024 in Ideas

  

  

  0
• 37 You must be logged in to vote

  🙌

  Tutorial: How to convert HuggingFace model to GGUF format

  samos123 started Sep 1, 2023 in Show and tell

  

  

  

  

  67
• 6 You must be logged in to vote

  💡

  4-bit KV Cache

  Dampfinchen started Mar 8, 2024 in Ideas

  

  

  

  

  7
• 1 You must be logged in to vote

  💬

  Could you share any papers/books that provide underlying explanations for this project?

  nicks64 started Apr 24, 2024 in General

  

  

  

  

  8
• 1 You must be logged in to vote

  🙏

  roberta-large support

  alexbenari asked Apr 30, 2024 in Q&A · Unanswered

  

  

  0
• 2 You must be logged in to vote

  🙏

  AMD R7 7840HS Device

  IcedSoyaBeanMilk asked Apr 30, 2024 in Q&A · Unanswered

  

  

  0
• 2 You must be logged in to vote

  🙏

  Clip, how can i get the clip features ?

  therealRogueWarlock asked Apr 30, 2024 in Q&A · Unanswered

  

  

  0
• 1 You must be logged in to vote

  🙏

  Backends Supported with llama.cpp

  lalith1403 asked Apr 30, 2024 in Q&A · Unanswered

  

  

  0
• 1 You must be logged in to vote

  🙏

  Understanding the Quantization Support in llama.cpp

  lalith1403 asked Apr 30, 2024 in Q&A · Unanswered

  

  

  0
• 2 You must be logged in to vote

  🙏

  Am I being limited by single core CPU performance when fully offloaded?

  reversebias asked Feb 29, 2024 in Q&A · Unanswered

  

  

  

  

  6
• 1 You must be logged in to vote

  🙏

  How to use llama.dll

  SevaSk asked Apr 29, 2024 in Q&A · Answered

  

  

  4
• 7 You must be logged in to vote

  🙏

  How to use embedding ?

  cclinus asked Jul 11, 2023 in Q&A · Unanswered

  

  

  

  

  6
• 1 You must be logged in to vote

  💬

  Models for testing inference / sampling

  danemadsen started Apr 29, 2024 in General

  

  

  4
• 1 You must be logged in to vote

  💡

  Any viable Assembly-level optimizations ideas?

  KaelaSavia started Apr 29, 2024 in Ideas

  

  

  0
• 1 You must be logged in to vote

  🙏

  cudaDeviceReset() not working?

  crashr asked Apr 29, 2024 in Q&A · Unanswered

  

  

  0
• 3 You must be logged in to vote

  🙏

  GGUF huge files size / Split possible ?

  split GGUF split model sharding

  phymbert asked Mar 11, 2024 in Q&A · Closed · Answered

  

  

  

  

  37
• 1 You must be logged in to vote

  💬

  perplexity output of LLAMA2 7B quantized by IQ3_S seems to be weired

  penghongbo started Apr 29, 2024 in General

  

  

  0
• 1 You must be logged in to vote

  🙏

  Why do I get prompted when I use convert.py to convert llama13b to GGUF format, give me a bug, that is Why do I get prompted when I use convert.py to convert llama13b to GGUF format, zipfile.BadZipFile: File is not a zip file

  yimisiyang asked Apr 29, 2024 in Q&A · Unanswered

  

  

  0
• 1 You must be logged in to vote

  🙏

  How do you use large context?

  segmond asked Apr 28, 2024 in Q&A · Closed · Answered

  

  

  1
• 1 You must be logged in to vote

  🙏

  Pointers for the minimum needed to embed inference in a c++ app ?

  gitced asked Apr 22, 2024 in Q&A · Answered

  

  

  

  

  5
• 2 You must be logged in to vote

  💬

  Is llama 3 more prone to damage from quantization?

  Dampfinchen started Apr 25, 2024 in General

  

  

  

  

  4
• 8 You must be logged in to vote

  🙏

  Parallelization / Batching Explanation

  lapp0 asked Nov 19, 2023 in Q&A · Unanswered

  

  

  

  

  8
• 5 You must be logged in to vote

  💬

  Can someone explain what are the meanings of these timings

  Tommy787576 started May 4, 2023 in General

  

  

  

  

  7
• 1 You must be logged in to vote

  🙏

  The AI suddenly stops the conversation

  toomy78 asked Mar 12, 2024 in Q&A · Unanswered

  

  

  

  

  6
• 3 You must be logged in to vote

  🙏

  llama3 generation time (GPU)

  gopalgk asked Apr 24, 2024 in Q&A · Answered

  

  

  3