Discussions

• 2 You must be logged in to vote

  🙏

  Inquiry Regarding DeepSpeed Trademark Use for Hardware Vendor Fork

  question Further information is requested legal Questions and discussions related to legal matters.

  kevint324 asked Apr 11, 2024 in Q&A · Closed · Answered

  

  

  2
• 1 You must be logged in to vote

  🙏

  Poor GPU utilization when using multi-node ZeRO-3 on small model

  BramVanroy asked Apr 30, 2024 in Q&A · Unanswered

  

  

  5
• 2 You must be logged in to vote

  💬

  Invalidate trace cache @ step 13: expected module xx, but got module xxx

  mrluin started Aug 3, 2023 in General

  

  

  

  

  12
• 1 You must be logged in to vote

  🙏

  Why ZeRO-offload parameter update after model backward? can do them pipeline?

  Zijie-Tian asked Apr 29, 2024 in Q&A · Unanswered

  

  

  1
• 1 You must be logged in to vote

  🙏

  How to use TiedLayerSpec

  josephwong14wkh asked Apr 29, 2024 in Q&A · Unanswered

  

  

  0
• 2 You must be logged in to vote

  🙏

  Pipeline parallelism for models in transformers

  KaiLv69 asked Apr 23, 2024 in Q&A · Unanswered

  

  

  0
• 1 You must be logged in to vote

  💬

  AutoTP for training?

  KaiLv69 started Jan 11, 2024 in General

  

  

  

  

  4
• 1 You must be logged in to vote

  🙏

  How to set different learning rate for different type of weights?

  tusiqi1 asked Apr 22, 2024 in Q&A · Unanswered

  

  

  0
• 1 You must be logged in to vote

  🙏

  Why activations memory is computed through an experiment rather formulating it for DeepSpeed autotuner?

  kmehant asked Apr 19, 2024 in Q&A · Unanswered

  

  

  0
• 1 You must be logged in to vote

  🙏

  Custom tensor parallelism

  sarathkondeti asked Apr 19, 2024 in Q&A · Unanswered

  

  

  0
• 1 You must be logged in to vote

  🙏

  LMOD: Error while loading shared libraries: libpython3.10.so.1.0: cannot open shared object file: No such file or directory

  BramVanroy asked Jan 20, 2023 in Q&A · Answered

  

  

  

  

  6
• 1 You must be logged in to vote

  🙏

  Zero inference is too slow

  garyyang85 asked Apr 16, 2024 in Q&A · Unanswered

  

  

  5
• 1 You must be logged in to vote

  🙏

  build_win.bat 报错

  songyu1980 asked Apr 15, 2023 in Q&A · Answered

  

  

  

  

  2
• 1 You must be logged in to vote

  🙏

  DeepSpeed with StableDiffusion v2.1

  saqibameen asked Feb 20, 2024 in Q&A · Unanswered

  

  

  1
• 7 You must be logged in to vote

  🙏

  autotuning throws [Errno 36] File name too long

  eavidan asked Feb 5, 2023 in Q&A · Unanswered

  

  

  

  

  3
• 1 You must be logged in to vote

  🙏

  deepspeed zero stage 0 vs pytorch ddp

  asdfry asked Mar 26, 2024 in Q&A · Unanswered

  

  

  0
• 6 You must be logged in to vote

  🙏

  DeepSpeed Sequence Parallel

  vijetadeshpande asked Mar 13, 2024 in Q&A · Unanswered

  

  

  

  

  2
• 2 You must be logged in to vote

  🙏

  Is ZeRO-3 compatible to tensor parallel?

  skyshine102 asked Dec 27, 2023 in Q&A · Answered

  

  

  2
• 1 You must be logged in to vote

  💬

  DeepSpeed MPI error

  Sabiha1225 started Mar 16, 2024 in General

  

  

  0
• 1 You must be logged in to vote

  🙏

  Do we have any other scheduler in ds_config?

  128Ghe980 asked Feb 28, 2024 in Q&A · Unanswered

  

  

  3
• 1 You must be logged in to vote

  💬

  deepspeed for t5-11b on single gpu a100 failed

  sycao5 started Feb 12, 2024 in General

  

  

  0
• 1 You must be logged in to vote

  🙏

  OOM is encountered during training, then skip and continue training

  980202006 asked Feb 7, 2024 in Q&A · Unanswered

  

  

  0
• 1 You must be logged in to vote

  🙏

  Sharing model parameters across multiple GPUs

  Basso42 asked Jan 24, 2024 in Q&A · Unanswered

  

  

  1
• 1 You must be logged in to vote

  🙏

  Problem when training with FULL FP32 model

  bugm asked Feb 2, 2024 in Q&A · Unanswered

  

  

  0
• 1 You must be logged in to vote

  🙏

  Is the hierarchical all-to-all communication primitive released in the latest version?

  roychen9462 asked Jan 27, 2024 in Q&A · Unanswered

  

  

  0