Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[SYCL] fix set main gpu error, support single/mul gpu mode #6073

Merged
merged 1 commit into from Mar 15, 2024
Merged

[SYCL] fix set main gpu error, support single/mul gpu mode #6073

merged 1 commit into from Mar 15, 2024

Conversation

NeoZhangJianyu
Copy link
Collaborator

  1. fix error to set main gpu as non-zero.

  2. add new APIs: ggml_backend_sycl_set_single_device_mode(), ggml_backend_sycl_set_mul_device_mode() to handle single/multiple cards by split-mode.
    In CI, enable ggml_backend_sycl_set_mul_device_mode() as default.
    If split-mode==layer (default), use all GPUs with top max compute unit.
    Else, use the main-gpu set by user as only device. It supports level_zero:gpu, opencl:gpu.

  3. refactor the shown device list.

  • change the order in shown device list.
    sort the devices by type (level_zero, opencl:gpu, opencl:cpu, opencl:acc) and max compute unit.
  • add type item to follow the output style of tool sycl-ls.
found 6 SYCL devices:
|  |                  |                                             |compute   |Max compute|Max work|Max sub|               |
|ID|       Device Type|                                         Name|capability|units      |group   |group  |Global mem size|
|--|------------------|---------------------------------------------|----------|-----------|--------|-------|---------------|
| 0|[level_zero:gpu:0]|               Intel(R) Arc(TM) A770 Graphics|       1.3|        512|    1024|     32|    16225243136|
| 1|[level_zero:gpu:1]|                    Intel(R) UHD Graphics 770|       1.3|         32|     512|     32|    53651849216|
| 2|    [opencl:gpu:0]|               Intel(R) Arc(TM) A770 Graphics|       3.0|        512|    1024|     32|    16225243136|
| 3|    [opencl:gpu:1]|                    Intel(R) UHD Graphics 770|       3.0|         32|     512|     32|    53651849216|
| 4|    [opencl:cpu:0]|         13th Gen Intel(R) Core(TM) i7-13700K|       3.0|         24|    8192|     64|    67064815616|
| 5|    [opencl:acc:0]|               Intel(R) FPGA Emulation Device|       1.2|         24|67108864|     64|    67064815616|
  1. update examples/sycl/run-llama2.sh
    support to switch in single/multiple cards by set device id as parameter.

@NeoZhangJianyu
Copy link
Collaborator Author

@ggerganov @slaren I recreate the PR due to the CI always be fault.
It includes same code change of #6022.

@aahouzi aahouzi mentioned this pull request Mar 15, 2024
Open
ggerganov
ggerganov approved these changes Mar 15, 2024
View reviewed changes
Copy link
Owner

@ggerganov ggerganov left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The CI using sanitizers is failing due to unrelated problems (some linux kernel update), so it is normal. If the SYCL builds are passing, it's ok to merge

@NeoZhangJianyu
Copy link
Collaborator Author

The CI using sanitizers is failing due to unrelated problems (some linux kernel update), so it is normal. If the SYCL builds are passing, it's ok to merge

OK! Thank you!

@NeoZhangJianyu
Copy link
Collaborator Author

@slaren Could you review and approve if it's OK!

@NeoZhangJianyu NeoZhangJianyu merged commit 46acb36 into ggerganov:master Mar 15, 2024
57 of 61 checks passed
hodlen pushed a commit to hodlen/llama.cpp that referenced this pull request Apr 1, 2024
@NeoZhangJianyu @hodlen
fix set main gpu error (ggerganov#6073)
31b7c95
hodlen pushed a commit to hodlen/llama.cpp that referenced this pull request Apr 3, 2024
@NeoZhangJianyu @hodlen
fix set main gpu error (ggerganov#6073)
5cbc43e
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@ggerganov ggerganov ggerganov approved these changes

@slaren slaren slaren approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
3 participants
@NeoZhangJianyu @ggerganov @slaren