Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Temperature slider not working #6676

Open
GlasslessPizza opened this issue Apr 14, 2024 · 2 comments
Open

Temperature slider not working #6676

GlasslessPizza opened this issue Apr 14, 2024 · 2 comments

Comments

@GlasslessPizza
Copy link

Tried in version b2671. The temperature slider doesn't seem to do anything no matter the model, even when cranked all the way to 2 which should produce gibberish but the model behaves coherently instead. All other frontend params are set to default using the button on the top.

.\bin\server.exe --model .\mistral-7b-instruct-v0.2.Q4_0.gguf --ctx-size 2048 --n-gpu-layers 99 --log-disable --path .\frontend

The frontend folder is also updated to the latest as of today in examples/server/public.

@GlasslessPizza
Copy link
Author

I did a few tests and it seems the slider does do something. If i set it to zero the output is the same every time. If I set to 2 the output is always different. So I think recently the behavior of the temperature slider was toned down. I'll try to tweak the javascript in the frontend files to increase it past 2.

@GlasslessPizza
Copy link
Author

Reopening this issue as even using the API directly, which allows for any arbitrary value, the response from server.exe is still coherent even at temperature=100:

curl --request POST --url http://localhost:8080/completion --header "Content-Type: application/json" --data "{\"prompt\": \"Building a website can be done in 10 simple steps:\",\"n_predict\":128,\"temperature\": 100.0}"

Response:

{"content":" \n\n1. Plan the design: Decide on the website's design. Will it have a classic web look, a modern, sleek look, or something else entirely? Will it include a header and footer? Where will the navigation go, and will the home page, about page, and contact page exist on your site?\n2. Select a CMS (content management system): Will you build your site yourself using HTML, CSS, JavaScript and other coding languages, or do you want a simpler method to do so? In many cases, using a Content Management System (like WordPress or Joomla) is faster,","id_slot":0,"stop":true,"model":".\\codeqwen-1_5-7b-chat-q8_0.gguf","tokens_predicted":128,"tokens_evaluated":12,"generation_settings":{"n_ctx":8192,"n_predict":-1,"model":".\\codeqwen-1_5-7b-chat-q8_0.gguf","seed":4294967295,"temperature":100.0,"dynatemp_range":0.0,"dynatemp_exponent":1.0,"top_k":40,"top_p":0.949999988079071,"min_p":0.05000000074505806,"tfs_z":1.0,"typical_p":1.0,"repeat_last_n":64,"repeat_penalty":1.0,"presence_penalty":0.0,"frequency_penalty":0.0,"penalty_prompt_tokens":[],"use_penalty_prompt_tokens":false,"mirostat":0,"mirostat_tau":5.0,"mirostat_eta":0.10000000149011612,"penalize_nl":false,"stop":[],"n_keep":0,"n_discard":0,"ignore_eos":false,"stream":false,"logit_bias":[],"n_probs":0,"min_keep":0,"grammar":"","samplers":["top_k","tfs_z","typical_p","top_p","min_p","temperature"]},"prompt":"Building a website can be done in 10 simple steps:","truncated":false,"stopped_eos":false,"stopped_word":false,"stopped_limit":true,"stopping_word":"","tokens_cached":139,"timings":{"prompt_n":12,"prompt_ms":300.235,"prompt_per_token_ms":25.019583333333333,"prompt_per_second":39.96869119189968,"predicted_n":128,"predicted_ms":1702.467,"predicted_per_token_ms":13.3005234375,"predicted_per_second":75.1850109282588}}

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants