I was never able to reproduce

'str' object has no attribute 'copy'

Guess the LLM I've tried were not "weak" enough.

litellm allows you to provide a custom model endpoint. If we could write a dummy & dumb custom LLM, we could write better integration tests, and easily reproduce errors like this (I suppose).

I still cannot reproduce it locally. I have no chance to see the true response. Our code didn't log response, so no user can provide it. If some one can provide the true error response message, that will be very helpful.

When initialization the agent, can we add some prompts emphasize the action type?

I found https://www.youtube.com/watch?v=RJ6NN8Y-xok&ab_channel=OfirPress very worth watching. At 18:58, the author mentioned that they had to emphasize the current directory the agent is on, otherwise it would often mess up. This redundant piece of information seemingly helps the LLM strengthen its memory.

That being said, apparently powerful LLMs have no problem of remembering what actions are legal. Maybe you want to tune prompt only for "very weak" LLMs, but we don't have a framework to tune prompt per LLM (group) yet.

When initialization the agent, can we add some prompts emphasize the action type?

I found https://www.youtube.com/watch?v=RJ6NN8Y-xok&ab_channel=OfirPress very worth watching. At 18:58, the author mentioned that they had to emphasize the current directory the agent is on, otherwise it would often mess up. This redundant piece of information seemingly helps the LLM strengthen its memory.

That being said, apparently powerful LLMs have no problem of remembering what actions are legal. Maybe you want to tune prompt only for "very weak" LLMs, but we don't have a framework to tune prompt per LLM (group) yet.

Yeah, I also see this case. This is what trigger me idea. I wonder whether add some redundant info in prompts can be helpful to solve it.

This issue is stale because it has been open for 30 days with no activity. Remove stale label or comment or this will be closed in 7 days.