Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BUG] Error: Error: 400 Bad Request: Payload error: JSON payload (37327342 bytes) is larger than allowed (limit: 33554432 bytes). #1824

Closed
vipervs opened this issue Feb 27, 2024 · 6 comments
Labels
question Further information is requested

Comments

@vipervs
Copy link

vipervs commented Feb 27, 2024

Im trying to embed a vector db (quadrant) with langchain docs with sitemap.xml link extraction.

everything works fine and the embedding starts, but after a while I get this error:

Error: Error: 400 Bad Request: Payload error: JSON payload (37327342 bytes) is larger than allowed (limit: 33554432 bytes).

is it possible to increase the JSON payload so it can manage all links?

image
@HenryHengZJ
Copy link
Contributor

Is the error coming from Qdrant not able to handle too large of a payload? Or its from Flowise side that is not able to pass the payload over to Qdrant?

@HenryHengZJ HenryHengZJ added the question Further information is requested label Feb 29, 2024
@mrabbah
Copy link
Contributor

mrabbah commented Apr 15, 2024

I got the same error, I think the qdrant client must send the data in batch mode, and I think we must be able to customize the batch_size in the qdrant node

@prithvi151080
Copy link

@vipervs: solution found for the problem. plz check #2191

@xmaiconx
Copy link

Increase QDrant limit size is not a solution. I am trying to upload 211kb csv file and I am getting the same error.
Very strange behaviour.

@prithvi151080
Copy link

@xmaiconx: increase the max file upload size for Qdrant in the custom config.yaml for Qdrant as per ur use case...this is not a limitation from flowise but Qdrant...i faced problem with a txt file of less than 1 mb. use the config.yaml for configuring Qdrant to accept big files. size of the source file is not the issue but the no of vectors that are being generated from the content of the file that is being pushed in Qdrant in a single go is the bottleneck...Qdrant by default accepts max 32 MB (33554432 bytes) in a single go...increase that size as per ur requirement and the problem goes away.

@HenryHengZJ
Copy link
Contributor

should be fixed via PR, closing for now

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
question Further information is requested
Projects
None yet
Development

No branches or pull requests

5 participants