Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

6MB AWS Lambda Hard Limit Strikes Again #50

Open
jonsharratt opened this issue Mar 3, 2017 · 7 comments
Open

6MB AWS Lambda Hard Limit Strikes Again #50

jonsharratt opened this issue Mar 3, 2017 · 7 comments
Labels

Comments

@jonsharratt
Copy link
Member

jonsharratt commented Mar 3, 2017

This is a Bug Report

Description

For bug reports:

  • What went wrong?
    When attempting to use npm i npm --save failed.

It actually came about attempting to install cordova as npm is a dependency of it.

  • What did you expect should have happened?
    npm module should have installed

  • What stacktrace or error message did you
    experience?

image

Similar or dependent issues:

Additional Data

  • NPM CLI version you are using:
    3.10.16
  • Serverless version you're using:
    1.6.1
  • Node version you're using:
    v6.2.0
@jonsharratt jonsharratt added the bug label Mar 3, 2017
@jonsharratt
Copy link
Member Author

CloudWatch Logs:

Lambda:
image

API Gateway:
image

@brimworks
Copy link

BTW, I also ran into this issue... wasted a few hours trying to figure out what went wrong. To bad we can't send an S3 signed PUT URL down to the client. Perhaps if we started a conversation with the npm developers we could change the client/server interaction during publish such that an S3 signed URL could be used?

@jonsharratt
Copy link
Member Author

Yer @brimworks we really are shackled by the way npm and Lambda works just now. I spoke to Amazon and they said, they can not do anything for me in regards to these hard limits.

I have some things in the pipeline just now that will probably make the whole js community go crazy :trollface: with a codebox package manager that uses streams against a codebox private registry.

One thing I am aiming to achieve also is that each module is it's own ".json" file so that when we work with Docker it can cache every layer and only have to invalidate those dependencies that have a version change.

Watch this space 👍

@joebowbeer
Copy link
Contributor

I think it would be worth mentioning this issue in the README.

It may also be worth mentioning that one strategy for escaping the 6MB limit is to configure codebox-npm to manage a single (private) scope, in which case the default registry is used for everything else:

npm config set @myscope:registry https://xyz.amazonaws.com/stage/registry/
"publishConfig": {
  "@myscope:registry": "https://xyz.amazonaws.com/stage/registry/"
},

@micksabox
Copy link

I’m having trouble parsing this issue and what it means for my decision to deploy Codebox-npm for my organization.

Does the 6mb limit mean that packages published or used via npm install cannot be over 6mb? I think this is why you described an error when attempting to install npm. I think this is also why @joebowbeer solution works for publishing/using npm modules from the public registry. But it still would not work if the scoped private modules go over 6mb in size, correct?

Thanks for the clarification and great work

@joebowbeer
Copy link
Contributor

@micksabox AWS Lambda limits response body payload (e.g., your npm package) to 6MB

https://docs.aws.amazon.com/lambda/latest/dg/limits.html

@JordanSinko
Copy link

I was able to bypass this, specifically for the package 'npm', by forking the code and changing the response of the package-get be a redirect, instead of directly fetching its meta, when fetching NPM packages.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

5 participants