RequestEntityTooLargeException: Request must be smaller than 69905067 bytes for the CreateFunction operation #236

jschimmoeller · 2019-11-12T15:42:42Z

Hey daniel,

I finally had some time to check the latest code against my large next.js project and I am get this error;

error:
RequestEntityTooLargeException: Request must be smaller than 69905067 bytes for the CreateFunction operation
at Object.extractError (C:\Users\james.serverless\components\registry\npm\[email protected]\node_modules\serverless-next.js\node_modules\aws-sdk\lib\protocol\json.js:51:27)
at Request.extractError (C:\Users\james.serverless\components\registry\npm\[email protected]\node_modules\serverless-next.js\node_modules\aws-sdk\lib\protocol\rest_json.js:55:8)
at Request.callListeners (C:\Users\james.serverless\components\registry\npm\[email protected]\node_modules\serverless-next.js\node_modules\aws-sdk\lib\sequential_executor.js:106:20)
at Request.emit (C:\Users\james.serverless\components\registry\npm\[email protected]\node_modules\serverless-next.js\node_modules\aws-sdk\lib\sequential_executor.js:78:10)
at Request.emit (C:\Users\james.serverless\components\registry\npm\[email protected]\node_modules\serverless-next.js\node_modules\aws-sdk\lib\request.js:683:14)
at Request.transition (C:\Users\james.serverless\components\registry\npm\[email protected]\node_modules\serverless-next.js\node_modules\aws-sdk\lib\request.js:22:10)
at AcceptorStateMachine.runTo (C:\Users\james.serverless\components\registry\npm\[email protected]\node_modules\serverless-next.js\node_modules\aws-sdk\lib\state_machine.js:14:12)
at C:\Users\james.serverless\components\registry\npm\[email protected]\node_modules\serverless-next.js\node_modules\aws-sdk\lib\state_machine.js:26:10
at Request. (C:\Users\james.serverless\components\registry\npm\[email protected]\node_modules\serverless-next.js\node_modules\aws-sdk\lib\request.js:38:9)
at Request. (C:\Users\james.serverless\components\registry\npm\[email protected]\node_modules\serverless-next.js\node_modules\aws-sdk\lib\request.js:685:12) {
message: 'Request must be smaller than 69905067 bytes for the CreateFunction operation',
code: 'RequestEntityTooLargeException',
time: 2019-11-12T14:54:17.708Z,
requestId: '84af6699-36b9-4a2b-ab1c-88c468b47d19',
statusCode: 413,
retryable: false,
retryDelay: 86.11399872761147
}

135s » nextApp » RequestEntityTooLargeException: Request must be smaller than 69905067 bytes for the CreateFunction operation

nodabladam · 2019-11-12T23:05:21Z

I am slightly terrified of my Next.js app growing to the point where it exceeds the max direct upload size. I bought this theme and rolled it out to serverless as my first test of serverless and got a similar error.

Perhaps we can have a flag that enables deployment using the s3 zip method which has a higher size?
https://hackernoon.com/exploring-the-aws-lambda-deployment-limits-9a8384b0bec3

danielcondemarin · 2019-11-13T09:05:45Z

@jschimmoeller could you provide what is inside your .serverless_nextjs directory including sizes please?

jschimmoeller · 2019-11-13T13:44:11Z

@danielcondemarin ok as you will know there are 2 directories in this folder.

api-lambda - pages/api - i have 52 items with the average size 6.9MB

default-lambda - pages - i have 23 items with the average size 8.2MB

also if i zip up these directories the sizes are
api-lambda 50 mb
default-lambda 41mb

danielcondemarin · 2019-11-13T16:07:07Z

There is a lot of duplication across all page bundles when building using the next serverless target. There should be a way to output the common chunks across page bundles in a separate file which would greatly reduce the lambda artifact size. That would involve going deep into the webpack config that next implements for the serverless target though.
Happy to listen to any other suggestions like @jschimmoeller ’s

nodabladam · 2019-11-13T20:19:36Z

Related side note...I noticed an issue open that next.js serverless doesn't enable treeshaking:
vercel/next.js#8956

Seems strange to me given all the focus and performance improvements lately Next seems to be putting into serverless but perhaps it is true? Would something like this have any positive effect?
https://www.npmjs.com/package/serverless-plugin-optimize

hnprashanth · 2019-11-23T15:54:31Z

Would something like this have any positive effect?
https://www.npmjs.com/package/serverless-plugin-optimize

Tried this plugin, didn't help

Enalmada · 2019-11-24T18:02:25Z

If you put this in your webpack config, it might help as a workaround until treeshaking is fixed in Next.js serverless (vercel/next.js#8956)

      config.optimization.minimize = true;

(Note that this might increase build time so you would want to make this only happen during production build...probably using using phases.)

vishwasnavadak · 2019-11-26T08:13:37Z

If anybody is facing this issue has opted out of Automatic Static Optimisation in _app.js and if your app has few pages which can be statically rendered, then try removing the getInitialProps from _app.js and place it in the pages which needs the SSR. This will decrease the bundle size.

danielcondemarin · 2019-11-26T08:38:25Z

If anybody is facing this issue has opted out of Automatic Static Optimisation in _app.js and if your app has few pages which can be statically rendered, then try removing the getInitialProps from _app.js and place it in the pages which needs the SSR. This will decrease the bundle size.

Great advice! With the next component you can have an indefinite number of static pages as they’re stored in S3. On the other hand, SSR pages are zipped up and used in lambda@edge so the lambda artifact size limit applies there.

jschimmoeller · 2019-12-02T14:43:30Z

thanks for the recommendation; i did try both and each still failed.

config.optimization.minimize = true; was able to build it via next (which took a very long time 15 plus minutes) however when I ran it via "serverless" i get the following error:

`

C:\Users\james\projects\new-refiner>serverless

error:
Error: Command failed with exit code 1 (Unknown system error -1): node_modules/.bin/next build
at makeError (C:\Users\james.serverless\components\registry\npm\[email protected]\node_modules\serverless-next.js\node_modules\execa\lib\error.js:59:11)
at handlePromise (C:\Users\james.serverless\components\registry\npm\[email protected]\node_modules\serverless-next.js\node_modules\execa\index.js:112:26)
at processTicksAndRejections (internal/process/task_queues.js:85:5) {
command: 'node_modules/.bin/next build',
exitCode: 1,
exitCodeName: 'Unknown system error -1',
stdout: 'Creating an optimized production build...',
stderr: 'events.js:180\r\n' +
" throw er; // Unhandled 'error' event\r\n" +
' ^\r\n' +
'\r\n' +
'Error: write UNKNOWN\r\n' +
' at ChildProcess.target._send (internal/child_process.js:806:20)\r\n' +
' at ChildProcess.target.send (internal/child_process.js:676:19)\r\n' +
' at ChildProcessWorker.send (C:\Users\james\projects\new-refiner\node_modules\jest-worker\build\workers\ChildProcessWorker.js:286:17)\r\n' +
' at WorkerPool.send (C:\Users\james\projects\new-refiner\node_modules\jest-worker\build\WorkerPool.js:32:34)\r\n' +
' at Farm._process (C:\Users\james\projects\new-refiner\node_modules\jest-worker\build\Farm.js:129:10)\r\n' +
' at onEnd (C:\Users\james\projects\new-refiner\node_modules\jest-worker\build\Farm.js:122:12)\r\n' +
' at ChildProcessWorker._onProcessEnd (C:\Users\james\projects\new-refiner\node_modules\jest-worker\build\workers\ChildProcessWorker.js:280:14)\r\n' +
' at ChildProcessWorker.onMessage (C:\Users\james\projects\new-refiner\node_modules\jest-worker\build\workers\ChildProcessWorker.js:220:14)\r\n' +
' at ChildProcess.emit (events.js:203:13)\r\n' +
' at emit (internal/child_process.js:876:12)\r\n' +
"Emitted 'error' event at:\r\n" +
' at internal/child_process.js:810:39\r\n' +
' at processTicksAndRejections (internal/process/task_queues.js:75:11) {\r\n' +
" errno: 'UNKNOWN',\r\n" +
" code: 'UNKNOWN',\r\n" +
" syscall: 'write'\r\n" +
'}',
all: 'Creating an optimized production build...\n' +
'events.js:180\r\n' +
" throw er; // Unhandled 'error' event\r\n" +
' ^\r\n' +
'\r\n' +
'Error: write UNKNOWN\r\n' +
' at ChildProcess.target._send (internal/child_process.js:806:20)\r\n' +
' at ChildProcess.target.send (internal/child_process.js:676:19)\r\n' +
' at ChildProcessWorker.send (C:\Users\james\projects\new-refiner\node_modules\jest-worker\build\workers\ChildProcessWorker.js:286:17)\r\n' +
' at WorkerPool.send (C:\Users\james\projects\new-refiner\node_modules\jest-worker\build\WorkerPool.js:32:34)\r\n' +
' at Farm._process (C:\Users\james\projects\new-refiner\node_modules\jest-worker\build\Farm.js:129:10)\r\n' +
' at onEnd (C:\Users\james\projects\new-refiner\node_modules\jest-worker\build\Farm.js:122:12)\r\n' +
' at ChildProcessWorker._onProcessEnd (C:\Users\james\projects\new-refiner\node_modules\jest-worker\build\workers\ChildProcessWorker.js:280:14)\r\n' +
' at ChildProcessWorker.onMessage (C:\Users\james\projects\new-refiner\node_modules\jest-worker\build\workers\ChildProcessWorker.js:220:14)\r\n' +
' at ChildProcess.emit (events.js:203:13)\r\n' +
' at emit (internal/child_process.js:876:12)\r\n' +
"Emitted 'error' event at:\r\n" +
' at internal/child_process.js:810:39\r\n' +
' at processTicksAndRejections (internal/process/task_queues.js:75:11) {\r\n' +
" errno: 'UNKNOWN',\r\n" +
" code: 'UNKNOWN',\r\n" +
" syscall: 'write'\r\n" +
'}',
failed: true,
timedOut: false,
isCanceled: false,
killed: false,
signal: undefined
}

491s » nextApp » Error: Command failed with exit code 1 (Unknown system error -1): node_modules/.bin/next build
`

since the optimization was causing an issue, I commented it out and removed from _app.js the getInitialProps method; it failed with the following error:

`
C:\Users\james\projects\new-refiner>serverless

error:
RequestEntityTooLargeException: Request must be smaller than 69905067 bytes for the CreateFunction operation
at Object.extractError (C:\Users\james.serverless\components\registry\npm\[email protected]\node_modules\serverless-next.js\node_modules\aws-sdk\lib\protocol\json.js:51:27)
at Request.extractError (C:\Users\james.serverless\components\registry\npm\[email protected]\node_modules\serverless-next.js\node_modules\aws-sdk\lib\protocol\rest_json.js:55:8)
at Request.callListeners (C:\Users\james.serverless\components\registry\npm\[email protected]\node_modules\serverless-next.js\node_modules\aws-sdk\lib\sequential_executor.js:106:20)
at Request.emit (C:\Users\james.serverless\components\registry\npm\[email protected]\node_modules\serverless-next.js\node_modules\aws-sdk\lib\sequential_executor.js:78:10)
at Request.emit (C:\Users\james.serverless\components\registry\npm\[email protected]\node_modules\serverless-next.js\node_modules\aws-sdk\lib\request.js:683:14)
at Request.transition (C:\Users\james.serverless\components\registry\npm\[email protected]\node_modules\serverless-next.js\node_modules\aws-sdk\lib\request.js:22:10)
at AcceptorStateMachine.runTo (C:\Users\james.serverless\components\registry\npm\[email protected]\node_modules\serverless-next.js\node_modules\aws-sdk\lib\state_machine.js:14:12)
at C:\Users\james.serverless\components\registry\npm\[email protected]\node_modules\serverless-next.js\node_modules\aws-sdk\lib\state_machine.js:26:10
at Request. (C:\Users\james.serverless\components\registry\npm\[email protected]\node_modules\serverless-next.js\node_modules\aws-sdk\lib\request.js:38:9)
at Request. (C:\Users\james.serverless\components\registry\npm\[email protected]\node_modules\serverless-next.js\node_modules\aws-sdk\lib\request.js:685:12) {
message: 'Request must be smaller than 69905067 bytes for the CreateFunction operation',
code: 'RequestEntityTooLargeException',
time: 2019-12-02T14:41:00.854Z,
requestId: '9b4fb4c9-df53-4ab8-acd9-9b15fefe597e',
statusCode: 413,
retryable: false,
retryDelay: 13.134552790061571
}

94s » nextApp » RequestEntityTooLargeException: Request must be smaller than 69905067 bytes for the CreateFunction operation

C:\Users\james\projects\new-refiner>
`

kartikag01 · 2020-04-30T10:42:01Z

I have tried modifying webpack config to,

config.optimization.splitChunks.cacheGroups = { }

and this worked,

But what i want config is

    config.optimization.splitChunks.cacheGroups = {
      vendor: {
        name: "vendor",
        test: /[\\/]node_modules[\\/](react|react-dom|my-package)[\\/]/,
        chunks: "all",
        enforce: true,
      },
    };

this starts to gives error RequestEntityTooLargeException.

danielcondemarin · 2020-05-02T21:40:50Z

FYI - Next.js recently added a new build target called "serverless trace". I haven't had time to look into it properly but looks like it would solve this issue. vercel/next.js#8246

If someone has time to do some initial investigations that would be much appreciated 🙏

Enalmada · 2020-05-02T22:52:04Z

I do think that serverless trace could make all serverless-next pages load faster because everything would be so much smaller
vercel/next.js#8956 (comment)

danielcondemarin · 2020-05-02T23:10:31Z

I do think that serverless trace could make all serverless-next pages load faster because everything would be so much smaller
zeit/next.js#8956 (comment)

I am very keen on getting this working. It would not just solve this particular issue with the size of the deployment artefact but also speed up deployments considerably. In terms of page load I don't think there will be much (if any) improvements as this only affects server side artefact size.

@Enalmada Have you tried using the serverless trace with this project? It may give us a rough idea of what needs to be done to support it.

danielcondemarin · 2020-05-02T23:31:07Z

So I've been doing a bit of investigation around this already.

The serverless-trace target does not transpile node_modules via webpack like theserverless target does. This is what leads to smaller page bundles which is great.

What this means is this project has to zip up the node_modules and upload that to Lambda@Edge. The good news is that's exactly what @now/next already does here so we could replicate it. Also, we can use https://github.com/zeit/node-file-trace to track exactly which node_modules need zipping up!

Enalmada · 2020-05-03T01:04:23Z

I tried passing serverless-trace but quickly realized the errors were just a bit beyond my ability. In terms of performance I was thinking of cold starts...that server side artifact being smaller might help with that.

danielcondemarin · 2020-05-03T21:06:35Z

I have got a working proof of concept using the approach I mentioned in my comment above. This is what Zeit NOW uses in their stack as well so I'm considering making experimental-serverless-trace the default target. Watch this space I should have something ready for testing soon.

mertyildiran · 2020-05-07T05:41:42Z

I also got this error when I tried to deploy ~4000 pages website into AWS Lambda:

  error:
  { RequestEntityTooLargeException: Request must be smaller than 69905067 bytes for the UpdateFunctionCode operation
    at Object.extractError (/home/mertyildiran/.serverless/components/registry/npm/[email protected]/node_modules/aws-sdk/lib/protocol/json.js:51:27)
    at Request.extractError (/home/mertyildiran/.serverless/components/registry/npm/[email protected]/node_modules/aws-sdk/lib/protocol/rest_json.js:55:8)
    at Request.callListeners (/home/mertyildiran/.serverless/components/registry/npm/[email protected]/node_modules/aws-sdk/lib/sequential_executor.js:106:20)
    at Request.emit (/home/mertyildiran/.serverless/components/registry/npm/[email protected]/node_modules/aws-sdk/lib/sequential_executor.js:78:10)
    at Request.emit (/home/mertyildiran/.serverless/components/registry/npm/[email protected]/node_modules/aws-sdk/lib/request.js:683:14)
    at Request.transition (/home/mertyildiran/.serverless/components/registry/npm/[email protected]/node_modules/aws-sdk/lib/request.js:22:10)
    at AcceptorStateMachine.runTo (/home/mertyildiran/.serverless/components/registry/npm/[email protected]/node_modules/aws-sdk/lib/state_machine.js:14:12)
    at /home/mertyildiran/.serverless/components/registry/npm/[email protected]/node_modules/aws-sdk/lib/state_machine.js:26:10
    at Request.<anonymous> (/home/mertyildiran/.serverless/components/registry/npm/[email protected]/node_modules/aws-sdk/lib/request.js:38:9)
    at Request.<anonymous> (/home/mertyildiran/.serverless/components/registry/npm/[email protected]/node_modules/aws-sdk/lib/request.js:685:12)
  message:
   'Request must be smaller than 69905067 bytes for the UpdateFunctionCode operation',
  code: 'RequestEntityTooLargeException',
  time: 2020-05-07T05:24:59.307Z,
  requestId: 'dc34cd8b-1215-4f50-a55f-61ce277c4af1',
  statusCode: 413,
  retryable: false,
  retryDelay: 74.52977207528329 }

There are actually 30 - 40 serverless functions in our application. By page, I mean the different pages created by dynamic routes which only contiant getStaticProps and getStaticPaths. When I tried ~500 pages it didn't exceed this limit. But when I tried ~4000 pages it exceeded the limit. Clearly the function size is increasing by the dynamic page count which should not happen.

danielcondemarin · 2020-05-07T07:15:52Z

There are actually 30 - 40 serverless functions in our application. By page, I mean the different pages created by dynamic routes which only contiant getStaticProps and getStaticPaths. When I tried ~500 pages it didn't exceed this limit. But when I tried ~4000 pages it exceeded the limit. Clearly the function size is increasing by the dynamic page count which should not happen.

The function size will increase depending on how many server side rendered pages you have. Unfortunately next's serverless target doesn't dedup common dependencies across page bundles, which is why they've introduced a new target, the serverless-trace which I will be adding pretty soon to the project. It still outputs page bundles but common dependencies in the node_modules are marked as externals in webpack, so there is no duplication.

mertyildiran · 2020-05-07T10:03:05Z

@danielcondemarin OK, let me rephrase what I said:

With serverless function, I mean: pages/post/[id].js

and with page, I mean:

/post/1.html
/post/2.html
...
/post/3999.html
/post/4000.html

If there are 500 posts in the database, I can deploy the website successfully. But if there are 4000 posts in the database then I get RequestEntityTooLargeException and deployment fails.

So I don't see any relation between node_modules or webpack and the number of posts in my database. I mean how does it create duplicate dependencies for /post/1.html and /post/2.html?

danielcondemarin · 2020-05-07T11:15:12Z

@mertyildiran I understand now. They're all prerendered static pages. They should not end up in the lambda@edge artefact.
Could you check your .serverless_nextjs/default-lambda folder. It shouldn't have html pages in it. Same goes for .serverless_nextjs/api-lambda

mertyildiran · 2020-05-07T12:18:04Z

@danielcondemarin there are no HTML files but in .serverless_nextjs/default-lambda there are tons of prerendered JSON files. I think they don't suppose to be there?

mertyildiran · 2020-05-07T12:22:27Z

@danielcondemarin I think we should add a control like !isJSONPage to here. Let me do that, I'm preparing another PR.

danielcondemarin · 2020-05-07T13:13:04Z

@danielcondemarin there are no HTML files but in .serverless_nextjs/default-lambda there are tons of prerendered JSON files. I think they don't suppose to be there?

Good catch! You're right they shouldn't be there.

This only became an issue after getStaticProps was introduced.

danielcondemarin · 2020-05-17T21:58:26Z

Hi folks 👋

Some good news, I've managed to get the new serverless-trace target working on serverless-next.js! It means node_modules are not bundled anymore within the page bundles generated by next build. Instead node_modules are treated as externals (in webpack) therefore the page bundles don't contain duplicated dependencies code.
There are still some optimisations I could make, like compressing the node_modules, same to what Now does.
However I am hoping with the current implementation it's good enough and solve this issue altogether.

Ideally this will be tested with a large app. could I have some volunteers please?
It is currently released in [email protected]. All you'd need to do is make sure you are targeting experimental-serverless-trace in your next.config.js:

module.exports = {
  target: "experimental-serverless-trace",
  ...
}

# serverless.yml
yourApp:
  component:  [email protected]
  ...

Don't be put off by the target being "experimental". This is actually what Now uses under the hood for serverless deployments. I've pretty much replicated the same the now-next builder does.

Reference PR: #405

thiagozf · 2020-05-18T14:51:34Z

@danielcondemarin Thank you for this awesome feature! I've tested [email protected] with a large(ish) application and the default lambda size went from 23.1 MB down to 4.0 MB. However, I had to do some minor tweaks to get it working.

PS.: I'm not sure if I should open separate issues, so I'll just report the said tweaks below.

1. next.config.js is not normalized
There is a new function called isServerlessTraceTarget that checks if we are using experimental-serverless-trace. However, one can also export a function from next.config.js (see docs) and we are not considering this scenario. To get it working, I've patched the function to return true, but ideally we should normalize the config import, like Next.js does here.

2. Next config dir is being used as the base path for "@zeit/node-file-trace"
The Next.js application I'm working on is part of a monorepo. Its dependencies (node_modules) are not located on ${nextConfigDir}, but on the repository root. Since the base param is being set to ${nextConfigDir} here and here, @zeit/node-file-trace was ignoring all my dependencies :(. ~~Since I'm triggering my deploys from the repo root, I've removed the base param and used the default process.cwd(), but I don't know if this is the best approach to fix this case.~~ EDIT: I think this is a problem with @zeit/node-file-trace and should be fixed there. I'm going to send a PR.

3. Wrong API Lambda referenced on CloudFront distribution
I don't think this one is related to #405, but probably to #282. When I tried to deploy my app for the first time, the component threw an error saying that "Setting custom cache behaviour for api/ route "api" is not supported". I had to change my CloudFront config on serverless.yml from api to api/* and then the app was deployed successfully. However, my CloudFront distribution was now pointing the api/* path to the default Lambda instead of the API Lambda 🤔. I think it is happening because the default Lambda CloudFront config overwrites it here. I'm not sure how I'm supposed to configure the API's CloudFront distribution now, but I need to forward some headers (Authorization, Accept, etc) to the API handlers. Any tips here?

krish-dev · 2020-06-03T13:45:45Z

Hi @danielcondemarin

I did some modification and its works for me. Please 🙏 review the PR and let me know, what do you think.

krish-dev · 2020-06-03T13:52:14Z

sorry @thiagozf I didn't notice you already created PR for the same fix.

thiagozf · 2020-06-03T13:56:34Z

@danielcondemarin I've submitted PR #425 that should fix both chunks and monorepo issues by using .next/serverless as the base path when calculating the file destination. All my apps are running fine with the new target now 🔥🚀

@krish-dev Hey! I think we were working at the same thing at the same time.

danielcondemarin · 2020-06-03T21:57:13Z

@thiagozf @krish-dev Thank you both for the amazing contributions to this issue.

I've now merged & published #425 in 1.12.0-alpha.8. If you are both happy with this release I'll be closing this issue and getting this stuff into stable ASAP 🚀
Thanks 🙏

krish-dev · 2020-06-04T06:30:22Z

Thanks, @danielcondemarin for the new release. Works like a champ 👍 . great work @thiagozf 🙌

krish-dev · 2020-06-04T08:53:36Z

I just want to mention one observation. the deployment time highly increased. Previously my total lambda bundle size was around 50 MB (using [email protected]) and static assets nearly 4.3 MB. Total deployment time was near about 4 min (including build time). However, currently, the total lambda bundle down to 17 MB (using traced enabled) and static assets are the same as previously. But now its takes around 25 min to deploy. Noticed its takes a long time to upload not for build generation. @thiagozf what's your observation? 👀

thiagozf · 2020-06-04T11:53:46Z

Hey @krish-dev, sorry to hear that. My build time in a large(ish) app has increased some seconds, but the deploys are faster. Has this started after #425? By build generation, you mean the Serverless Component build or the Next.js build? Can you deploy your app with the serverless --debug and confirm if "Packaging lambda code from ..." or "Uploading <your lambda name> lambda code."are the steps taking a long time? Or is it before? Does the .serverless_nextjs folder size changes when you build with 1.12.0-alpha.7 and 1.12.0-alpha.8? My patch happens before the upload and, if your final bundle size is smaller, I don't see a reason for the deploy time increase.

kenberkeley · 2020-06-07T02:21:42Z

It seems that 1.12.0-alpha.8 is yet to support monorepo (I am using Yarn workspace). The workaround is nohoist all npm packages...

It would be great if we can use serverless-plugin-monorepo.

But anyway, thanks everyone's efforts!

thiagozf · 2020-06-07T19:26:34Z

@kenberkeley Do you have a minimum repro? I was able to get it working in a monorepo setup.

kenberkeley · 2020-06-08T01:36:12Z

Only scoped packages are included in the node_modules which will be deployed.

repo.zip

thiagozf · 2020-06-08T14:57:26Z

@kenberkeley This is happening because the component uses process.cwd() as the base path for node-file-trace and any files/folders above the base are ignored in the listing and analysis.

@danielcondemarin Can we have the base path for node-file-trace as a component input, which defaults to process.cwd()? Another alternative is to symlink node_modules, but I think this is out of the scope for the component and could be done with a custom build script.

krish-dev · 2020-06-09T02:49:16Z

Hey @krish-dev, sorry to hear that. My build time in a large(ish) app has increased some seconds, but the deploys are faster. Has this started after #425? By build generation, you mean the Serverless Component build or the Next.js build? Can you deploy your app with the serverless --debug and confirm if "Packaging lambda code from ..." or "Uploading <your lambda name> lambda code."are the steps taking a long time? Or is it before? Does the .serverless_nextjs folder size changes when you build with 1.12.0-alpha.7 and 1.12.0-alpha.8? My patch happens before the upload and, if your final bundle size is smaller, I don't see a reason for the deploy time increase.

Hey @thiagozf sorry for the delayed response. it's working perfectly, probably that day I had the internet connectivity issue, tested full process with BitBucket Pipeline and it increased few second only which is expected.

danielcondemarin · 2020-06-14T20:31:26Z

@danielcondemarin Can we have the base path for node-file-trace as a component input, which defaults to process.cwd()? Another alternative is to symlink node_modules, but I think this is out of the scope for the component and could be done with a custom build script.

Could we not follow symlinks to solve this instead? Similar to what now-next does - https://github.com/vercel/vercel/blob/master/packages/now-next/src/utils.ts#L521.

danielcondemarin · 2020-07-04T14:26:53Z

@kenberkeley I had a look at your repo.zip example and an option could be to move "deploy": "serverless" from the client/ package.json to the monorepo root. You would need to do this in combination with nextConfigDir: packages/client documented in the inputs.

kenberkeley · 2020-07-04T22:27:25Z

@danielcondemarin 🙏🙏🙏

jaypeng2015 · 2020-07-23T07:52:27Z

@danielcondemarin I am having this similar issue with mono repo so I also tried to deploy from the root. However I get this error at the end:

Error: Source and destination must not be the same.
    at /root/.serverless/components/registry/npm/@sls-next/[email protected]/node_modules/fs-extra/lib/util/stat.js:39:17
    at cb (util.js:207:31)
    at processTicksAndRejections (internal/process/task_queues.js:84:21)

Our serverless.yml looks like:

the-project-name:
  component: '@sls-next/[email protected]'
  inputs:
    bucketName: the-bucket-name
    memory: 1024
    nextConfigDir: './apps/web'
    nextStaticDir: './apps/web/public'
    timeout: 30
    runtime: 'nodejs12.x'
    useServerlessTraceTarget: true

I also tried to understand what the destination looked like and it was:

BigIntStats {

dev: 16777220n,

mode: 33188n,

nlink: 1n,

uid: 501n,

gid: 20n,

rdev: 0n,

blksize: 4096n,

ino: 8690198267n,

size: 1002828n,

blocks: 1960n,

atimeMs: 1595479257372n,

mtimeMs: 1595479159278n,

ctimeMs: 1595479159278n,

birthtimeMs: 1595479159272n,

atimeNs: 1595479257372092099n,

mtimeNs: 1595479159278799714n,

ctimeNs: 1595479159278799714n,

birthtimeNs: 1595479159272092562n,

atime: 2020-07-23T04:40:57.372Z,

mtime: 2020-07-23T04:39:19.278Z,

ctime: 2020-07-23T04:39:19.278Z,

birthtime: 2020-07-23T04:39:19.272Z

}

Could you please help to understand what's happening here?

dphang · 2020-08-12T18:04:00Z

@jaypeng2015 there is a separate thread on this: #461 (comment), I posted my findings. For me it was because it was somehow trying to copy my ts/tsx files from src and pages into the Lambda handler folder (but it shouldn't be copied). Because it had ../../ the src and dest ended up the same, which is what this error message was. My quick workaround was to manually add a check to build.js (compiled from

serverless-next.js/packages/libs/lambda-at-edge/src/build.ts

Lines 112 to 116 in d0a6300

    
           .filter((file) => { 
        
             // exclude "initial" files from lambda artefact. These are just the pages themselves 
        
             // which are copied over separately 
        
             return !reasons[file] || reasons[file].type !== "initial"; 
        
           })

) to skip filepaths except those with node_modules in the path.

nodabladam · 2020-08-12T18:39:20Z

I am trying out useServerlessTraceTarget and getting AccessDenied loading pages:

<Error>
<Code>AccessDenied</Code>
<Message>Access Denied</Message>
<RequestId>0371982A2F36F582</RequestId>
<HostId>RPip2l7OlJ9D8yDzjgCv+MLpmGdXJgIlXAKXaNvNdlFQwXUl4PO0YChUF5braxImlj+MZk142Sk=</HostId>
</Error>

Is there something obvious I might have done wrong or could try?

jaypeng2015 · 2020-08-12T22:52:07Z

In my example, I've found out the reason was because of yarn workspace.

apps
- web
  - src
  - next.config.js
  - .serverless_nextjs
packages
- foo
  - index.js

If the package is in root node_modules and is required in /apps/web, nodeFileTrace can find it then copyLambdaHandlerDependencies will copy it into /apps/web.
However if the package is a local private package, for example, /packages/foo, then the src and dest in copyLambdaHandlerDependencies function will be the same and then throw that error.

const dst = normalizeNodeModules(
          path.relative(this.serverlessDir, resolvedFilePath)
);

This piece of code returns '../../../../packages/foo/index.js', and then join with

join(this.outputDir, handlerDirectory, dst)

It becomes apps/web/.serverless_nextjs/default-lambda/../../../../packages/foo/index.js, which is packages/foo/index.js itself.

So the root reason it that normalizeNodeModules function only handles the files under a node_modules folder, but in this case, it cannot handle the packages in a yarn workspace.

zoozalp · 2020-08-18T12:29:17Z

For me it's the package.json file in the source root folder.

I have been able to deploy by using following steps;

set inputs.build.enabled = true in serverless.yml
build
deploy (fails)
disable build in serverless.yml
deploy again (works now)

I can't do this in CI/CD but at least this workaround helps me with urgent deployment needs for the time being. Not sure if it's creating other issues though.

zoozalp · 2020-08-18T12:31:22Z

For me it's the package.json file in the source root folder.

I have been able to deploy by using following steps;

set inputs.build.enabled = true in serverless.yml

build

deploy (fails)

disable build in serverless.yml

deploy again (works now)

I can't do this in CI/CD but at least this workaround helps me with urgent deployment needs for the time being. Not sure if it's creating other issues though.

Using @sls-next/[email protected] with [email protected] by the way.

zoozalp · 2020-08-18T16:30:26Z

I solved it by adding following to my CI/CD pipeline.

FILE=~/.serverless/components/registry/npm/@sls-next/[email protected]/node_modules/@sls-next/lambda-at-edge/dist/build.js
if [[ -f "$FILE" ]]; then
  sed -i.bak 's/return !reasons\[file\] || reasons\[file\]\.type !== "initial";/return \(!reasons\[file\] \|\| reasons\[file\]\.type !== "initial"\) \&\& file != "package.json";/' $FILE
fi

khaitranhq · 2020-10-30T03:11:58Z

@danielcondemarin I've updated next.config.js like you said

const withCSS = require('@zeit/next-css');

module.exports = withCSS({
  webpack: function (config) {
    config.module.rules.push({
      test: /\.(eot|woff|woff2|ttf|svg|png|jpg|gif)$/,
      use: {
        loader: 'url-loader',
        options: {
          limit: 100000,
          name: '[name].[ext]'
        }
      }
    });
    return config;
  },
  target: 'experimental-serverless-trace'
});

fe: 
  component: [email protected]

However I still get error like title when run command $ serverless.

dphang · 2021-05-22T06:45:41Z

I think long-running issue can be closed, the newest versions of Next.js + this component are now fairly optimized and you can also add minification to solve it if it is not enough. Please check the FAQs for related tips on this.

danielcondemarin added the next component label Nov 18, 2019

danielcondemarin mentioned this issue Apr 6, 2020

'Unzipped size must be smaller than 262144000 bytes' because code for all pages is deployed to each Lambda #335

Closed

mertyildiran mentioned this issue May 7, 2020

Fix the default-lambda filter by excluding the JSON files too #391

Merged

danielcondemarin mentioned this issue May 17, 2020

Serverless trace target support #405

Merged

thiagozf mentioned this issue Jun 3, 2020

Resolve dependencies using .next/serverless as base path #425

Merged

krish-dev mentioned this issue Jun 3, 2020

fix support for next/dynamic import #426

Closed

danielcondemarin mentioned this issue Jun 7, 2020

RequestEntityTooLargeException: Request must be smaller than 69905067 bytes for the CreateFunction operation #336

Closed

zoozalp mentioned this issue Aug 18, 2020

Error: Source and destination must not be the same. When using useServerlessTraceTarget #461

Closed

zoozalp added a commit to asyasoft/serverless-next.js that referenced this issue Aug 18, 2020

Fixes serverless-nextjs#461 and serverless-nextjs#236

f0a925e

dphang closed this as completed May 22, 2021

RequestEntityTooLargeException: Request must be smaller than 69905067 bytes for the CreateFunction operation #236

RequestEntityTooLargeException: Request must be smaller than 69905067 bytes for the CreateFunction operation #236

Comments

jschimmoeller commented Nov 12, 2019

nodabladam commented Nov 12, 2019 • edited Loading

danielcondemarin commented Nov 13, 2019

jschimmoeller commented Nov 13, 2019

danielcondemarin commented Nov 13, 2019

nodabladam commented Nov 13, 2019

hnprashanth commented Nov 23, 2019

Enalmada commented Nov 24, 2019 • edited Loading

vishwasnavadak commented Nov 26, 2019

danielcondemarin commented Nov 26, 2019

jschimmoeller commented Dec 2, 2019

kartikag01 commented Apr 30, 2020

danielcondemarin commented May 2, 2020 • edited Loading

Enalmada commented May 2, 2020

danielcondemarin commented May 2, 2020 • edited Loading

danielcondemarin commented May 2, 2020 • edited Loading

Enalmada commented May 3, 2020

danielcondemarin commented May 3, 2020

mertyildiran commented May 7, 2020

danielcondemarin commented May 7, 2020

mertyildiran commented May 7, 2020

danielcondemarin commented May 7, 2020 • edited Loading

mertyildiran commented May 7, 2020

mertyildiran commented May 7, 2020 • edited Loading

danielcondemarin commented May 7, 2020

danielcondemarin commented May 17, 2020 • edited Loading

thiagozf commented May 18, 2020 • edited Loading

krish-dev commented Jun 3, 2020

krish-dev commented Jun 3, 2020

thiagozf commented Jun 3, 2020

danielcondemarin commented Jun 3, 2020

krish-dev commented Jun 4, 2020

krish-dev commented Jun 4, 2020 • edited Loading

thiagozf commented Jun 4, 2020 • edited Loading

kenberkeley commented Jun 7, 2020 • edited Loading

thiagozf commented Jun 7, 2020

kenberkeley commented Jun 8, 2020 • edited Loading

thiagozf commented Jun 8, 2020 • edited Loading

krish-dev commented Jun 9, 2020

danielcondemarin commented Jun 14, 2020

danielcondemarin commented Jul 4, 2020 • edited Loading

kenberkeley commented Jul 4, 2020

jaypeng2015 commented Jul 23, 2020

dphang commented Aug 12, 2020 • edited Loading

nodabladam commented Aug 12, 2020

jaypeng2015 commented Aug 12, 2020 • edited Loading

zoozalp commented Aug 18, 2020

zoozalp commented Aug 18, 2020

zoozalp commented Aug 18, 2020 • edited Loading

khaitranhq commented Oct 30, 2020 • edited Loading

dphang commented May 22, 2021

nodabladam commented Nov 12, 2019 •

edited

Loading

Enalmada commented Nov 24, 2019 •

edited

Loading

danielcondemarin commented May 2, 2020 •

edited

Loading

danielcondemarin commented May 2, 2020 •

edited

Loading

danielcondemarin commented May 2, 2020 •

edited

Loading

danielcondemarin commented May 7, 2020 •

edited

Loading

mertyildiran commented May 7, 2020 •

edited

Loading

danielcondemarin commented May 17, 2020 •

edited

Loading

thiagozf commented May 18, 2020 •

edited

Loading

krish-dev commented Jun 4, 2020 •

edited

Loading

thiagozf commented Jun 4, 2020 •

edited

Loading

kenberkeley commented Jun 7, 2020 •

edited

Loading

kenberkeley commented Jun 8, 2020 •

edited

Loading

thiagozf commented Jun 8, 2020 •

edited

Loading

danielcondemarin commented Jul 4, 2020 •

edited

Loading

dphang commented Aug 12, 2020 •

edited

Loading

jaypeng2015 commented Aug 12, 2020 •

edited

Loading

zoozalp commented Aug 18, 2020 •

edited

Loading

khaitranhq commented Oct 30, 2020 •

edited

Loading