Error 413 #52

marcelinhov2 · 2023-05-09T14:45:35Z

Hey, first of all thanks for the solution. :)

I'm trying to do an implementation using that, I prepared everything and also removed all the auth piece (in order to test the failure I'm getting)

The point is that when I try to push a 5mb file to code commit, after implement everything you shared, I'm getting a 413 error.

After some research, I see the problem is related to pushing big files. Do you already faced it?
Maybe you can point me to the right direction here.

Thanks for the help 🙏 .

troyready · 2023-05-09T16:32:34Z

That's an interesting one @marcelinhov2 , I haven't run into anything like it. There definitely isn't an architectural issue with sending a 5mb (or much larger) - the Lambda functions don't see any of the files, just a hash.

What loggroup/function is that error from? None of the functions should be getting any particularly large requests.

I just ran a quick test from scratch to make sure nothing new was amiss:

npm ci
npx sls deploy -s troytest -r us-west-2 --verbose
aws cognito-idp admin-create-user --user-pool-id us-west-2_jIYHdGO5j --username tready --user-attributes Name=email,Value=troy@example.com Name=phone_number,Value="+12135551212" --message-action SUPPRESS --region us-west-2
aws cognito-idp admin-set-user-password --user-pool-id us-west-2_jIYHdGO5j --username tready --password Testpass@111 --permanent --region us-west-2
aws codecommit create-repository --repository-name testgitlfs --region us-west-2
# repo & LFS in place, now testing them
cd $(mktemp -d)
CC_REPO_URL=https://git-codecommit.us-west-2.amazonaws.com/v1/repos/testgitlfs
git init
git config credential."$CC_REPO_URL".helper '!aws codecommit credential-helper $@'
git config credential."$CC_REPO_URL".UseHttpPath true
git remote add origin $CC_REPO_URL

truncate -s 20M test.dat
git lfs track "*.dat"
git config -f .lfsconfig remote.origin.lfsurl https://eubor6lku0.execute-api.us-west-2.amazonaws.com/troytest
git add .gitattributes .lfsconfig test.dat
git commit -m "initial commit"
git push -u origin main

The push output looks like:

Username for 'https://eubor6lku0.execute-api.us-west-2.amazonaws.com': tready
Password for 'https://tready@eubor6lku0.execute-api.us-west-2.amazonaws.com': 
Locking support detected on remote "origin". Consider enabling it with:
  $ git config lfs.https://eubor6lku0.execute-api.us-west-2.amazonaws.com/troytest.locksverify true
Uploading LFS objects: 100% (1/1), 21 MB | 3.8 MB/s, done.                                                                                        
Enumerating objects: 5, done.
Counting objects: 100% (5/5), done.
Delta compression using up to 20 threads
Compressing objects: 100% (4/4), done.
Writing objects: 100% (5/5), 531 bytes | 531.00 KiB/s, done.
Total 5 (delta 0), reused 0 (delta 0), pack-reused 0
remote: Validating objects: 100%
To https://git-codecommit.us-west-2.amazonaws.com/v1/repos/testgitlfs
 * [new branch]      main -> main
branch 'main' set up to track 'origin/main'.

The LFS elements of the git config should look something like this:

$ cat .git/config
[core]
	repositoryformatversion = 0
	filemode = true
	bare = false
	logallrefupdates = true
[credential "https://git-codecommit.us-west-2.amazonaws.com/v1/repos/testgitlfs"]
	helper = !aws codecommit credential-helper $@
	UseHttpPath = true
[remote "origin"]
	url = https://git-codecommit.us-west-2.amazonaws.com/v1/repos/testgitlfs
	fetch = +refs/heads/*:refs/remotes/origin/*
[lfs]
	repositoryformatversion = 0
[lfs "https://eubor6lku0.execute-api.us-west-2.amazonaws.com/troytest"]
	access = basic
[branch "main"]
	remote = origin
	merge = refs/heads/main

$ cat .lfsconfig 
[remote "origin"]
	lfsurl = https://eubor6lku0.execute-api.us-west-2.amazonaws.com/troytest

If nothing stands out as being particularly different, increasing git verbosity (e.g. GIT_CURL_VERBOSE=1 GIT_TRACE=1 ) when pushing might give info pointing to where the issue is.

marcelinhov2 · 2023-05-09T17:04:19Z

Thanks @troyready , I'll try.

The log group I'm showing you is from a lambda that is triggered when a file is dropped in a specific S3 bucket. So my lambda get this file content and try to push to code commit.

The message is from here.

troyready · 2023-05-09T18:50:17Z

Ah, I see.

The primary issue here is that CodeCommit (or whatever other repo you'd be using) doesn't store the actual content of the large files, but instead replaces them with a text pointer as the file in the repo. When a git-lfs enabled git client then receives the file (e.g. during a clone), it reaches out to the git-lfs server (what this project provides) and swaps the file in the working tree with the real large one.

So for your design, a couple of thoughts:

If you have control over the file uploads as well, I'd see about switching them to hash in sha256. Then your function handler ^ wouldn't need to getObject on the new file as the hash should already be in the event or available via s3:GetObjectAttributes
The file commited to codecommit would look something like (I haven't tested this):

import { createHash } from 'node:crypto'

const fileContent = `version https://git-lfs.github.com/spec/v1
oid sha256:${createHash('sha256').update(s3Object.Body).digest('hex')}
size ${record.s3.object.size}
`

troyready · 2023-05-09T18:53:03Z

My original comment above was sent early on accident; updated now.

troyready · 2023-05-10T19:19:42Z

I don't have any advice about the missing lfsattributes file in the repo; maybe the commit with it wasn't pushed?
The thing you're trying to do is to implement the git-lfs client in Lambda. To do that would look something like:

Receive the S3 notification
Call the git-lfs server batch API (e.g. a node-fetch to https://yourapi.execute-api.region.amazonaws.com/stage/objects/batch) requesting an upload of the new object.
It will respond with a pre-signed S3 URL to which you upload the file
Then you commit the pointer file in the repo. The output from my example looks mostly correct, just needs to drop the extra 4 spaces at the start of lines 2 &3)

marcelinhov2 · 2023-05-11T13:23:54Z

Hey @troyready , thanks for all the help. Apparently it worked.

I didn't understood this part
Then you commit the pointer file in the repo. The output from my example looks mostly correct, just needs to drop the extra 4 spaces at the start of lines 2 &3)

Maybe I'm still missing something here.

troyready · 2023-05-11T18:29:32Z

I mean that the repo file contents should look like this instead:

const fileContent = `version https://git-lfs.github.com/spec/v1\noid:sha256:${createHash('sha256').update(s3Object.Body).digest('hex')}\nsize ${record.s3.object.size}\n`

(no extra spaces before oid & size)

marcelinhov2 · 2023-05-11T20:35:30Z

I'm having some problem, I didn't identified. but I'm moving forward lol. Now everything seems to be working, except when I try to pull the big files (stored in S3) with the git lfs fetch or git pull command, but It don't download the files. something is wrong from my side yet :(

troyready · 2023-05-11T22:03:20Z

For the client to substitute the files properly (replacing the text pointer files stored in the repo with the actual large file in the working tree) you need:

Git LFS installed & initialized on the client system
The client's repo settings configured for the appropriate filenames. You can see that happening in my example here, where git lfs track is creating the .gitattributes & .lfsconfig files and modifying the .git/config to specify the git lfs server

marcelinhov2 · 2023-05-11T22:13:06Z

Yeah, I didn't made the .git/config piece. going to work on that and try to make it work.

Also a good point, I'm using AWS_IAM as the authorizer instead of Cognito. As my lambda has all the roles set, API Gateway enables the request and do the commit. all the main flow is working, now it's a matter of make this link between the pointer and the file correct.

Thank you very much for all your help here.

troyready · 2023-05-11T22:49:44Z

My pleasure!

troyready closed this as completed May 11, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Error 413 #52

Error 413 #52

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Error 413 #52

Error 413 #52

Comments

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!