Finally solved the long-pending issue related to uploading the file larger than 10 GB through the AWS S3 plugin.
Before explaining how I resolved this issue, we should understand AWS S3 facts.
- AWS S3 allows uploading a max of 5GB in one request.
- AWS S3 recommends using the Multipart upload feature to upload a file larger than 5 GB.
- There is a restriction of 1000 parts for a single file for the multipart upload route.
In my case, my file's uploading truncating exactly at 9.8 GB even file was 13 GB+
I discussed this issue with aaPanel_Jose. His clue helped me a lot. He mentioned that aaPanel uses the Multipart route for uploading a file larger than 100 MB. Then I started finding the rule for the same in the code of the AWS S3 plugin itself.
I found the rule in aws_s3.py at line No 159. I made changes in that line as follow,
part_size = 10 * 1024 * 1024
to part_size = 20 * 1024 * 1024
That's it, now AWS S3 plugin is capable to upload 20 MB x 1000 parts = 20,000 MB (or 20GB)
My request aaPanel_Jose try to fix this issue officially, if possible. You can offer an input box to define a part size for multipart uploads.
Please feel free to reply in case of any query. Thank you. 🤝 🙂