I wrote an AWS lambda to get video metadata from youtube with youtube-dl (Code).
Lessons learned:
- There are Docker containers that match the AWS environment – these could be good for doing testing on a CI server.
- If you need to run a binary like ffmpeg, you need a version build for AWS linux – the Exodus bundler made this painless
- It took me a few iterations to get the right syntax for events coming from dynamodb – if you do test runs of a lambda, make sure to get a good sample event, or you’ll spend a lot of time re-writing your code going from test to real events.
- I had to write a second script to upload a CSV to DynamoDB. It appears that there is a cap on 15 lambdas running at once (currently unclear what is triggering that limitation)
- Write I/O is limited – make sure it all goes to /tmp
- Use the aws cli to count the rows in your dynamodb table to track status (aws dynamodb scan –table-name findlectures-videos –select COUNT)
- Because the lambda can shell out to run external code, you can run any language, as long as it fits in a 250 MB zip
- If you use the aws cli npm package, you shouldn’t need any configuration inside your node script, aside from setting a region – it picks up credentials locally from your account.