Also, I have seen a visualization using Kinesis to-from a web interface, where there is not compute happening in AWS at all. It was really cool - they took audio input from the microphone via the browser, and streamed it through Kinesis to all the other browsers; compute happened in-browser.
This makes me wonder if, rather than using S3 as a backing store, you could have Lambda insert messages into Kinesis and have the browser read from the Kinesis stream.
Of course, the goal here is to build something with Lambda - otherwise I think you could do the whole thing with Kinesis and client-side compute.
I've built awsletter [1] as a quick PoC of the cloud-only web app some time ago. Last week (inspired by HN thread [2]) I've jumped into this project once again. In January Lambda was a big "wow" for me, but now I see some things that have to be taken into consideration:
1/ Fine-Grained Access Control is there [3], but it is performed on a pretty late stage of user-system interaction. When user assumed a role with some auth provider, then he/she can freely play with all AWS methods bundled in a well documented SDK [4].
2/ Codebase distribution means security distribution. You are authenticating user with FB/G+/Amz, then you have FGAC on DynamoDB table, then you need caller policy to call Lambda function remotely, then in Lambda function you need to check permissions one more time and the Lambda function have to have policy of what can be executed.
3/ In theory, Lambda functions are highly decoupled and testable (fn(payload, ctx)). But most of the time using Lambda makes sense in conjunction with other AWS services. Mocking Amazon's ecosystem isn't so easy.
4/ This model of computing may lead to oversized architectures, using "one more" AWS service to "close the loop", some strange solutions like watching/analyzing files in S3 buckets and so on...
I've seen a huge vendor lock-in from the beginning, I've been aware of potential security/architecture problems. These concerns are still valid for me. But at the same time, Lamba as a concept is still pretty exciting.
On top of that, it seems to encourage an extremely wasteful and expensive use of resources. The blog post claims that it "costs just a few dollars per month, and in theory scales near infinitely" but as soon as you start scaling, the cost is going to grow a lot.
As an example, take the use of S3 as a "cache" for the most recent messages in each channel. Suppose you're hosting a decent-sized collection of chat rooms, handling a total of 100 messages/second. The PUT requests alone would run you over $1000/month, even though the same amount of traffic could be easily served by a single small EC2 instance for a few percent of the cost.
S3 alone is expensive enough that it often pays to rent managed servers or instances elsewhere to act as a frontend cache for any external access to the files if you for whatever reason "must" use S3 (durability would be a good justification - getting that right is hard; getting it to the point where you trust it to be right is even harder)
E.g. 1TB worth of retrievals from S3 can buy me a dedicated server with 16TB of disk space and 30TB of inclusive traffic. Doesn't take a very big cache hit ratio before caching all external reads becomes profitable.
If cost is a consideration, it's pretty rare for AWS to be the way to go for anything but batch processing where you need instances or short periods of time in the first place.
Lambda is good at discrete units of work, in an application already integrated with AWS, that may come either frequently or infrequently, where you don't necessarily want to have an app server running the whole time. Things like
- generating a thumbnail for an image uploaded to an S3 bucket
- processing in response to queue events
- generating stats and metrics on a schedule or in a response to events
This is the typical unbalance of code + configuration. The code can be very clear but you need to read a lot of documents to be able to configure it in a particular system.
Interesting article. This is good, but for chat applications, I feel Lambda is exactly the technology to be not used unless you have deep pockets. The number of S3 PUT requests for caching chat messages will eventually grow high as the chat app scales.
Which part of the architecture causes the latency? The author says it's between 2 and 13 seconds and is faster the more users there are - why would that be the case?
