Reduction 0.0.5 Changes​

Reduction 0.0.5 is out and is the first release that's been deployed to AWS. Getting to a good deployment workflow required some new features and bugfixes:

• ✨ Config parameters resolved at runtime
• ✨ Read config files from S3 URLs like s3://my-bucket/job.config
• ✨ Quieter logs with default log level
• 🐛 Terminate on AccessDenied and InvalidArgument Kinesis errors and retry network errors forever
• 🐛 Fix deadlock when there are fewer Kinesis shards than source readers
• 🐛 Reduce S3 API calls when restarting a job in an unhealthy cluster
• 🐛 Correctly store the sequence number (not the shard iterator) in Kinesis checkpoints
• 🐛 Handle expired iterators when shards are idle for more than 5m

Making good on the promise of multi-language support, Reduction now has a TypeScript SDK (github repository).

Getting back into the Node ecosystem, I am loving [Bun][bun] and think you should use it with Reduction if you can. You can get started by by initializing a new project and installing reduction-ts:

bun init
bun add reduction-ts

What's Possible vs What We Get​

Early in my career, I helped a friend install Wordpress right after his company spent $300,000 on a failed software project to build a "content library". This Wordpress instance became their content library and I got a $10,000 contract to build a prototype of a collaborative voting system. I had to call it a prototype because the owners didn't believe a single developer could build real software in a few weeks. They used it until their acquisition 9 years later.