Juttle’s Cross Platform Optimization Strategy

Juttle can analyze data that is in any of several data stores. It has processors like reduce, sort, head, and tail that operate over a sequence of data points. For instance, head n emits the first n points it receives and drops the rest. reduce count() increments a counter for each data point and returns the total. The built-in implementations of these processors are written in Node.js.

These Node.js processors are functionally correct, but they are often not the most efficient way to operate on data stored in a given database. For instance, the fastest way to count the records in a SQL table is SELECT COUNT(*) FROM my_table. But to count the records in a table using the Node.js implementation of reduce count(), we’d have to SELECT * from my_table, build a Juttle data point from every record, and perform the count in Javascript. It would be much faster if Juttle knew it could use SELECT COUNT to calculate the result of reduce count(). The Juttle Optimizer is the piece in Juttle’s architecture that turns Juttle programs into efficient queries like this.

Continue reading “Juttle’s Cross Platform Optimization Strategy”

How does Bluebird promisify work?

High-performance code generation in Javascript

In describing the ConcurrencyMaster, I referred to Bluebird promisify as a magic function. Of course, it’s not really magical, it’s just a computer program. It only seemed magical because I didn’t understand how it worked. So this week, I’ve taken the opportunity to fill this hole in my knowledge by studying the internal workings of promisify. I like to learn about projects by fixing bugs in them, but Bluebird has no open bugs. Instead, we’ll just run through the working code and see what makes the magic happen. Here goes!

Continue reading “How does Bluebird promisify work?”