Richard Searle

home

100 fold speedup using Akka Streams

04 Feb 2016

This slideshare describes a re-implementation of a system that dumps transaction data from MySql into a csv file. The original implementation used Rails while the new implementation uses Akka Streams.

The standard Rails restriction to a single core is part of the problem, where Akka Streams utilizes all the available cores. However, given that the test hardware is only dual core; one might have expected at most a 2 fold improvement. The implication is that Rails is very inefficient, which is no surprise.

One might wonder what improvement might be seen from a simple, single threaded Java implementation.

Update

Further study of the sample code in the slides indicates an application level join between the transactions, products and customers.

The immediate question is thus: Why not use SQL joins to implement the entire process as a single SQL statement?

Such an implementation would be both faster and simpler. The slides indicate a single DB server, so the “excuse” that MySQL does not permit a distributed join does not apply.