Daily techtalk: Importance of latency

The most frustrating question what IT could get is that:

Why my application is so slow?
This problem first happens occasionally, so IT not really notice that there could be some problem which cause this situation. Everyone knows the typical answers:

* image source

Please check again later, and it will be fine!
On my desktop it runs fine, there could be some problem on your PC!
The problem in a lot of cases happens from concurrent resource use. With concurrent resource usage, the resource response time increases not linearly, much more exponentially. If our thinking about this problem, this problem not only affects the end users. It is also affecting the number of used software licenses. In many cases the licensing calculated on number of sockets, processor cores, used hosts. If we realize the connection between performance figures, with the improvement of the latency numbers, we can improve user experience, and also we can decrease software license usage.

This time the question is that, how we can put this into practice?

One good solution is the decrease of the disk storage system latency. In my practice, we were able to decrease the number of needed licenses from 50% to 10%. The main information which I learned from this lesson, CPU usage hasn't always come from computation tasks.

The other solution which much less of us realize, we can decrease the number of application layers. With the ease of object oriented programming, use a well structured, layered development, we continuously increasing the number of layers used.

For me the first project, which realized the importance of decrease of the layers, was the FusionIO reference project. In their example, they removed many layers, like filesystem, like RDBMS. They only used their API which directly stored the objects to their card.

Nine million IOPS from the singe PCIe card

This shows good example how they able to increase the performance of the card. The implementation not only increased the IOPS amount of the product, but also decreased the application latency. With faster program execution time, the application concurrency will also decrease. And again: This improvement is not linear! Much more exponential!

To have the best result, it is not enough, to only improve the disk latency, but we are also able to cut different application layers. For this, we need to check the application data pipeline end-to-end.

If we use all of our possibilities we are able to realize other aspects of improvements. With the application level latency decrease, we can increase the latency on the disk subsystem side, to decrease the system overall costs, and at the end of the day we can serve the same performance with much less cost.

Again on Custom Developments, which was already mentioned in my past post. With boxed solutions, we cannot decrease the levels of software components. Most of the software vendors optimize their solutions for faster time to market, than less resource need. With this conflict of interest, we need to realize that, if we utilize boxed solutions, we can only optimize on infrastructure level. Of course Boxed solutions have their advantages on faster implementation time. We need to discuss this situation with the Business Units, what is the importance? Faster time to market for the solution, or less costs?