Performance considerations in Application logging

In Distributed Systems, Performance, Software Development, Software Engineering, web application by Prabhu Missier

Logging is an important part of an application’s architecture. And logging if used incorrectly can hamper the performance of your application. Let’s look at a few scenarios and ways to optimize performance.

1. Conditional logging and lazy logging

Logger.debug(‘This is a call to %s and the energy emitted is ‘, getMethodName(), getEnerygyEmitted());

The call to getMethodName() will execute even if the logging level configured is different. So for instance even if the logging level is set to “Error messages only” this call to getMethodName() will execute and the results then discarded.
To avoid this you can introduce conditional logging which checks if logging is enabled before executing the logger statement.
Alternatively modern logging frameworks such as Log4J 2 and slf4J support lambdas which execute lazily meaning they will execute only if necessary.
Ensure that logging levels are set appropriately and log statements are present only when necessary.

2. Calling complex functions to compute parameters

Logger.debug(‘This is a call to %s and the energy emitted is ‘, getMethodName(), getEnerygyEmitted());

Let’s look at the logger call above and let’s assume that the call to getEnergyEmitted() is a complex calculation making several calls to external services. Even though the actual call to the logger function may be I/O bound the call to getEnergyEmitted() could consume CPU and delay the execution of your application. So be conscious of the calls being made inside a logging statement,

3. Logging asynchronously

As explained above if a call could result in stalling your application one way out would be to avoid such calls altogether or hard code the values in the logging statement which is not feasible.

Modern frameworks such as log4J offer Async Appenders which prevent blocking and buffer messages in a queue which can be read at a later time by a dedicated thread.

4. Logging metadata

While emitting logs be conscious about the metadata being computed and logged. Metadata could be computed in your logging statement or could be implicitly added by the logging framework.

Do you actually need to print the timestamp with millisecond precision and do you need to print the thread name for each and every log statement. Are there any redundant or superfluous pieces of information being injected into your log statements by the logging framework. Have a look at how you configure the logging framework so as to remove any extraneous informantion.

5. Aggregator friendly logs

Aggregators like Splunk and ElasticSearch eventually use a format which enables them to process logs. For instance Logstash transforms logs into JSON before sending them to ElasticSearch. This unnecessary overhead can be avoided by using JSON directly in your log statements.
This however is part of a larger debate known as the “schema on write” vs “schema on read”. Your strategy should involve deciding whether you want your log storage to be more efficient as compared to your log search and querying capabilities.

Conclusion

In short logging is a topic which should not be taken lightly and has to be considered as a key consideration when working on the architecture of your application