Tag Archives: Splunk

Freedomotic logging for development and debugging Part 2

Ok, so now we have all our log messages coded in, with appropriate log levels like warn, info, debug. How do we control where they are sent, and how much we want to see?

Freedomotic  uses log4j to handle its logging output and the config file is at framework/freedomotic-core/src/main/resources/log4j.properties

The default at the moment is to log to the console but I want to log some to the console, and some to syslog. While I am developing, the console should show me important errors from all parts of the application and the output to syslog should show me the most granular output of the module I am working on, but also what was sent to the console. This allows me to see any important errors to do with my interactions with other parts of the application straight away on the console where I started the application, but also to see them in-line with my more detailed errors/debugging log via syslog.

Here is my log4j config:

# root logger
log4j.rootLogger=WARN, sysout, SYSLOG

# Output to console
log4j.appender.sysout=org.apache.log4j.ConsoleAppender
log4j.appender.sysout.threshold=WARN
log4j.appender.sysout.layout=org.apache.log4j.PatternLayout
log4j.appender.sysout.layout.ConversionPattern=%-5p <%t> [%C{1}]: %m%n

# Output to Syslog
log4j.appender.SYSLOG=org.apache.log4j.net.SyslogAppender
log4j.appender.SYSLOG.threshold=DEBUG
log4j.appender.SYSLOG.syslogHost=localhost
log4j.appender.SYSLOG.facility=local1
log4j.appender.SYSLOG.facilityPrinting=false
log4j.appender.SYSLOG.layout=org.apache.log4j.PatternLayout
log4j.appender.SYSLOG.layout.conversionPattern=%p <%t> [%C]: %m%n

# Detailed logging of my plugin
log4j.logger.com.freedomotic.plugins.devices.emoncms=DEBUG, SYSLOG
log4j.additivity.com.freedomotic.plugins.devices.emoncms=false
  • Root logger defines all the places I want to log to, and what starting level to send there. I have it set to WARN which means sysout and SYSLOG will get all messages of levels Warning or Error (since I am not expecting levels Critical, Alert or Emergency from my application – they are for the operating system).
  • The Console Appender is defined with a maximum level of WARN. This means that if I were to alter the root logger to DEBUG, I would still only get WARN and worse to it.
  • The Syslog Appender is defines with a maximum level of DEBUG, which means any log entry can potentially get there.
  • I am sending to the syslog facility ‘local1’ on localhost which I have configured in my syslog config to go to its own file (for tidyness).
  • My conversionPattern of ‘%p <%t> [%C]: %m%n’ is slightly different to what I have seen everywhere else on the internet (clue – it has a colon) and is specifically to stop a formatting error which messes up some entries when syslogging on my Mac. (I was getting spurious ‘Unknown’ and some truncation.)
  • The ‘log4j.additivity.com.freedomotic.emoncms=false’ line basically means don’t double-up messages – only add them if they aren’t already there.

So, now I can debug by tailing my log, or better still I can deal with more awkward problems with my favourite log tool, Splunk:

Splunk log analysis