Athena - Part 2 - Logging

Back to Part 1 - System Design | Forward to part 3 - Data Transformation

Part 2 - Logging the Data

This is by far the easiest part. If you're reading this guide then you probably know about setting up a lambda, but if not head over to

Run a Serverless "Hello, World!" with AWS Lambda tutorial on AWS.

The Logging

I used the pino logging library to create a custom analytics logger so that the log type would always be 'analytics'. This logs the correct details in CloudWatch with little impact on the performance of the host application.

const pino = require('pino')
// the default logger for other things
const logger = pino()
// an instance of the logger for analytics
const analyticsLogger = logger.child({ logtype: 'analytics' })
//... more code
// log the details required for analytics
analyticsLogger.info({
  request: event.path,
  requestId: context.awsRequestId,
  sessionId,
  country,
  city,
  region,
  latitude,
  longitude,
  viewer,
  customData,
})

This would log the following to CloudWatch logs:

{
  "logtype": "analytics",
  "request": "/the/request/path",
  "requestId": "c6af9ac6-7b61-11e6-9a41-93e8deadbeef",
  "sessionId": "g8e7r5a6-d2f3-4g5h-6j7k-8l9z0xbada55",
  "country": "United Kingdom",
  "city": "Shibden",
  "region": "England",
  "latitude": "21.509865",
  "longitude": "-0.118092",
  "viewer": "Chrome",
  "customData": {
    "foo": "bar"
  },
  "level": 30,
  "time": 1471277496517
}

Continue to part 3 to see what to do with this logged data >>