An Apicast module that logs API requests and responses

Description

This project is an Apicast module that logs API requests and responses for non-repudiation purposes.

How it works

This Apicast module intercepts API requests and sends them to a syslog server. The request, response, headers and additional information are serialized as JSON and sent to a syslog server.

Pre-requisites

This projects requires :

• an Apicast gateway
• a syslog server (such as syslog-ng or rsyslog)
• the lua-resty-logger-socket module

Installation

If not already done, start your syslog server and configure it to listen for TCP connections on port 601. An exemple is given below with syslog-ng:

oadm policy add-scc-to-user privileged -z default
oc new-app balabit/syslog-ng --name syslog-ng
oc volume dc/syslog-ng --add --name log --type emptyDir --mount-path /var/log/
oc create configmap syslog-ng --from-file=syslog-ng.conf
oc volume dc/syslog-ng --add --name=conf --mount-path /etc/syslog-ng/conf.d/ --type=configmap --configmap-name=syslog-ng

Then, update your apicast-staging to embed the required code, module and environment variables as explained:

Put resolver.conf in /opt/app-root/src/apicast.d/resolver.conf:

oc create configmap apicast.d --from-file=resolver.conf
oc volume dc/apicast-staging --add --name=apicastd --mount-path /opt/app-root/src/apicast.d/ --type=configmap --configmap-name=apicast.d

Put the lua-resty-logger-socket module in /opt/app-root/src/src/resty/logger/:

git clone https://github.com/cloudflare/lua-resty-logger-socket.git
oc create configmap lua-resty-logger-socket --from-file=lua-resty-logger-socket/lib/resty/logger/socket.lua
oc volume dc/apicast-staging --add --name=lua-resty-logger-socket --mount-path /opt/app-root/src/src/resty/logger/ --type=configmap --configmap-name=lua-resty-logger-socket

Put the verbose.lua module in /opt/app-root/src/src/custom/:

oc create configmap apicast-custom-module --from-file=verbose.lua
oc volume dc/apicast-staging --add --name=apicast-custom-module --mount-path /opt/app-root/src/src/custom/ --type=configmap --configmap-name=apicast-custom-module

Set the configuration required by verbose.lua as environment variables and re-deploy apicast:

oc env dc/apicast-staging APICAST_MODULE=custom/verbose
oc env dc/apicast-staging SYSLOG_PROTOCOL=tcp
oc env dc/apicast-staging SYSLOG_PORT=601
oc env dc/apicast-staging SYSLOG_HOST=syslog-ng.3scale.svc.cluster.local
oc rollout latest apicast-staging

NOTE: You need to adjust the value of SYSLOG_HOST to match your environment. Namely, make sure you are using a FQDN that resolves to your syslog server. If the syslog server is deployed in OpenShift, it needs to be in the same project as the apicast (of course, unless you are using a flat network...).

In an OpenShift environment, the SYSLOG_HOST would look like:

<service-name>.<project>.svc.cluster.local

WARNING: You cannot use a short name (ie syslog-ng). It has to be a FQDN. This is because nginx does not rely on the standard glibc API gethostbyname but uses instead a custom resolver.

Once, you get it to work on apicast-staging, you can do the same on apicast-production.

Message format

The requests and responses are serialized as follow:

{
  "request": {
    "request_id": "3b1b0d[...]",           # The unique ID of the request
    "raw": "R0VUIC8/dXN[...]",             # The raw request (request line + headers), base64 encoded
    "headers": {                           # The request headers as an object
      "host": "echo-api.3scale.net",
      "accept": "*/*",
      "user-agent": "curl/7.54.0"
    },
    "body": "3b1b0d587[...]",              # The body of the request, base64 encoded
    "method": "GET",                       # HTTP Method
    "start_time": 1503929520.684,          # The time at which the request has been received
    "uri_args": {                          # The decoded querystring as an object
      "foo": "bar"
    },
    "http_version": 1.1                    # The version of the HTTP protocol used to submit the request
  },
  "response": {
    "headers": {                           # The response headers as an object
      "cache-control": "private",
      "content-type": "application/json",
      "x-content-type-options": "nosniff",
      "connection": "keep-alive",
      "content-length": "715",
      "vary": "Origin"
    },
    "body": "ewogICJtZXRob2Qi[...]",       # The body of the response, base64 encoded
    "status": 200                          # The HTTP Status Code
  },
  "upstream": {                            # See http://nginx.org/en/docs/http/ngx_http_upstream_module.html#variables
    "response_length": "715",
    "header_time": "0.352",                             
    "addr": "107.21.49.219:443",
    "response_time": "0.352",
    "status": "200",
    "connect_time": "0.261"
  },
}

Development

First of all, setup your development environment as explained here.

Then, issue the following commands:

git clone https://github.com/nmasse-itix/apicast-logger.git
git clone https://github.com/3scale/apicast.git
cd apicast
luarocks make apicast/*.rockspec --local
ln -s ../apicast-logger custom

Configure your apicast as explained here.

export THREESCALE_DEPLOYMENT_ENV=sandbox
export THREESCALE_PORTAL_ENDPOINT=https://<YOUR-TOKEN-HERE>@<YOUR-TENANT-HERE>-admin.3scale.net
export APICAST_LOG_LEVEL=debug

Configure the module:

export SYSLOG_HOST=127.0.0.1.xip.io
export SYSLOG_PORT=1601
export SYSLOG_PROTOCOL=tcp
export APICAST_MODULE=custom/verbose

Then, you need to register a resolver in the nginx configuration (example using the Google DNS):

cat <<EOF > apicast/apicast.d/resolver.conf
resolver 8.8.8.8:

Finally, launch apicast:

bin/apicast -i 0 -m off

And in another terminal, launch netcat so that you can simulate a syslog server:

nc -l 1601

Troubleshooting

When troubleshooting, keep in mind that the underlying lua-resty-logger-socket module is asynchronous. When the logs cannot be sent to the syslog server, the error is caught ONLY UPON THE NEXT REQUESTS. So, you might have to send a couple requests before seeing errors in the logs.

If you need to troubleshoot DNS issue :

dig syslog-ng.3scale.svc.cluster.local
dig -p5353 @127.0.0.1 syslog-ng.3scale.svc.cluster.local