jq - a JSON commandline tool (freifunk) - part I

Abusing the freifunk api to get more data

3 minute read

Freifunk logo

Everybody is interested in collecting data and build proper graphs out of it. In most cases the technical solution doesn’t matter until it shows graphs. Graph pr0n - basically.

I am running a freifunk node to support the idea of an open WLAN network that is useful to everyone. If you need a network connection for whatever reason - freifunk is one solution if you are in desperate need of a network connection that is free and reliable.

I also support their idea of providing a free network for everyone. After the latest suicide run in Munich such a network might come in handy to notify the family, friends, etc.

So a freifunk node is running at my side. One of the policy of freifunk is that you won’t get usage data for your router directly. Instead freifunk provides an api that you can use to query. The api is reasonable simple and it provides the result in JSON. In order to create graphs on my own I just need to query the api and store the data in a rrd database and display it. It is that simple. But wait JSON requires serious tools to parse it - or does it?

jq is a wonderful command line tool that helps you to parse and evaluate a JSON document.

Let’s have a run down for the whole thing and let’s see what’s required to use filters for jq for attribute values.

The freifunk api

The endpoint for the api is located at: **

https://api.freifunk.net/data/ffSummarizedDir.json

This url points to a list (called the directory in freifunk slang) with all the connected communities. I looked for the Munich community and use the url for the nodemaps. In the nodemaps you’ll find the url to the nodelist.

[...]
  "nodeMaps": [
      {
          "technicalType": "nodelist",
          "url": "https://ffmuc.net/data/nodelist.json",
          "interval": "1 minute"
      },
[...]      

Great that means that I will find all the nodes from the Munich community within the nodelist. The list itself will be updated every minute. That value is important for the rrd database settings later.

Calling the JSON node list gave me the following data:

{
  "version": "1.0.1",
  "nodes": [
    {
      "name": "sisters-of-eve",
      "id": "e8de27ad7fb4",
    [...]
      },
      "status": {
        "clients": 1,
    [...]
      }
    }
[...]
  ]
}

jq is a commandline parser that gets its input from standard in using a pipe. In order to get the file we’ll use the curl command with some parameters to turn off any output.

$ curl -s https://ffmuc.net/data/nodelist.json

Before we pass the JSON to jq let’s have a closer look at the data. Assume that we are interested in the number of clients that are connected to the WLAN access point.

Our logical search path would look like:

node -> id: ..... -> clients

We also should be aware of the data types that we are looking into. The nodes data is stored in an array. Where as the data for clients is being stored within a key value structure. I won’t replicate all the manual here from jq, it’s fantastic and you can find it here. However, if we query a certain value we must ask for a specific data type.

Let’s check out the full command line:

$ curl -s https://ffmuc.net/data/nodelist.json \
  jq ".nodes[] | select(.id == \"e8de27ad7fb4\") | .status.clients"

Let’s recap - we are taking all the nodes array, pass it on (pipe), search (select) for a specific id. Once this element has been selected from the array of nodes we pass it on with the pipe bar and print the key-value for the clients within the status.

This output can be used for a rrdtool command to feed a rrd database.

comments powered by Disqus