Wednesday, March 23, 2011

Day 39 - GET parameters from your module

First, google analytics says that I got a visitor from Islamabad. So, I want to say hello to all our fellow pakistani nginx lovers. OK, there is only one for now, but I wasn't counting on seeing any visitor from there. It looks like nginx is getting truly international...

My RRD module is working fine now and I'm getting plenty of nice graphs. Unfortunately, they are all pretty boring. Looking something like that:

Most of the time I'm only interested with what happened in the last hour or so and that might not be very easy to read if the values from the last hour are very different from the ones in the last 24 hours. Like in the picture above: the values at around 1:30-2:00 make the latest values completely impossible to read. So, I decided to use a start parameter to be able to easily change the starting point on the time axis of my graph. That's already a parameter supported by the rrdtool graph command and all I needed was to parse it from the query string.

First thing to know is that nginx Core module already parses the request and makes the values available as $arg_PARAMETER (see arg_PARAMETER section of the HttpCoreModule documentation). So, really there is no point in doing the job twice. But there are unfortunately a couple of places to look at before you get the right way of doing this (or at least what I consider so far the best way to do it: I might change my mind at any point in the future):

  • request->variables sounds like a perfectly reasonable place to look at. Well, well, well. You see, the variables in here are not the kind of variables you are looking for. They are less "variable" variables ;) than the one we are looking for. Basically the variables in this list are all the variables that exist regardless of what the request is. Things like $scheme (also called the protocol: http vs. https) or $is_args (are there arguments in the query string). It does not include the dynamic variables like $arg_PARAMETER or $cookie_COOKIE because before parsing the request, it doesn't even know what the variables are going to be.
  • ngx_http_variable_argument function. Except that it's static and you cannot access it from your module. So, not a good candidate although it does exactly what we want.
  • ngx_http_arg is a good choice. Except that this is really low-level and just does the parsing. You have to do all the allocations required around it and I got sick of doing the memory allocations by myself (and even more of checking the results, but that's a different story). So, I did not go for this.
  • I went for ngx_http_get_variable which does probably too much for what I want to do (the variable can be one of the "dynamic" variables but it can also be one of the more static ones, it will use the right way to extract it). But it offers a nice simple interface easy to remember:
    ngx_http_variable_value_t *
    ngx_http_get_variable(ngx_http_request_t *r, ngx_str_t *name,
                          ngx_uint_t key)

So, I was left pondering what the key parameter could be. After a little bit of research I found out it is a hash of the name parameter. The two are made distinct probably for optimisation reasons. This way, the code inside ngx_http_get_variable doesn't have to recompute the hash on every call. As my name is really a constant, I figured out that I would play nice with this optimization and keep both the name and the key as constants:

static ngx_str_t ARG_START = ngx_string("arg_start");
static ngx_uint_t ARG_START_KEY = ngx_hash_key(,

And that is exactly when the compiler started barking at me with initializer element is not constant. Not a nice guy this compiler, I tell you. Everything is constant from a logical standpoint but it doesn't like the call to a function in the initialization (it cannot optimize it to compile-time). Please note that it does not complain at ngx_string("arg_start") because this is actually a macro expanded by the pre-processor. So, I had to go for something like this:

static ngx_str_t ARG_START = ngx_string("arg_start");
static ngx_uint_t ARG_START_KEY;

static ngx_int_t ngx_http_rrd_init_process(ngx_cycle_t *cycle) {
    ARG_START_KEY = ngx_hash_key(, ARG_START.len);
    return NGX_OK;

And here is the result with an extra ?start=now-5h (after fixing a few stupid bugs, of course... ;)):

Looks good, no?

1 comment:

  1. Thanks! Useful post.
    For some reason I didn't came across ngx_http_get_variable and used ngx_http_get_variable_index + ngx_http_get_indexed_variable :)