Day 43 - module config and auto/feature

Looking at other people modules gave me an opportunity to look closer at the config file they are providing. Most of them (especially the modules written by Igor himself) use the file auto/feature to figure out whether a library is available on your system and to figure out the directories where it's deployed. This way, it will work under the various flavours of Linux (Fedora/Redhat/CentOS or Debian/Ubuntu) or under the BSDs of the world.

Good thing is that I have my rrd-nginx-module handy. And I did not do a very good job on the config file. So, now is a good time to review/refactor this and do somethign nicer. Let's first have a look at what I have in there for now:

ngx_addon_name=rrd-nginx-module
HTTP_MODULES="$HTTP_MODULES ngx_http_rrd_module"
NGX_ADDON_SRCS="$NGX_ADDON_SRCS $ngx_addon_dir/ngx_http_rrd_module.c"
CORE_LIBS="$CORE_LIBS -lrrd_th"
CFLAGS="${CFLAGS/-Werror/-Wno-deprecated-declarations -Werror}"

Yes, this is ugly. But it works (at least on my Fedora Core 14) and there is even worse than that: it's nto int he right place: it's in the src directory whereas the "standard" is to put it in the root of the module. I told you: never paid much attention to this part of the process... But before we start improving this, let me justify/explain what I'm doing:

• ngx_addon_name appears during the configuration process to let you know that your configuration script was invoked:

  configuring additional modules
  adding module in ../rrd-nginx-module/
   + rrd-nginx-module was configured
  

• HTTP_MODULES is where you should add the name of the ngx_module_t variable defining your module that is exported by your code.
• NGX_ADDON_SRCS is fairly easy: where your source code is.
• CORE_LIBS adds the extra library to link with. I use rrd_th which is the thread-safe version of the RRDtool library.
• CFLAGS are altered because I use the rrd_init, rrd_open, rrd_close and rrd_free which are deprecated. They are deprecated but the replacement function (rrd_info_r) does a lot more than what I need. I might change my mind about this (and use the non-deprecated function) but for the beauty of it, let's assume there is no other way and the CFLAGS MUST be altered.

Now, my first discovery was to realise that the nginx distribution comes with a file named auto/feature which is used by Igor's modules to display the lines like the following:

checking for system md library ... not found
checking for system md5 library ... not found
checking for OpenSSL md5 crypto library ... found

This little piece of code will actually try to compile and run a piece of code you provide to it and based on the result will set variables so that you can decide what to do. But the best is probably to look at an example:

ngx_feature='RRD library'
ngx_feature_name=
ngx_feature_run=yes
ngx_feature_incs='#include '
ngx_feature_path=
ngx_feature_libs='-lrrd_th'
ngx_feature_test='rrd_info_r("/tmp/invalid_rrd");'
. auto/feature

if [ $ngx_found = yes ]; then
    CORE_LIBS="$CORE_LIBS $ngx_feature_libs"
else
    cat << END

$0: error: the RRD module requires the RRD multi-threaded
library. You can either not enable the module or install the library.

END
    exit 1
fi

High-level, this creates a "temporary" C program with the content provided and tries to compile and run it. If everything went smoothly, the ngx_found variable is set to yes and the ngx_feature_libs is set to the appropriate options for linking with the library. All this is nice and nifty but if, let's say you want your module to be available for your buddy who's a "FreeBSD-only" dude it won't be enough. Why? Just because the way the RRD tool is packaged on FreeBSD is to put the rrd.h file in /usr/local/include. Which means that we should handle this in our script with something like:

if [ $ngx_found = no ]; then
    # FreeBSD port
    ngx_feature="RRD library in /usr/local/"
    ngx_feature_path="/usr/local/include"

    if [ $NGX_RPATH = YES ]; then
        ngx_feature_libs="-R/usr/local/lib -L/usr/local/lib -lrrd_th"
    else
        ngx_feature_libs="-L/usr/local/lib -lrrd_th"
    fi
    . auto/feature
fi

That works. But if you have a lot of friends, you have to do the same for NetBSD, MacOS, etc. A guy who probably had a lot of friends actually wrote a script to do that "automatically" for you. His name is Marcus Clyne and his tool is called the Nginx Auto Lib module (although this is not a module in the traditional sense of the term). I strongly recommend you RTFM (or rather you RTF README_AUTO_LIB): it is very good and it even documents all the ngx_feature_* variables supported by auto/config and by ngx_auto_lib_core.

I was honestly about to move to this method. The only problem is that it does not work with my Fedora 64bits. It checks too many things. In particular, it checks the path of the library that was actually used to build the test program and if it's not the one it expects it fails. So, in our case, the program compiles (and runs) because gcc looks for libraries in /usr/lib and /usr/lib64 (that's where it is found). ngx_auto_lib issues the following compilation command (reformatted for the pleasure of the eye):

gcc -D_GNU_SOURCE -D_FILE_OFFSET_BITS=64 -I /usr/include \
    -o objs/autotest objs/autotest.c -L/usr/lib -lrrd_th \
    -Wl,--rpath -Wl,/usr/lib ;
for l in rrd_th; do 
    o=\`ldd objs/autotest | grep /usr/lib/lib\$l\\.so\`;
    if [ ! \"\$o\" ]; then
        chmod -x $NGX_AUTOTEST;
        echo Linker does not link to correct version
    else
        chmod +x $NGX_AUTOTEST;
    fi
done

And if you do just the ldd, you'll realise that the RRD library is included as /usr/lib64/librrd_th.so.4 (on my system):

librrd_th.so.4 => /usr/lib64/librrd_th.so.4 (0x0000003a95200000)

Till Marcus fixes the problem, I won't make friends that are on the "exotic" platforms.

Day 42 - Setting up ngx_openresty : the missing directives

I told you, I'm trying to setup a test environment for Test::Nginx and to do so I tried to get the tests from all the modules in ngx_openresty to pass. In the previous episode we managed to get a lot through using the magic TEST_NGINX_IGNORE_MISSING_DIRECTIVES=1. Now, let's get rid of it and try to take care of the missing directives... First one to break is:

[emerg]: unknown directive "eval_subrequest_in_memory" in
.../ngx_openresty-0.8.54.2/t/servroot/conf/nginx.conf:30

So, we need to add the EvalRequestModule http://github.com/agentzh/nginx-eval-module (originally written by Valery , forked by agentzh).

cd bundle
git clone https://github.com/agentzh/nginx-eval-module.git
cd ..
./configure --add-module=../nginx-eval-module/
make

Unfortunately, even with this, testing fails with (only first failure, there are quite a few other):

#   Failed test 'TEST 1: eval - response_body - response is expected
# @@ -1,2 +1 @@
# -!!! [BEFORE
# +!!! [hi]
# -hi]
# '
#   at /[...]/lib/Test/Nginx/Socket.pm line 398.

It appears that for the eval module to work correctly, you need to add it in a certain order (i.e. before the modules you use in the eval directives). And by default when you add it amnually it's added at the end. The trick is therefore to disable the echo module (used in the eval directive) and to add it again after the eval one:

./configure --without-http_echo_module \
            --add-module=../nginx-eval-module/ \
            --add-module=../echo-nginx-module-0.36rc2
make  # From now on, I wont tell you to make just after configure

This solves the problem with the tests of the eval module. Next one to break is iconv. Actually if you look at the config file in the ngx_openresty-0.8.54.2 directory, you'll noticethe module is 'disabled'. All you have to do is to force it in:

./configure --without-http_echo_module \
            --with-http_iconv_module \
            --add-module=../nginx-eval-module/ \
            --add-module=../echo-nginx-module-0.36rc2

The module that caused me the most trouble was actually ngx_postgres. Not due to the module itself but due to the fact that it was my first PostgreSQL install and that their default authentication model is quite different from the ones (Oracle, MySQL, SQLServer) I'm used to (see below). Since, you might be as well, here is the recipe for Fedora Core 14 (should work on pretty much everything yum-based):

yum install postgresql postgresql-devel postgresql-server
service postgresql initdb
service postgresql start

Now, for the module tests to work, you need to create a ngx_test database on which the ngx_test user (with same password) has all permissions. With the standard installation of PostgreSQL the only way to log into the database server is to use the postgres user that was created by the package installation. And luckily this user can create other users and databases. So, from root, it looks something like:

su - postgres
psql # Should start the Postgres SQL client
create database ngx_test;
create user ngx_test with password 'ngx_test';
grant all privileges on database ngx_test to ngx_test;

Now, the database and users are created but there is no way you can connect to this user. A psql -U ngx_test -d ngx_test -h 127.0.0.1 will systematically return a:

psql: FATAL:  Ident authentication failed for user "ngx_test"

If you do really want, you can even look at the logs from nginx: it does not work. The problem here is that by default PostgreSQL authorizes only one authentication method: ident. Basically, we have setup a user with a password but we cannot connect with it because the database server does not accept password-based authentication. Luckily, this can be changed in the /var/lib/pgsql/data/pg_hba.conf file by changing the last field of the line for "IPv4 local connections" from ident to password. If you're interested, the explanations on the file format are here: The pg_hba.conf file. And if you're not, here is the mantra to make it work:

sed -i.bkp 's/\(127.0.0.1\/32 \+\)ident/\1password/' \
                 /var/lib/pgsql/data/pg_hba.conf
service postgresql restart

As a side note: at this point, all connections going through TCP/IP from localhost must be authenticated using password and not ident. As the postgres user has no password, you will never be able to connect to the database with something like psql -U postgres -h 127.0.0.1. Luckily, the simpler psql issued by the postgres unix user uses domain sockets and will get you through (just in case you would have to perform some admin tasks on the database).

At this point, things look better but the tests from ngx_postgres complain about:

[emerg]: unknown "remote_passwd" variable

This is actually provided by the ngx_coolkit module. So, from the bundle directory:

git clone https://github.com/FRiCKLE/ngx_coolkit.git
cd ..
./configure --without-http_echo_module \
            --with-http_iconv_module \
            --with-http_postgres_module \
            --add-module=../nginx-eval-module/ \
            --add-module=../echo-nginx-module-0.36rc2 \
            --add-module=../ngx_coolkit

At this point, the only failing tests are the tests from Maxim Dounim (which were not written using Test::Nginx). You can get rid of them with this:

rm -rf bundle/auth-request-nginx-module-0.2/t/ \
       bundle/upstream-keepalive-nginx-module-0.3/t/

And see the following command proudly return the "All test successful" we have all been looking for.

PATH="$PATH:build/nginx-0.8.54/objs/" TEST_NGINX_NO_SHUFFLE=1 \
      TEST_NGINX_LUA_PACKAGE_CPATH="$(pwd)/bundle/lua-yajl/?.so" \
      prove -r bundle/*/t

Day 41 - Setting up ngx_openresty (WAS: testing Test::Nginx)

A bit of background first. As I told you in yesterday's post, I started working on changing Test::Nginx (as a result of my frustration trying to move the tests I wrote to test my RRD module from my original "let's use python" approach to a more community-friendly "let's work with agentzh's Test::Nginx approach). But I was scared to break everything (especially with one of the features I want to introduce that changes the way requests and expected responses work). So, I did the smart thing (for once): tried to figure out a way to test what I was about to do. And more specifically to perform regression testing. But to do so, I needed more tests written with Test::Nginx than the ones I wrote myself for my RRD module.

I had read at some point in the mailing list that agentzh packages a bunch of "3rd party nginx modules" into ngx_openresty : Turning Nginx into a Full-fledged Web App Server. So, I figured agentzh would have all his modules there with all the tests. That sounded like a good place to look for tests for Test::Nginx. Actually, he confirmed it would be «the easiest approach though some of the dependencies may be missing"». He also said (in a follow-up email): «But we ourselves usually use the util/build.sh script to build separate nginx instance for the current module».

As dependencies scare neither me nor my good old pal yum I went for the first approach that would give me a simple way to perform a lot of tests (the ones from all the modules) in one go.

It all started with:

wget 'http://agentzh.org/misc/nginx/ngx_openresty-0.8.54.2.tar.gz' 
tar xvzf ngx_openresty-0.8.54.2.tar.gz
cd ngx_openresty-0.8.54.2/
./configure
make

Of course, I had to install a few things:

yum install readline-devel libdrizzlesee comments for why this is deleted

At this point everything pretty much looked fine (which honestly surprised me). So, I went for the ultimate in testing:

LD_LIBRARY_PATH="build/libdrizzle-root/usr/local/openresty/libdrizzle/lib:build/lua-root/usr/local/openresty/lua/lib" \
    PATH="build/nginx-0.8.54/objs/:$PATH" TEST_NGINX_NO_SHUFFLE=1 \
    prove -I ~/nginx/test-nginx/lib/ -r bundle/*/t

Let me explain a little bit what is going on here:

• PATH="build/nginx-0.8.54/objs/:$PATH" makes sure you are using the nginx we just built (i.e. the openresty one).
• TEST_NGINX_NO_SHUFFLE=1 is my personal favorite (I hate when I can't predict the order in which tests are run: it makes my head spin.
• -I ~/nginx/test-nginx/lib/ is to use the Test::Nginx I'm working on (not a CPN installed one). You might not need this if using the CPAN version installed normaly.
• -r bundle/*/t because it's where all the tests I'm interested in are (and some of them have subdirectories, so I want to recurse through that).
• LD_LIBRARY_PATH="build/..." tells where to look for the drizzle and lua libraries (which are part of openresty). You might want to look at the comments to understand why I need this. This line being way too long, I won't mention it anymore.

At this point, the thing blew up in my face and I had to "fix" the obvious with: mkdir -p t/servroot and set TEST_NGINX_IGNORE_MISSING_DIRECTIVES to 1 to avoid the problems with the uninstalled modules. So, I started using something like:

PATH="build/nginx-0.8.54/objs/:$PATH" TEST_NGINX_NO_SHUFFLE=1 \
    TEST_NGINX_IGNORE_MISSING_DIRECTIVES=1 \
    prove -I ~/nginx/test-nginx/lib/ -r bundle/*/t

At this point, I started having tests that worked (not all of them, but most of them). I still needed a few more things:

yum install memcached redis
service mysqld start
service memcached start
service redis start

Also make sure you log into your MySQL DB and create the ngx_test user and database (for the tests of the drizzle module to work):

create user 'ngx_test'@localhost identified by 'ngx_test';
create database ngx_test;
grant all on ngx_test.* to 'ngx_test'@localhost;

That's when things got hairy: tests for ngx_lua refused to work. Lua is packaged with openresty but not everything in lua. Namely, you don't have yajl-lua (Yet Another JSON Library for LUA). So, as yajl-lua is not packaged as a rpm on my platform, I went (wait before you do the same):

yum install yajl yajl-devel cmake
cd bundle
git clone https://github.com/brimworks/lua-yajl.git
cd lua-yajl
LUA_DIR=../lua-5.1.4/src cmake .

Unfortunately this will not get you anywhere as it's version 1.0.7 of yajl that is packaged for Fedora Core 14 and you need at least 1.0.9... :( So, I also had to get yajl compiled:

cd .. #now you should be in bundle
git clone git://github.com/lloyd/yajl
cd yajl
cmake .
make
cd ../lua-yajl
LUA_DIR=../lua-5.1*/src CMAKE_PREFIX_PATH=../yajl/yajl-1.0.* cmake .
make

Still, test bundle/ngx_lua-0.1.6rc2/t/005-exit.t (and quite a few others) was failing because LUA could not find the yajl library. The only way I managed to fix this was by adding a new TEST_NGINX_LUA_PACKAGE_CPATH variable and change the test file itself:

sed -i.bkp s/lua_package_cpath.\*\$/lua_package_cpath\ \'\$TEST_NGINX_LUA_PACKAGE_CPATH\'\;/g bundle/ngx_lua-0.1.6rc2/t/005-exit.t
sed -i.bkp s/lua_package_cpath.\*\$/lua_package_cpath\ \'\$TEST_NGINX_LUA_PACKAGE_CPATH\'\;/g bundle/ngx_lua-0.1.6rc2/t/*/exit.t

Yes, the line gets out of the blog, so get a bigger screen. ;) And yes, this is a lot of backslahes (and the line gets out of the blog), but I never manage to remember what double-quote escaping does in bash. So, for me it's either single-quote escaping (and no expansion happens but you cannot have single-quotes) or single character escaping (with backslash) and as soon as a character looks weird, I escape it... YMMV, of course.

From this point on, I was running my tests with:

PATH="$PATH:build/nginx-0.8.54/objs/" TEST_NGINX_NO_SHUFFLE=1 \
     TEST_NGINX_LUA_PACKAGE_CPATH="$(pwd)/bundle/lua-yajl/?.so" \
    TEST_NGINX_IGNORE_MISSING_DIRECTIVES=1 \
     prove -I ~/nginx/test-nginx/lib/ -r bundle/*/t/

And to be completely honest, at this point, I suspect I'm running the LUA that comes installed with Fedora Core 14 (not the one from openresty). I'll try to check that at some point.

The good news is that at this point, there are not many tests that fail. The most obvious ones are the tests from auth-request and upstream-keepalive. So, I looked at the perl code and it doesn't look at all like tests for Test::Nginx, although the tests have a use Test::Nginx;. They are not using the data-driven paradigm and invoking Test::Nginx->new() (and there has never been such a thing as new on Test::Nginx). As both modules were developed by Maxim Dounim, I guess he is using a "different" Test::Nginx. Could be fun to rewrite those tests with agentzh's Test::Nginx just to see if its expressive power is enough for what Maxim wants to test...

So, that will be it for today (that was one hell of a long post) as we are at a point where the only tests that fail use a different Test::Nginx or are for unknown directives. Next step, I'll try to get rid of the TEST_NGINX_IGNORE_MISSING_DIRECTIVES=1 in my prove.

Day 40 - Test::Nginx new features

I told you about my frustration with Test::Nginx in previous posts (Day 33 and Day 32) and managed to discuss them with agentzh (the module maintainer/author).

Here are the evolutions I offered to contribute:

1. Be able to have multiple requests in each test. As a replacement for this, I had to use pipelined_requests and matching on the response.
2. Better control over the stop/start sequence. The idea here is to restart the nginx server only when the config changes between two tests. With a TEST_NGINX_FORCE_RESTART_ON_TEST variable to preserve the current behavior.
3. Improve the documentation of the possible test sections.

And I started working on the second one because first one is likely to have a HUGE impact on the code base and I'm scared to break everything. From what I can tell, all the modules developed by agentzh and his friends at Taobao/Alibaba extensively rely on Test::Nginx for testing. This includes the famous Echo module (very useful for debugging configurations) and heavy-lifting modules like the LUA module (embeds a LUA scripting engine in nginx) or the Drizzle module (non-blocking access to you MySQL DB, a must have if you want to scale like nobody else). And I wouldn't want to break their work.

So, I started easy with just adding the following:

1. Support for environment variable TEST_NGINX_NO_NGINX_MANAGER (defaults to 0) which disables the nginx management code (stop/config/start). Very useful when you want to run tests on an already running NGINX (set TEST_NGINX_NO_NGINX_MANAGER to 1 and TEST_NGINX_CLIENT_PORT to the port your running nginx is listening on). Of course, this could be abused in every conceivable way (for example to test another web server ;)). As far as I'm concerned, the main purpose of this was to be able to run my tests on a nginx that I had started with debug on plus a few breakpoints).
2. Support for environment variable TEST_NGINX_FORCE_RESTART_ON_TEST (defaults to 1). If you don't provide any config section in your test (or if it does not change between two successive tests), the nginx server will not be restarted (if TEST_NGINX_FORCE_RESTART_ON_TEST is explicitely set to 0, of course). This way, using TEST_NGINX_FORCE_RESTART_ON_TEST=1 with TEST_NGINX_NO_SHUFFLE=1 and having a config section only on the first test, you can have all tests use the same configuration. I find this very useful to avoid one of the annoying features of Test::Nginx, namely that it removes all logs between two runs. Hopefully, at some point in the future, I'll convince agentzh to have shuffling and force restart turned off by default.

While I was working on this, I realised the code doing the stop/config/start could use some refactoring/rewriting. So, I think this is one more thing to add to my TODO list regarding Test::Nginx.

For those of you who are interested, my commits are on the devel branch of Test::Nginx. Feedback and testing quite welcome (as usual with FOSS)...

Day 39 - GET parameters from your module

First, google analytics says that I got a visitor from Islamabad. So, I want to say hello to all our fellow pakistani nginx lovers. OK, there is only one for now, but I wasn't counting on seeing any visitor from there. It looks like nginx is getting truly international...

My RRD module is working fine now and I'm getting plenty of nice graphs. Unfortunately, they are all pretty boring. Looking something like that:

Most of the time I'm only interested with what happened in the last hour or so and that might not be very easy to read if the values from the last hour are very different from the ones in the last 24 hours. Like in the picture above: the values at around 1:30-2:00 make the latest values completely impossible to read. So, I decided to use a start parameter to be able to easily change the starting point on the time axis of my graph. That's already a parameter supported by the rrdtool graph command and all I needed was to parse it from the query string.

First thing to know is that nginx Core module already parses the request and makes the values available as $arg_PARAMETER (see arg_PARAMETER section of the HttpCoreModule documentation). So, really there is no point in doing the job twice. But there are unfortunately a couple of places to look at before you get the right way of doing this (or at least what I consider so far the best way to do it: I might change my mind at any point in the future):

• request->variables sounds like a perfectly reasonable place to look at. Well, well, well. You see, the variables in here are not the kind of variables you are looking for. They are less "variable" variables ;) than the one we are looking for. Basically the variables in this list are all the variables that exist regardless of what the request is. Things like $scheme (also called the protocol: http vs. https) or $is_args (are there arguments in the query string). It does not include the dynamic variables like $arg_PARAMETER or $cookie_COOKIE because before parsing the request, it doesn't even know what the variables are going to be.
• ngx_http_variable_argument function. Except that it's static and you cannot access it from your module. So, not a good candidate although it does exactly what we want.
• ngx_http_arg is a good choice. Except that this is really low-level and just does the parsing. You have to do all the allocations required around it and I got sick of doing the memory allocations by myself (and even more of checking the results, but that's a different story). So, I did not go for this.
• I went for ngx_http_get_variable which does probably too much for what I want to do (the variable can be one of the "dynamic" variables but it can also be one of the more static ones, it will use the right way to extract it). But it offers a nice simple interface easy to remember:
  

  ngx_http_variable_value_t *
  ngx_http_get_variable(ngx_http_request_t *r, ngx_str_t *name,
                        ngx_uint_t key)
  

So, I was left pondering what the key parameter could be. After a little bit of research I found out it is a hash of the name parameter. The two are made distinct probably for optimisation reasons. This way, the code inside ngx_http_get_variable doesn't have to recompute the hash on every call. As my name is really a constant, I figured out that I would play nice with this optimization and keep both the name and the key as constants:

static ngx_str_t ARG_START = ngx_string("arg_start");
static ngx_uint_t ARG_START_KEY = ngx_hash_key(ARG_START.data,
                                               ARG_START.len);

And that is exactly when the compiler started barking at me with initializer element is not constant. Not a nice guy this compiler, I tell you. Everything is constant from a logical standpoint but it doesn't like the call to a function in the initialization (it cannot optimize it to compile-time). Please note that it does not complain at ngx_string("arg_start") because this is actually a macro expanded by the pre-processor. So, I had to go for something like this:

static ngx_str_t ARG_START = ngx_string("arg_start");
static ngx_uint_t ARG_START_KEY;

static ngx_int_t ngx_http_rrd_init_process(ngx_cycle_t *cycle) {
    ARG_START_KEY = ngx_hash_key(ARG_START.data, ARG_START.len);
[...]
    return NGX_OK;
}

And here is the result with an extra ?start=now-5h (after fixing a few stupid bugs, of course... ;)):

Looks good, no?

Day 38 - reading code is good for you

I am no Richard Stallman (I shave, he does not ;)), but there is one good thing about open-source: you can read other people code. You can read it before you write your own and copy/paste. Or you can read it after, fishing for good ideas or better ways to do things. It's kind of like tutorials: you can read them before learning something or after. The benefit you get is not the same: if you read first, you'll get to the point where you want to be faster. If you read after, you might feel like you lost time trying to do things a certain way which wasn't the best one. The point is: there is always benefit to reading source code from others.

So, today I decided to have a look at two things:

• Evan Miller mod_rrd_graph
• Valery Kholodkov nginx-upload-module

mod_rrd_graph. Evan's approach to the integration of rrd and nignx is completely different from mine. In a few words:

• My module assumes there is a RRD setup on a server (that might change in the future but that's another story) and gives you write and read access to it. With a very basic read access where you cannot specify anything in terms of data selected, colors, etc. My module tries to figure out something that makes sense.
• Evan's module also assumes there is a RRD setup on a server and provides a completely customizable read access. Basically, you have as much power at your fingertips as you would with the command line: rrdtool graph.

The approaches are very different and when I realized it I was "well, it's so different, there is not much I'll get from reading his code". And I was wrong. You know why? Because I did not RTFM well enough. Evan is using the rrd_graphv function where I was using rrd_graph. The main difference is that with rrd_graphv rrd won't necessarily write to a temporary file. And that was quite a revelation to me. So, I spent most of my day getting rid of code: the code to implement the rrd_image_temp_path directive, code to create the temporary file, code to retrieve the temporary file information and make it a ngx_buf_t. Quite a lot of code flushed down the toilet. So, I was pretty happy to get rid of all this stuff but of course, I was not happy with myself for nto reading more carefully the RRD manual.

nginx-upload-module. All that did not leave me much time to read Valery's work. But enough to realize that this module is very peculiar in the sense that it completely bypasses (or rewrites) the standard nginx code to read a POST entity. My module (like most modules using POST, I guess) waits for nginx to tell it that it is done with reading the body from the client and that the data is available in the request->request_body buffers. Valery's module, after receiving the headers does not handover back to the http core module of nginx, instead it hands over to the event module. This is really hardcore, but the direct benefit (which is the whole point of this module) is that this way it can be notified as data packets arrive and take appropriate action (store it in the appropriate file and manage partial uploads and resumes). And I must say I find it pretty awesome that you can do something like that with nginx: completely rewrite a part of the thing. It's a proof that the design is very modular. Now, I suspect this might break some other modules but at some point you have to make a choice (and that's fairly easy since the configuration mechanisms pretty much let you turn on and off modules at the location level).

Day 37 - the end of the last buffer or should it be the other way round ?

One of the mysteries of nginx I still have not figured out (and I think there are a lot of them) are buffers. Those little things called ngx_buf_t. I got the basics: it's a way to point at an existing zone of memory (or file) that already exist. The main objective (as far I understand it) is to avoid copying stuff (especially big chunks of memory) around. If you look at the Development of modules for nginx I translated you will realise the important fields in the structure are:

    u_char          *pos;
    off_t            file_pos;
    u_char          *last;
    off_t            file_last;

    u_char          *start;         /* start of buffer */
    u_char          *end;           /* end of buffer */

Most of the time, all you'll do is:

1. Allocate memory (let's call the pointer to this zone p)
2. Create a buffer pointing to this zone:pos=start=p and last=end=p+size_needed
   

This is so common, there is even ngx_create_temp_buf doing that for you. So, most of the time you don't even realize that last and end are not the same and the naming probably doesn't help: there is usually nothing left after the end and nobody after the last one... ;) The thing is : with nginx, after the last comes the end.

Now, there is only one situation I saw end != last and that was with very specific buffers: the ones I crafted on Day 28 - POST body buffers... to show you that the body of a request could be split in two buffers. In this very specific situation, the first buffer is actually the one pre-allocated by nginx to read the request method, uri, headers, etc. So, the buffer end is set even before the request starts arriving and its last is determined by how much was read from the network. Hence, you end up with a buffer in which there is room between the last and the end.

Before I let you go with this revelation, I want to tell you about one more thing that surprised me recently regarding buffers. So, I was trying to migrate from my python testing to agentzh's Test::Nginx. As there is no support for multiple requests (in the traditional sense) per test in the current version of Test::Nginx (we're working on it and this might actually be my first contribution to this project, but that's another story) I used pipelined_requests to simulate this. pipelined_requests were intended to test HTTP/1.1 so they send all the requests in pretty much one go. And that caused my rrd module to crash. Why, would you ask ? Pretty simple: I was assuming the body of the request to end with the last of request_body->bufs->buf. And I was wrong! Here the buffer goes all the way up to the last byte read from the network, which in my test happened to be the end of the second pipelined request. So, my module was basically considering the body of request one to be body of request 1+request 2+headers of request 2+body of request 2. Needless to say, inserting this data in my RRD did not work. Of course, I fixed this by refusing to look beyond r->headers_in.content_length_n (the length of the body as announced in the request header for those of you who are not familiar with the nginx requests attributes yet).

One more thing (not related to buffers at all): today I tried to setup my nginx to run some php. I found tons of posts/articles/forums entries telling that PHP had to be recompiled for fastcgi to be enabled on a Fedora 14 (and others as well). I even found people pointing as repositories just for that. And, you know what? The standard install (yum install php) comes with fastcgi installed. Yes, it takes to look at the configuration used by the packagers with php -i. Yes, it takes to Read The Fucking Manual. But if you do so, all you have to do to run you php as a fastcgi server is: php-cgi -b 127.0.0.1:9000. And no need to say that this work perfectly with nginx FastCGI module.