Day 43 - module config and auto/feature

Looking at other people modules gave me an opportunity to look closer at the config file they are providing. Most of them (especially the modules written by Igor himself) use the file auto/feature to figure out whether a library is available on your system and to figure out the directories where it's deployed. This way, it will work under the various flavours of Linux (Fedora/Redhat/CentOS or Debian/Ubuntu) or under the BSDs of the world.

Good thing is that I have my rrd-nginx-module handy. And I did not do a very good job on the config file. So, now is a good time to review/refactor this and do somethign nicer. Let's first have a look at what I have in there for now:

ngx_addon_name=rrd-nginx-module
HTTP_MODULES="$HTTP_MODULES ngx_http_rrd_module"
NGX_ADDON_SRCS="$NGX_ADDON_SRCS $ngx_addon_dir/ngx_http_rrd_module.c"
CORE_LIBS="$CORE_LIBS -lrrd_th"
CFLAGS="${CFLAGS/-Werror/-Wno-deprecated-declarations -Werror}"

Yes, this is ugly. But it works (at least on my Fedora Core 14) and there is even worse than that: it's nto int he right place: it's in the src directory whereas the "standard" is to put it in the root of the module. I told you: never paid much attention to this part of the process... But before we start improving this, let me justify/explain what I'm doing:

• ngx_addon_name appears during the configuration process to let you know that your configuration script was invoked:

  configuring additional modules
  adding module in ../rrd-nginx-module/
   + rrd-nginx-module was configured
  

• HTTP_MODULES is where you should add the name of the ngx_module_t variable defining your module that is exported by your code.
• NGX_ADDON_SRCS is fairly easy: where your source code is.
• CORE_LIBS adds the extra library to link with. I use rrd_th which is the thread-safe version of the RRDtool library.
• CFLAGS are altered because I use the rrd_init, rrd_open, rrd_close and rrd_free which are deprecated. They are deprecated but the replacement function (rrd_info_r) does a lot more than what I need. I might change my mind about this (and use the non-deprecated function) but for the beauty of it, let's assume there is no other way and the CFLAGS MUST be altered.

Now, my first discovery was to realise that the nginx distribution comes with a file named auto/feature which is used by Igor's modules to display the lines like the following:

checking for system md library ... not found
checking for system md5 library ... not found
checking for OpenSSL md5 crypto library ... found

This little piece of code will actually try to compile and run a piece of code you provide to it and based on the result will set variables so that you can decide what to do. But the best is probably to look at an example:

ngx_feature='RRD library'
ngx_feature_name=
ngx_feature_run=yes
ngx_feature_incs='#include '
ngx_feature_path=
ngx_feature_libs='-lrrd_th'
ngx_feature_test='rrd_info_r("/tmp/invalid_rrd");'
. auto/feature

if [ $ngx_found = yes ]; then
    CORE_LIBS="$CORE_LIBS $ngx_feature_libs"
else
    cat << END

$0: error: the RRD module requires the RRD multi-threaded
library. You can either not enable the module or install the library.

END
    exit 1
fi

High-level, this creates a "temporary" C program with the content provided and tries to compile and run it. If everything went smoothly, the ngx_found variable is set to yes and the ngx_feature_libs is set to the appropriate options for linking with the library. All this is nice and nifty but if, let's say you want your module to be available for your buddy who's a "FreeBSD-only" dude it won't be enough. Why? Just because the way the RRD tool is packaged on FreeBSD is to put the rrd.h file in /usr/local/include. Which means that we should handle this in our script with something like:

if [ $ngx_found = no ]; then
    # FreeBSD port
    ngx_feature="RRD library in /usr/local/"
    ngx_feature_path="/usr/local/include"

    if [ $NGX_RPATH = YES ]; then
        ngx_feature_libs="-R/usr/local/lib -L/usr/local/lib -lrrd_th"
    else
        ngx_feature_libs="-L/usr/local/lib -lrrd_th"
    fi
    . auto/feature
fi

That works. But if you have a lot of friends, you have to do the same for NetBSD, MacOS, etc. A guy who probably had a lot of friends actually wrote a script to do that "automatically" for you. His name is Marcus Clyne and his tool is called the Nginx Auto Lib module (although this is not a module in the traditional sense of the term). I strongly recommend you RTFM (or rather you RTF README_AUTO_LIB): it is very good and it even documents all the ngx_feature_* variables supported by auto/config and by ngx_auto_lib_core.

I was honestly about to move to this method. The only problem is that it does not work with my Fedora 64bits. It checks too many things. In particular, it checks the path of the library that was actually used to build the test program and if it's not the one it expects it fails. So, in our case, the program compiles (and runs) because gcc looks for libraries in /usr/lib and /usr/lib64 (that's where it is found). ngx_auto_lib issues the following compilation command (reformatted for the pleasure of the eye):

gcc -D_GNU_SOURCE -D_FILE_OFFSET_BITS=64 -I /usr/include \
    -o objs/autotest objs/autotest.c -L/usr/lib -lrrd_th \
    -Wl,--rpath -Wl,/usr/lib ;
for l in rrd_th; do 
    o=\`ldd objs/autotest | grep /usr/lib/lib\$l\\.so\`;
    if [ ! \"\$o\" ]; then
        chmod -x $NGX_AUTOTEST;
        echo Linker does not link to correct version
    else
        chmod +x $NGX_AUTOTEST;
    fi
done

And if you do just the ldd, you'll realise that the RRD library is included as /usr/lib64/librrd_th.so.4 (on my system):

librrd_th.so.4 => /usr/lib64/librrd_th.so.4 (0x0000003a95200000)

Till Marcus fixes the problem, I won't make friends that are on the "exotic" platforms.