Tuesday, March 29, 2011

Day 42 - Setting up ngx_openresty : the missing directives

I told you, I'm trying to setup a test environment for Test::Nginx and to do so I tried to get the tests from all the modules in ngx_openresty to pass. In the previous episode we managed to get a lot through using the magic TEST_NGINX_IGNORE_MISSING_DIRECTIVES=1. Now, let's get rid of it and try to take care of the missing directives... First one to break is:

[emerg]: unknown directive "eval_subrequest_in_memory" in

So, we need to add the EvalRequestModule http://github.com/agentzh/nginx-eval-module (originally written by Valery , forked by agentzh).

cd bundle
git clone https://github.com/agentzh/nginx-eval-module.git
cd ..
./configure --add-module=../nginx-eval-module/

Unfortunately, even with this, testing fails with (only first failure, there are quite a few other):

#   Failed test 'TEST 1: eval - response_body - response is expected
# @@ -1,2 +1 @@
# -!!! [BEFORE
# +!!! [hi]
# -hi]
# '
#   at /[...]/lib/Test/Nginx/Socket.pm line 398.

It appears that for the eval module to work correctly, you need to add it in a certain order (i.e. before the modules you use in the eval directives). And by default when you add it amnually it's added at the end. The trick is therefore to disable the echo module (used in the eval directive) and to add it again after the eval one:

./configure --without-http_echo_module \
            --add-module=../nginx-eval-module/ \
make  # From now on, I wont tell you to make just after configure

This solves the problem with the tests of the eval module. Next one to break is iconv. Actually if you look at the config file in the ngx_openresty- directory, you'll noticethe module is 'disabled'. All you have to do is to force it in:

./configure --without-http_echo_module \
            --with-http_iconv_module \
            --add-module=../nginx-eval-module/ \

The module that caused me the most trouble was actually ngx_postgres. Not due to the module itself but due to the fact that it was my first PostgreSQL install and that their default authentication model is quite different from the ones (Oracle, MySQL, SQLServer) I'm used to (see below). Since, you might be as well, here is the recipe for Fedora Core 14 (should work on pretty much everything yum-based):

yum install postgresql postgresql-devel postgresql-server
service postgresql initdb
service postgresql start

Now, for the module tests to work, you need to create a ngx_test database on which the ngx_test user (with same password) has all permissions. With the standard installation of PostgreSQL the only way to log into the database server is to use the postgres user that was created by the package installation. And luckily this user can create other users and databases. So, from root, it looks something like:

su - postgres
psql # Should start the Postgres SQL client
create database ngx_test;
create user ngx_test with password 'ngx_test';
grant all privileges on database ngx_test to ngx_test;

Now, the database and users are created but there is no way you can connect to this user. A psql -U ngx_test -d ngx_test -h will systematically return a:

psql: FATAL:  Ident authentication failed for user "ngx_test"

If you do really want, you can even look at the logs from nginx: it does not work. The problem here is that by default PostgreSQL authorizes only one authentication method: ident. Basically, we have setup a user with a password but we cannot connect with it because the database server does not accept password-based authentication. Luckily, this can be changed in the /var/lib/pgsql/data/pg_hba.conf file by changing the last field of the line for "IPv4 local connections" from ident to password. If you're interested, the explanations on the file format are here: The pg_hba.conf file. And if you're not, here is the mantra to make it work:

sed -i.bkp 's/\(\/32 \+\)ident/\1password/' \
service postgresql restart

As a side note: at this point, all connections going through TCP/IP from localhost must be authenticated using password and not ident. As the postgres user has no password, you will never be able to connect to the database with something like psql -U postgres -h Luckily, the simpler psql issued by the postgres unix user uses domain sockets and will get you through (just in case you would have to perform some admin tasks on the database).

At this point, things look better but the tests from ngx_postgres complain about:

[emerg]: unknown "remote_passwd" variable

This is actually provided by the ngx_coolkit module. So, from the bundle directory:

git clone https://github.com/FRiCKLE/ngx_coolkit.git
cd ..
./configure --without-http_echo_module \
            --with-http_iconv_module \
            --with-http_postgres_module \
            --add-module=../nginx-eval-module/ \
            --add-module=../echo-nginx-module-0.36rc2 \

At this point, the only failing tests are the tests from Maxim Dounim (which were not written using Test::Nginx). You can get rid of them with this:

rm -rf bundle/auth-request-nginx-module-0.2/t/ \

And see the following command proudly return the "All test successful" we have all been looking for.

PATH="$PATH:build/nginx-0.8.54/objs/" TEST_NGINX_NO_SHUFFLE=1 \
      TEST_NGINX_LUA_PACKAGE_CPATH="$(pwd)/bundle/lua-yajl/?.so" \
      prove -r bundle/*/t


  1. Regarding PostgreSQL:
    - you should really use "md5" instead of "password",
    - all your problems would be solved if you would pass "-A md5" to initdb command :)

  2. Thanks for the tip. I'll do that next time I re-install the whole thing.