Monday, February 28, 2011

Day 29 - nginx development tool-chain, reviewed

Originally, today I was planning to tell you about ngx_str_t and char * but life being what it is, you will have to wait. Because today, I got a very interesting mail from agentzh on the mailing list. If you're interested in the details, have a look at agentzh's message.

As we were talking about the difficulty there is to getting a nginx module right (in addition to the usual C traps you have the nginx traps) he pointed at a number of interesting tools. Therefore I decided to review/expand my nginx development tool-chain...

You remember how, on Day 24 - Revisiting HTTP verbs and testing, I decided to use Python for my testing. Don't get me wrong, I'm happy with Python. But I was starting to feel the need for something a bit more evolved (and more self-contained) in terms of testing. So, I was starting to contemplate the idea of putting together some way to setup configurations in my test cases, restart nginx and make testing less verbose than it currently is. Luckily, agentzh pointed at his Test::Nginx on CPAN. It already does the things I was contemplating to do. And all you have to do is to learn a little bit of the library syntax. I am starting to really feel like it's worth it. May be not today (because I'm fine with what I have now), but I really feel like it would be a good investment. But I will have to rewrite my test cases for RRD with this new approach. Should be fun...

Now, what I consider to be THE MOST valuable tip agentzh gave me is actually just a patch provided by one of this students that removes the "memory pool" mechanism of nginx: nginx no-pool. It is very valuable because it allows you to run valgrind on nginx (and more specifically on your module). Memory management is always a beast to tame in C and a tool like valgrind is a must for this kind of quest. Unfortunately nginx "memory pool" mechanisms prevent the use of such tools. This patch makes valgrind work on nginx. So, I installed valgrind and the patch right away and I'm trying to figure out how to configure my Eclipse to run with it. And then it will be a full-time member of my nginx development platform.

Agentzh also pointed at EtcProxy. From what I understand, this is a TCP proxy that lets you fragment your packets and delay them as you wish. It's in Erlang (a language I had never heard of before today) and very small but for now, I consider this as an ad-hoc tool to create very specific situations when you cannot change the "thing" nginx is talking to (I think this was MySQL when agentzh used it). I think I'd rather have a "stub server" that replies only a certain type of response with the right fragmentation rather than add an extra component. Anyway, as I don't feel any need for this right now, I'll wait and see.

Another thing I must tell you: I was looking at chaoslawful's github. He is watching Evan Miller's github. So I went and had a look at Evan's page to see what he is working on. I found about ChicagoBoss and a lot of other Erlang things (that's probably where the Erlang presence in nginx community comes from). And I found this: mod_rrd_graph. Yeah, I'm not the only one to have stupid ideas... ;) I promise, I won't look at it before I consider I'm done with my RRD module. And then, I'll see how different our modules are. That will be fun too...

Oh, and another side note: agentzh works for taobao.com. Look at their stats on google trends. I would say nginx does scale... ;)

No comments:

Post a Comment