The uWSGI Spooler

The Spooler is a queue manager that works very similar to a printing/mail system.

You choose a directory as a spool then a process looply look for files in this spool and use each one as the arguments for a python callable. After the callable exits, the file is removed.

For example you can enqueue the massive sending of emails, an image processing, a video encoding.. ecc. ecc. and let to spooler doing the hard work in background.

Using the message passing feature of the uWSGI server you can even send your spool request to a remote uWSGI server.

To use the spooler you need to pass the -Q argument followed by the directory you want to use as spool:

./uwsgi26 -Q myspool -s /tmp/uwsgi.sock -b 8192 -w testapp -A 10

If you think to pass big messages, set a bigger buffer size using the -b option.

Add a new function to your wsgi script:

def myspooler(env):
        print env
        for i in range(1,100):
                time.sleep(1)

uwsgi.spooler = myspooler

Using the uwsgi.spooler attribute you will set the callable to execute for every spool/queue/message file

Now if you want to enqueue a request (that is a simple python dictionary) use this function in your app:

uwsgi.send_to_spooler({'Name':'Serena', 'System':'Linux', 'Tizio':'Caio'})

The only argument is the dictionary to pass to the callable.

How the uWSGI server recognize spooler request ?

The queue/message/spool files are normally dictionaries encoded in the uwsgi protocol format. In this way we can use the uWSGI server to manage remote messaging.

Setting the uwsgi_modifier1 to UWSGI_MODIFIER_SPOOL_REQUEST (value: 17) you will inform the uWSGI server that the request is a spooling request.

This is transparent for your application using the uwsgi.send_to_spooler function, but you can use your webserver support for uwsgi_modifiers for doing funny things like passing spooler message without using your wsgi apps resource but only the spooler.

NGINX example:

 location /sendmassmail {
    uwsgi_pass 192.168.1.1:3031;
    uwsgi_modifier1 17;
    uwsgi_param ToGroup  customers;
 }

supposing you have a callable that sends email to a group specified in the ToGroup dict key, you can enqueue mass mails only using Nginx !.