uWSGI api python decorators (from 0.9.8.2)

The uWSGI api is very low-level (as it must be language-independent).

Being too low-level is not good for some languages style (read: python).

Decorators are one of the most kick-ass feature of python, so in the uWSGI source tree you will find a module exporting a bunch of decorators covering a good part of the uWSGI api.

@postfork

uWSGI is a fork-abuser server, so you could probably need to execute some fixup task after each fork() call

from uwsgidecorators import *

@postfork
def reconnect_to_db():
    myfoodb.connect()

@postfork
def hello_world():
    print("Hello World")

As you have probably noted, the @postfork decorator allows you to chain various hook: each decorated function will be executed in chain after each fork()

@spool

The Spooler is one of the most amazing uWSGI feature, but compared to solutions like Celery, it is very raw.

The @spool decorator will solve this problem

@spool
def a_long_long_task(arguments):
    print(arguments)
    for i in xrange(0, 10000000):
        time.sleep(0.1)

@spool
def a_longer_task(args):
    print(args)
    for i in xrange(0, 10000000):
        time.sleep(0.5)

# enqueue the tasks
a_long_long_task.spool(foo='bar',hello='world')
a_longer_task.spool({'pippo':'pluto'})

The functions will automatically return uwsgi.SPOOL_OK so they will be executed one time independently by their return status

@spoolforever

Sometimes you could want to continuosly execute a spool task

@spoolforever
def a_longer_task(args):
    print(args)
    for i in xrange(0, 10000000):
        time.sleep(0.5)

# enqueue the tasks
a_longer_task.spool({'pippo':'pluto'})

@spoolforever will always return uwsgi.SPOOL_RETRY

@spoolraw

Advanced users may want to control the return value of a task

@spoolraw
def a_controlled_task(args):
    if args['foo'] == 'bar':
        return uwsgi.SPOOL_OK
    return uwsgi.SPOOL_RETRY

a_controlled_task.spool(foo='bar')

@rpc

RPC is the fastest way to remotely call functions hosted in uWSGI instances. You can easily define exported functions with the @rpc decorator

@rpc('helloworld')
def ciao_mondo_function():
    return "Hello World"

@signal

You can register signal handlers in one shot:

@signal(17)
def my_signal(num):
    print("i am signal %d" % num)

@timer

Execute a function every 3 seconds:

@timer(3)
def three_seconds(num):
    print("3 seconds elapsed")

@rbtimer

Works like @timer but using red black timers

@cron

Execute functions with cron-like facility

@cron(59, 3, -1, -1, -1)
def execute_me_at_three_and_fiftynine(num):
    print("it's 3:59 in the morning")

@filemon

Execute function every time a file/directory is modified

@filemon("/tmp")
def tmp_has_been_modified(num):
    print("/tmp directory has been modified")

@erlang

Map a function as an erlang process

@erlang('foobar')
def hello():
    return "Hello"

@thread

Execute a function in a new thread. (threading must be enabled in uWSGI with --enable-threads or --threads <n>)

@thread
def a_running_thread():
    while True:
        time.sleep(2)
        print("i am a no-args thread")

@thread
def a_running_thread_with_args(who):
    while True:
        time.sleep(2)
        print("Hello %s (from arged-thread)" % who)

a_running_thread()
a_running_thread_with_args("uWSGI")

A good trick is combining @thread with @postfork

@postfork
@thread
def a_post_fork_thread():
    while True:
        time.sleep(3)
        print("Hello from a thread in worker %d" % uwsgi.worker_id())

The function a_post_fork_thread will be called after each fork() in a new thread in the new worker

@lock

This decorator will execute a function in fully locked environment, avoiding other workers/threads (or the master if you are brave/fool) to launch it at the same time

@lock
def dangerous_op():
    print("no concurrency in me !!!")

Obviously you can combine it with @postfork

@mulefunc

You can offload the execution of a function to a Mule

@mulefunc
def i_am_an_offloaded_function(argument1, argument2):
    print argument1,argument2

Now when you call i_am_an_offloaded_function(arg1, arg2) in your app, the function will just return and it will be executed asynchronously in one of the mules

You can configure the function to run on a specific mule:

@mulefunc(3)
def ...

will run the function only in the mule 3

@mulefunc('topogigio')
def ...

will run the function in the specified mule farm. Please remember to register your function with a uwsgi import configuration option.

Example: a Django session cleaner and video encoder

Let's define a task.py module (put it in your django project dir)

from uwsgidecorators import *
from django.contrib.sessions.models import Session
import os

@cron(40, 2, -1, -1, -1)
def clear_django_session(num):
    print("it's 2:40 in the morning: clearing django sessions")
    Session.objects.all().delete()

@spool
def encode_video(arguments):
    os.system("ffmpeg -i \"%s\" image%%d.jpg" % arguments['filename'])

The session cleaner will be executed every day at 2:40, to enqueue a video encoding we simply need to spool it from a django view (or model):

from task import encode_video

def index(request):
    # launching video encoding
    encode_video.spool(filename=request.POST['video_filename'])
    return render_to_response('enqueued.html')

Now run uWSGI with spooler enabled

[uwsgi]
; a couple of placeholder
django_projects_dir = /var/www/apps
my_project = foobar
; chdir to app project dir and set pythonpath
chdir = %(django_projects_dir)/%(my_project)
pythonpath = %(django_projects_dir)
; load django
module = django.core.handlers:WSGIHandler()
env = DJANGO_SETTINGS_MODULE=%(my_project).settings
; enable master
master = true
; 4 processes should be enough
processes = 4
; enable the spooler (mytasks dir must exists !!!)
spooler = %(chdir)/mytasks
; load the task.py module
import = task
; bind on a tcp socket
socket = 127.0.0.1:3031

The only relevant option is the import one. It works in the same way as module but skips the WSGI callable search. You can use it to preload modules before the loading of WSGI apps. You can specify an unlimited number of import directives.

Example: web2py + spooler + timer

First of all define your spooler and timer functions (we will call it mytasks.py)

from uwsgidecorators import *

@spool
def a_long_task(args):
    print(args)
    
@spool
def a_longer_task(args)
    print("longer.....")

@timer(3)
def three_seconds(signum):
    print("3 seconds elapsed")

@timer(10, target='spooler')
def ten_seconds_in_the_spooler(signum):
    print("10 seconds elapsed in the spooler")

Now run web2py

uwsgi --socket :3031 --spooler myspool --master --processes 4 --import mytasks --module web2py.wsgihandler

As soon as the app is loaded, you will see the 2 timer running in your logs (stderr/stdout if you have not redirected uwsgi logs)

Now we want to enqueue tasks from our web2py controllers

Edit one of them and add

# be sure mytasks.py is in the pythonpath !!!

import mytasks

# this is a web2py action

def index():
    mytasks.a_long_task.spool(foo='bar')
    return "Task enqueued"

Notes

Signal-based decorators execute the signal handler in the first available worker. If you have enabled the spooler you can execute the signal handlers in it (leaving workers free to manage normal requests). Simply pass target='spooler' to the decorator args

@timer(3, target='spooler')
def hello(signum):
    print("hello")