shad.cc

To content | To menu | To search

Friday, February 19 2010

Adding virtual host support to webpy

Webpy is a tiny web framework. I use it a lot for my web-services applications.

In general, I let my web server (lighttpd) to handle virtual hosting. But as you may know, I am working on a CDN solution on top of Google App Engine, named CirruxCache. In that case, while I have absolutely no control on the server configuration, I need to handle virtual hosting from the code.

Webpy maps urls by iterating through a tuple. So my solution is quite simple: wrapping the tuple to override the __iter__ function according to an environment variable (HTTP_HOST).

Let's take this basic webpy example, without vhosting:

import web
 
urls = ('/(.*)', 'hello')
 
class hello(object):
    def GET(self, name):
        if not name:
            name = 'World'
        return 'Hello, %s' % name
 
if __name__ == "__main__":
   app = web.application(urls, globals()) 
   app.run()

Let's add the VhostMapper class:

import web
 
urls = {
      'default' : ('/(.*)', 'hello'),
      'my-vhost.domain.tld' : ('/(.*)', 'helloVhost')
      }
 
class hello(object):
    def GET(self, name):
        if not name:
            name = 'World'
        return 'Hello, %s !' % name
 
class helloVhost(object):
    def GET(self, name):
        return 'Hello %s' % web.ctx.environ['HTTP_HOST']
 
class VhostMapper(object):
    def __iter__(self):
        url = urls['default']
        if 'HTTP_HOST' in web.ctx.environ:
            vhost = web.ctx.environ['HTTP_HOST']
            if vhost in urls:
                url = urls[vhost]
        return iter(url)
 
if __name__ == "__main__":
   app = web.application(VhostMapper(), globals()) 
   app.run()

Finally, you can use curl or wget to test your vhosts:

$> curl -H "Host: my-vhost.domain.tld" http://localhost:8080/

It is not so early to announce that the next version of CirruxCache will handle virtual hosting :)

I am sure this simple hack can be easily reproduced to use virtual hosting in some other Rest frameworks.

Thursday, February 4 2010

Exploiter le SIP de sa Freebox

J'ai commencé à jouer récemment avec le compte SIP de ma Freebox.

Le compte SIP se paramètre via le panel de gestion de Free, dans l'onglet Téléphonie. Deux options sont possibles:

  1. Rediriger les appels vers la Freebox
  2. Rediriger les appels vers le serveur SIP

Dans le deuxieme cas, il vous faudra un téléphone IP si voulez conserver un téléphone physique connecté à la Freebox. Pour ma part, ayant un téléphone analogique standard, je redirige les appels vers la Freebox. J'utilise donc le serveur SIP uniquement pour passer des appels, et non en recevoir.

Il existe de nombreux clients SIP. J'utilise Linphone sur Linux que je trouve léger et stable.

Un point interessant, le serveur SIP est accessible de l'exterieur, autrement dit, vous pouvez appeler en illimités sur des fixes en France et dans certains pays depuis n'importe où en utilisant votre compte SIP Freephonie.

De plus, si vous êtes l'heureux possesseur d'un iPhone (ou tout autre téléphone sur Symbian, Windows Mobile ou Android), il vous suffit d'installer un client SIP (j'utilise iSipSimple sur iPhone) et de solliciter votre forfait data illimité (ou pas) pour passer vos appels vers les fixes tout en utilisant votre généreux forfait Free.

Attention aux appels vers les mobiles qui ne seraient pas avantageux.

Wednesday, January 6 2010

How to auto-update Chromium on Mac OS X, Linux and other

I want to try to use Chromium as my first web browser. But for the moment, Chromium suffers from a lack of package maintaining though it targets developers.

On my Gentoo Linux, I let portage to handle my Chromium version. But I also use Mac OS X at work, so I have written a little shell script to auto update my Chromium.app.

Put the content of this scripts wherever you want and make it executable ($> chmod 755 chromium_update.sh).

#!/bin/sh
# Chromium update script
# - by shad <shad@zaphod.eu>

# mac | linux | ...
OS=mac
LATEST=$(wget -q -O - http://build.chromium.org/buildbot/snapshots/chromium-rel-${OS}/LATEST)
INSTALL_DIR=/Applications
TMP="/tmp/update-chrome-$RANDOM"

(
mkdir $TMP ; cd $TMP
echo Download...
wget -q -O chrome.zip http://build.chromium.org/buildbot/snapshots/chromium-rel-${OS}/${LATEST}/chrome-${OS}.zip
if [ $? -ne 0 ] ; then
	echo Cannot update.
	exit 1
fi
echo Unzip...
unzip -qq chrome.zip
echo Copying...
rm -rf "${INSTALL_DIR}/Chromium.app"
mv chrome-$OS/Chromium.app "$INSTALL_DIR"
)

rm -rf $TMP

Then, set a crontab rule to choose the update interval ($> crontab -e). Mine runs every day at 12:00.

# Run Chromium update script every day at 12:00
00 12 * * * /Users/shad/mbin/chromium_update.sh

I hope it will be useful for someone as much as for me.

Friday, October 30 2009

CirruxCache: speeds up your HTTP app using Google Appengine as a CDN

It is a great moment, for the first time since I have started to work at Zoomorama, I have just released as open-source an important part of our server platform.

I previously explained how to use Google AppEngine as a Content Delivery Network (CDN). CirruxCache project concretizes this idea. I released the first version based as the one we use in production.

Here is the features it currently supports:

  • honor Cache-Control
  • cache TTL override
  • several POP (Point Of Presence) configuration mapped on custom base-url
  • ignore query string
  • POST forwarding
  • expired entries garbage collection
  • extensibility


CirruxCache is not documented at the moment even if you would be able to use it after reading the comments in the app.py file. I'll document this app in the next few days, but if you need more documentation, don't hesitate to contact me.

The project website.

Thursday, July 30 2009

Speed up HTTP delivering using Google AppEngine

Google AppEngine provides an high-level cloud service which means that your application will be distributed automatically on top of the Google platform. All of your code will depends on the AppEngine SDK, so it could be risky to develop complex application on it.

I develop a webservice application for content delivering and content publishing at Zoomorama. We currently use Akamai CDN as a simple cache layer to improve data delivering accross the world. It is interesting for me to use AppEngine in the same way: without changing anything on my existing code base. I have found some posts on blogs dealing with this AppEngine usage, but they are not focused on dynamic HTTP caching like a real CDN.

Principle is very simple, all HTTP requests on my AppEngine application will be copied to the AppEngine Datastore. Moreover data which are delivered through AppEngine are cached by AppEngine servers. The code below is a tiny proof of concept:

# HTTP caching on Google App Engine
# - by shad <shad@zaphod.eu>
#
 
import web # webpy 0.3x
from google.appengine.ext import db
from google.appengine.api import urlfetch
 
origin = 'http://my.website.com'
 
urls = (
    '(/.*)', 'Root'
    )
 
class Cache(db.Model):
  data = db.BlobProperty(default=None)
  headers = db.ListProperty(str)
 
class Root(object):
  def GET(self, request):
    cache = self.readCache(request)
    if cache is None:
      cache = self.writeCache(request)
    for h in cache.headers:
      print h
    return cache.data
 
  def readCache(self, key):
    cache = cache = Cache.get_by_key_name(key)
    if cache:
      return cache
 
  def writeCache(self, request):
    url = origin + request
    response = urlfetch.Fetch(url=url)
    if response.status_code != 200:
      raise web.NotFound()
    cache = Cache(key_name=request)
    cache.data = db.Blob(response.content)
    cache.headers = []
    for k, v in response.headers.iteritems():
      cache.headers.append('%s: %s' % (k, v))
    cache.put()
    return cache
 
if __name__ == '__main__'
  app = web.application(urls, globals())
  app.cgirun()

I use webpy to depends on the AppEngine SDK as less as possible.

I have almost finished the production version of this application. I am doing some performance tests. This application is closed source for now. But I am going to release the code source in few weeks.

This version will include:

  • Fetch from Memcache (about 10 times faster).
  • Headers forwarding.
  • Read "Cache-Control" and "Expires" to define a TTL (rfc 2616).
  • Multi origins (according to url mount points).
  • Other small features (force TTL, ignore query string, etc...).


It is important to note that AppEngine does not keep running instances of your application (your CGI is distributed and it is executed on demand). So this application have to start very quickly (no configuration file, no dynamic generation, etc...).

- page 2 of 6 -