TurboGears 2.3 Hidden Gems #3 – Minimal Mode

Recently it has been pointed out to me that the Wikipedia TurboGears page was pretty outdate, so I ended up reading and updating a few sections. As that page reported a “2.x Components” section where Pylons was listed, I wasn’t sure what to do as it is not a component of TurboGears anymore since 2.3. Should I create a new “2.3+ Compoenents” section? Should I explain that it was used before? Then I realized that many people probably don’t even know that it got replaced and that gave me the idea of writing a blog post about the TurboGears2 Minimal Mode which is what replaced most of the Pylons functionalities in 2.3.

For people that didn’t follow TurboGears recent evolutions, the Minimal Mode is actually a micro-framework like configuration of TurboGears2 which has been available since version 2.3.

Minimal Mode was actually created as a side-effect of TurboGears becoming independent from Pylons while the team was also working on a major framework speed-up. Apart from the speed gain and opening the way to Python3, through Pylons removal, Minimal Mode proved itself to be quite convenient for rapidly prototyping Apps, HTTP APIs or showcase small examples. This is particularly visibile through the various Snippets that got created in Runnable.io TurboGears category.

The impact of the refactoring that lead to minimal mode is actually clear when comparing the TurboGears2 dependencies on a recent release

prova

to the more than forty dependencies of one of the earliest 2.x releases

tg2_dot

The new internal core made possible to greatly reduce dependencies to those that were really needed. Removing a few and moving some, that were only needed when special features were enabled, to the application itself.

By default TurboGears2 starts in full-stack mode, expecting the application to reside in a python package and enabling all the features that are commonly available to TurboGears2 applications. For backward compatibility reasons, minimal mode must be enabled explicitly through the minimal=True option. This ensures that all the apps created before 2.3 continue to work while the framework can be used as a micro-framework with minimum effort.

Another requirement is that the root controller must be explicitly passed, as TurboGears looks for it in the application package, and as we have no package configured it won’t be able to find any.

config = AppConfig(minimal=True, root_controller=RootController())

While tg.devtools and the gearbox quickstart command can continue to be used to create full stack applications, creating a minimal mode app is as easy as installing the TurboGears2 package itself and creating an application with any root controller (see TurboGears Documentation for a complete tutorial)

from tg import expose, TGController, AppConfig

class RootController(TGController):
    @expose()
    def index(self):
        return 'Hello World'

config = AppConfig(minimal=True, root_controller=RootController())

application = config.make_wsgi_app()

then the application can be served with a WSGI compatible server

from wsgiref.simple_server import make_server

print "Serving on port 8080..."
httpd = make_server('', 8080, application)
httpd.serve_forever()

If people need additional features that TurboGears usually provide like sessions, caching, authentication, database, helpers and so on… they can enable them with a bunch of options available through the AppConfig object and only in that case additional dependencies like Beaker or SQLAlchemy are required.

The recently released 2.3.7 explicitly worked on making minimal mode easier to use, especially when database is involved. Before 2.3.7 you were required to have a package for your application from where the model could be imported, but now the model can actually be any object that exposes an init_model function. The TurboGears tutorial actually uses a Bunch object which is in fact a dictionary.

After playing around with the minimal mode for a bunch of small projects I realized that while many projects start small they quickly tend to became big, and being able to switch from a micro-framework to a full stack framework when needed, without actually changing framework at all, has proved a valuable feature. One that totally adheres the TurboGears mission of being “the web-framework that scales with you” 🙂

As not many people, even in TurboGears community, were comfortable with minimal mode I hope this post clarified a bit what it is minimal mode and how you can benefit from it.

TurboGears 2.3 Hidden Gems #2 – Application Wrappers

One of the less known features introduced in TurboGears 2.3 are application wrappers.
Application wrappers are much like controller wrappers (available since 2.2), but instead of wrapping controllers they actually wrap the whole application providing an easier way to implement what in plain WSGI is done through middlewares.

The advantage of application wrappers over middlewares is that they have full access to TurboGears stack, they can access the current request, the database, session and caches as the application itself would do.

The great part is that, as they run between TGApp and TGController, they can also replace the TurboGears Context and the TurboGears Response providing a great way to hijack requests, responses or even replace entire features of the framework like the cache layer. A very similar concept is available in other frameworks like Pyramid Tweens.

A very simple application wrapper that intercepts exceptions and logs them without messing with the standard TurboGears error handling might look like:

class ErrorLoggingWrapper(object):
    def __init__(self, handler, config):
        self.handler = handler
        self.config = config

    def __call__(self, controller, environ, context):
        path = context.request.path
        try:
            return self.handler(controller, environ, context)
        except:
            log.exception('Error while handling %s', path)
            raise

The wrapper can then be enabled calling

base_config.register_wrapper(ErrorLoggingWrapper)

inside config/app_cfg.py

Now that we have an application wrapper able to log exceptions we can decide for example to add another one that suppresses exceptions and prints “Something went wrong!”, as it is possible to specify the order of execution for application wrappers we can register a SuppressErrorsWrapper that should execute after the ErrorLoggingWrapper:

from webob import Response

class SuppressErrorsWrapper(object):
    def __init__(self, handler, config):
        self.handler = handler
        self.config = config

    def __call__(self, controller, environ, context):
        try:
            return self.handler(controller, environ, context)
        except:
            return Response('Oh! Oh! Something went wrong!', status=500, content_type='text/plain')

Then it can be registered after the ErrorLoggingWrapper using:

base_config.register_wrapper(SuppressErrorsWrapper, after=ErrorLoggingWrapper)

While applications wrappers are a powerful feature, most of their power comes from the new response management refactoring that makes possible to access the current context and replace the outgoing response while working with high level objects instead of having to manually cope with WSGI.

Stroller goes open source

Stroller
Stroller

Stroller it is our way to do e-commerce! We have already some clients that we fit on this module written in python and easily importable to have a fully e-commerce section to work with.

Just after testing it in production for some customers and already in a stable shaped we decided to deliver it as a community edition in free software.

In fact you can now download it from pypi:

http://pypi.python.org/pypi/stroller

or just checkout the sources over our repositories:

hg clone http://repo.axant.it/hg/stroller

For now stroller does not have a site and a home ;( but fortunately all of our projects are listed in a fancy “temp house” on project.axantlabs.com as it has stroller, just checkout http://projects.axantlabs.com/Stroller

You’ll see there the main features of this art of software and enjoy the sweetness!

Updates are coming for it we have just in mind some strategic moves to make it a ‘first choice’ product.

libACR 0.6 and ACRcms 2.0 are out

As we are going deep with ACR our cms, it’s becoming a full featured product, we are using it more and more in production and we decided to release a new version on pypi.

New features include:

  • Plugins Support. You can now add your own sections to the administration panel, views to the add slice menu and implement new functions or views.
  • Multisite support, serve multiple sites from one single WSGIDaemonProcess
  • Themes Support, create your own themes for the ACRCMS
  • A simple Scripting language to automate some actions on theme setup
  • User Defined Views, create new type of contents without having to write a single line of code
  • SliceGroup Admin to permit to editors to change content of image galleries and news without have to need access to the admin panel
  • Support for Disqus comments
  • Export html slicegroups as RSS feeds
  • Various Plugins for Slideshows, Image Galleries, Accordions, etc.
  • New Script type and Script menu to make easier to add and manage javascript inside your pages
  • RDisk should now be faster and has content caching
  • Change My Password for current logged in user
  • Permit to store binary data inside contents as base64 and serve them through the /data call.
  • Integration with the Stroller eCommerce (stroller will be released opensource in the near future)

Here is the complete changelog since our latest release:

* work around to make it go with buggy sprox 0.6.10 (patched version still not released)
* Fix problem with Tg2.0 not casting headers to str
* Add delete udv and preview template for udvs
* Finalize user defined views with support for single selection fields, html and files
* Permit to serve fields encoded as per HTML5 base64 data source definition
* Add etag caching to rdisk based on file modified time
* Skeleton for user defined views
* Merge changes from master branch which improve multisite support
* Move every reference to stroller inside the stroller plugin itself
* Working delete action for slicegroup admin
* ACR might be mounted averywhere, never suppose its url, always use url() from libacr to generate acr urls
* Fix for TG2.0.3 (before tg2.1 remainder is a tuple and is not editable)
* SliceGroupAdmin plugin seems to work fine for adding things
* Make it work with Pylons1.0
* First import of sgadmin
* Stroller plugin preparatives
* Google Analytics change section
* Added new LinkedImage view same as the Image, just with a link utility. Must be evolve it, in the future to make him reuse the original view, without code duplication.
* Try to solve problems with repoze.who and repoze.what when running multiple acr sites inside the same daemon process and group
* Remove code render template from views as it collides with plugins
* Add setup script support to themes and fix default page url
* MCE options
* Fix icons and section for new plugins
* Fix for setup-app failing due to plugins
* gmap working (hopefully)
* gmap.js from static to plugin
* Merged single process mode, seems to be stable enough
* Moved GoogleMaps viewer to GoogleMaps plugin
* Disqus plugin
* Google Analytics from plugin static to site instance
* GoogleMaps key moved to database & added modify plugin
* Added tag classes for slices
* Add slice cloning
* Make rdisk_root dynamic to be able to run multiple instances in same process
* Permit to force lang from request
* Use genshi dict dotted access instead of a module
* permit to access and manipulate content from genshi slice
* add tabs plugin
* Added a class to the page as the uri of the page to make possible Custom css classes for page, hence different style for different pages
* Style fixes to administration menu
* fix crash when unable to contact pypi
* multiengine
* tests with multiengine
* engine from config
* first attemp at making a single process acr in the simplest way
* bind engine at each request
* cache session per db
* More experiments to make acr work on single process
* try to make acr work inside one single process
* Automated merge with ssh
* make section use id instead of class and declare in a less colliding way
* detect script slices also derived from default page
* Removed unused imports
* Fix done to correct the position of the excerpts under IE7
* Closes #43 it adds exception handling in case of failure of pypi version check, and just logs a warning to notify
* Minor edits
* Replaceing file templating, with python's Template string module
* Removed javascript putting css rules in place to allow 100% with of the edit menu; So added a relative container to the absolute positioned edit menu
* Minor edits
* Added new plugin to insert google analytics tracking to the site
* removed useless import
* Moved update check under helpers as it is more appropriate and, changed naming convention to follow standards
* Added new plugin to add uservoice feedback tab to pages
* add slice type class to slices, refactor properties management and add find_by_property helper
* Added help on accordion plugin, to help user interaction on creation
* Added same size of the edit button menus as the slice/slicegroup element
* Created new container for the heading admin section as pseudo-tabbed links + minor style edits on the css
* Added release update notification
* make script view wrap content with script tag and migrate existing plugins to use it
* add script slice type and menu
* Make rdisk upload view type dependant, fix videos and make deletion work on actual slice content instead of slice name
* Removed oops, console.log + Removed timestamp from end of slicegroup names to allow, reusability of the acrodion within a page, if you edit and recreate the sliceroup
* add description placement management for image slice
* Added Accordion Plugin, it will put a template for creation of Accordion galleries filters by tag on uploaded images
* fix problem with image thumbnails not showing if not logged in and add Slideshow plugin
* plugin injected resources
* add image, video and file slices to add slice menu if there is rdisk available
* themes plugin for acr
* initial work to make acr_cms working on multiple sites with only on installation, necessary to implement themes support
* improvements to edit bar

ACR goes for “Packt Open Source CMS Awards”!

Like every year Packt Publishing organizes the Open Source CMS Awards, right now they started the nomination phase.

The following categories make up the contest.

  • Open Source CMS

  • Hall of Fame CMS

  • Most Promising Open Source Project

  • Open Source E-Commerce Applications

  • Open Source JavaScript Libraries

  • Open Source Graphics Software

We decided to propose ACR for the Most Promising Open Source Project and Open Source CMS.

Even if young our CMS is already interesting since it’s quite easy to deploy,  it integrates in other turbogears applications (like we did for iJamix) in a breeze and has already most of the features you would expect from a full fledged CMS.

In our humble opinion is the best and most promising Turbogears2 based CMF/CMS out there.

VideoLAN Web Plugin: xpi vs crx

One of the main issue while preparing streaming solution is answering the obnoxious question:

  • Question: Is possible to use the service through a browser?
  • Answer: No, rtsp isn’t* http, a browser isn’t a tool for accessing any network content.
  • * Actually would be neat having rtsp support within the video tag but that’s yet another large can of worms

Once you say that you have half of your audience leaving. Non technical people is too much used to consider the browser the one and only key to internet. The remaining ones will ask something along those lines:

  • Question: My target user is a complete idiottechnically impairednaive and unaccustomed and could not be confronted with the hassle of a complex installation procedure, is there something that fits the bill?
  • Answer: VideoLAN Web Plugin

Usually that makes some people happy since it’s something they actually know or at least they have heard about. Some might start complaining since they experienced an old version and well it crashed a lot. What would you be beware of is the following one:

  • Question: Actually I need to install the VideoLAN Web Plugin and it requires attention, isn’t there a quicker route?
  • Answer: Yes xpi an crx for Firefox an Chrome

Ok, that answer is more or less from the future and it’s the main subject of this post: Seamless bundling something as big and complex as vlc and make our non tecnical and naive target user happy.

I picked the VideoLAN web plugin since it is actually quite good already, has a nice javascript interface to let you do _lots_ of nice stuff and there are people actually working on it. Additional points since it is available on windows and MacOSX. Some time ago I investigated how to use the extension facility of firefox to have the fabled “one click” install. The current way is quite straightforward and has already landed in the vlc git tree for the curious and lazy:


  
    vlc-plugin@videolan.org
    VideoLAN
    1.2.0-git
    
      
        {ec8030f7-c20a-464f-9b0e-13a3a9e97384}
        1.5
        3.6.*
      
    
  

Putting that as install.rdf in a zip containing a directory called plugins with libvlc, it’s modules and obviously the npapi plugin does the trick quite well.

Chrome now has something similar and it seems also easier so that’s what I put in the manifest.json:

{
"name": "VideoLAN",
"version": "1.2.0.99",
"description": "VideoLan Web Plugin Bundle",
"plugins": [{"path":"plugins/npvlc.dll", "public":true }]
}

Looks simpler and neater, isn’t it? Now we get to the problematic part about chrome extension packaging:

It is mostly a zip BUT you have to prepend to it a small header with more or less just the signature.

You can do that either by using chrome built-in facility or by a small ruby script. Reimplementing the same logic in Makefile using openssl is an option, for now I’ll stick with crxmake.

Then first test build for win32 are available as xpi and crx hosted on lscube.org as usual.

Sadly the crx file layout and the not so tolerant firefox xpi unpacker make impossible having a single zip containing both the manifest.xpi and the install.rdf served as xpi and crx.

by the way, wordpress really sucks

The zoom factor in webkit and gecko

Apparently all the major browsers tried to provide a zoom facility to improve the overall accessibility for the web users. Sadly that often breaks horribly your layout, if you are developing pixel precise interaction you might get a flood of strange bug reports you might not reproduce.

We got bitten by it while developing Glossom, severely…

Our way to figure out it’s value is quite simple once you discover it: Firefox scales proportionally the borders and makes the block dimensions constant, Webkit seems to do the opposite. It’s enough to check if a element with known dimensions and border width has it’s value reported as different and your can find our which is the factor.

This obviously is quite browser dependent and nobody grants that in different version it might get changed, anyway so far it seems to serve us well.

ACR gets Slice Templates

ACR is a flexible and quite powerfull CMS library, but users have to learn Slices and Slicegroups to be able to insert more advanced content like photo galleries, videos and a news section. To simplify this process we created “Slice Templates”.

Slice Templates are actually a set of common ways to use slices to insert more advanced content. The first two slice templates implemented are:

  • Youtube Videos which make easy to insert a youtube video inside an ACR Page.
  • Photo Gallery which makes easy to insert a photo gallery with cool effects inside an ACR Page.

Next template to come will be a news section, for now you can upgrade your libACR and start using the templates feature or take a look at the screenshots of the currently implemented templates

Turbogears Glossom finally live

We started to collaborate with the Glossom project more than an year ago, the first version was a Ruby on Rails prototype and has served well thousand of users for about an year. When the Glossom team decided to rewrite the software to move from a prototype to a more complete software we studied which frameworks were available and finally decided for Turbogears2.

Turbogears is a quite complete and really flexible framework, we used it since version 1.0 for about 3 years and the more we used it the more we loved it. It might have its problems but differently from other solutions when the framework has a limit it is really easy to bypass the framework and implement your own solution without any hack and with a clean way. Turbogears developers really focused on letting your tune each component in any way, and even totally replace it with something else if you don’t like it.

After six months of development the application has finally gone live and is now serving ~10000 registered users with its servers based on an apache load balancer, mod_wsgi turbogears application servers and mysql database servers.

So welcome to Glossom and thank you to Turbogears team for all the help!

We also have to say thank you to FFmpeg for now converting our videos and FlyPDF for generating our PDFs! 😀 (you can download FlyPDF from sourceforge if you want to give it a try)