ACR Slice Preview support, remote disk, Comment and File views.

Latest version of ACR, our opensource cms for turbogears, got some new interesting features:

Now each view can have a “preview mode” which shows a minimized version of the slice to which is binded. For example you can show in your home page a short version of a news linking to the complete one, or you can show the thumbnail of an image and link to the full version. This can be quite useful in some situations and can be triggered by setting preview=1 inside a slice group. Each slice inside the group will render in preview mode.

Also the Remote Disk feature has been implemented and can be accessed as /rdisk. Inside this file manager you will be able to upload any file which can be referred and used inside you web page. The File View itself has been implemented to permit to quickly link to files and serve them, it will show images or videos or it will provide a link to download files of unknown types.

Also cloned from our iJamix project now ACR has a Comment view, which permits to insert a comments thread inside any page.

Quick fix for ToscaWidget 0.9.6 bug with ListForm

There is a little bug in ToscaWidget 0.9.6 when creating a form with a ListForm. The solution can be to use a TableForm or to fix the ListForm itself. The problem is inside the genshi template for the ListForm and can be fixed by changing tw.forms-0.9.6-py2.5.egg/tw/forms/templates/list_form.html in each line where there is a call to field.display(value_for(field), **args_for(field)) this must be surrounded with XML(…) so that the result from field.display won’t be escaped by genshi.

This should fix the problem when you see the XML code instead of the form itself inside your web page.

(Val)grinding your code

One of of the most exoteric tool when working on software projects that developers should be using more often than they do, is certainly Valgrind. It’s a multi-faced tool: timing profiler, memory profiler, race conditions detector, … it can do all this because it’s mainly a simulated CPU, which executes the code, keeping track of what happens, and replacing some basic functions like the allocation routines.

So this works pretty well, to a point; unfortunately it does not always work out that well because there are a few minor issues: it needs to know about all the instructions used in the programs (otherwise it raises SIGILL crashes), and right now it does not know about SSE4 on x86 and x86-64 processors. And it has to know about the places in the C library where it cannot correctly keep scores via a series of suppression files. Turns out it currently fails at properly knowing about the changes in glibc 2.10, which makes it pretty difficult to find the real issues right in this moment with the most bleeding edge tools available.

But beside these problems with the modern tools and hardware, there are other interesting edges; one of these, the one I’m going to write about today, relate to the way the memory checker tool identifies leaks in the software: all the memory areas that cannot be reached in some way at the end of the execution, are considered leaked. Unfortunately this often enough counts in the memory areas that are allocated at one point and would just be freed at the end of the process, which can be said not to be a leak at all.

All the memory areas that are only to be freed at the end of a process don’t really need to be freed at all: the operating system will take care of that; leaks are those where the memory is used up and never freed nor used during execution; especially bad are those leaks that happen inside either iterative cycles or upon similar inputs, because they build up.

Actually freeing up the resources that does not need to be freed up can increase the size of the executable section of a binary (executable or library) for no good reason during production usage of the software, it is thus often considered a bad idea. At the same time, freeing them up makes it easier to find the actual bad leaks in the software. My choice for this is to write the actual source code to do the freeing up, but only compile it when debug code is enabled, just like assertions and debug output.

This way, the production builds won’t have unneeded code, while the debug builds will give proper results out on tools like valgrind. But it doesn’t stop at freeing the resources at the end of the main() function, because sometimes you don’t preserve reference to the actual memory areas allocated during initialization (since they are not strictly needed), or you just keep them as static local variables, which are both reported as unreachable by the end of the process, and are not easy to free up. To work this out, in feng, I’ve been using a slightly more sophisticated solution, making use of a feature of recent GCC and ICC versions: destructor functions.

Destructor functions are called within the fini execution path, after the main() and before the process is unloaded and the resource freed automatically. Freeing the memory at that point is the ideal situation to make sure that all the resources are freed up properly. Of course, this reduces the cases when the code is enabled to having both debug system enabled, and a compiler that supports the destructor functions. But this is a broad enough definition to allow proper development. A simple check with autoconf and the thing is done.

By switching the local static variables with unit-static variables (which, once compiled, are basically the same thing), and adding a cleanup function, almost all memory can be freed without having an explicit deallocation function. There is still the matter of initializing these memory areas; in this case there are two options: either you just initialize them explicitly, or you initialize them the first time they are needed. Both these options allow for the size of the structure to be chosen at runtime (from a configuration file), for example for the number of configuration contexts to initialize. The third way, that is to initialize them with constructor functions, also works out well for fixed-size memory areas, but since I don’t really have a good reason for which they could be better than the alternative the to just call a function during initialization, I don’t have any reason to take them into consideration.

For dynamic initialization, the interface usually used is the “once” interface, which also allows for multithreaded applications to have run-once blocks of code without having to fiddle with mutexes and locks; this is implemented by the pthread_once_t handling functions from the POSIX thread interface, or GOnce as provided by glib. This allows delayed allocation and initialization of structures which is especially useful for parts of the code that are not used constantly but just in some specific cases.

All the code can be found in your local feng repository or the nearest git server!

ACR TurboGears2 CMS

We recently developed a CMF as a base for some sites for some of our client, facing the fact that actually there are really few CMS and CMF developed in turbogears we decided to release it opensource so that it can be useful to other people.

ACR (Advanced Content Repository) it is more a developer oriented CMS than an end-user oriented one, we actually used it already as the base for 3 of our sites currently under development and we found the content-slice-view separation quite powerful, but also quite hard for our users to face with. This can be worked around by implementing alternative administration tools specific for the site domain, but ACR also has a few quick tools like HTML Node editor and HTML content editor built inside which might suffice in most cases.

ACR is free software under GPL license and If you want to give it a try you can find it as usual on labs at