MongoDB Aggregation Pipeline and Pagination

One of the most common problems in web development is paginating a set of data. It’s common that the set of data we need to show our users is pretty big and we might want to show only a part of it retrieving the next slices only when requested. Skip/Limit Pagination The most common way to achieve this is usually to count the number of items and split them in pages, each of a fixed set of items. This is usually achieved through the limit, skip and count operators. For the purpose of this post I’ll be relying on a collection containing tickets for a project management tool, each document looks like: {u’_id': ObjectId(‘4eca327260fc00346500000f’), u’_sprint': ObjectId(‘4eca318460fc00346500000b’), u’complexity': u’days’, u’description': u’This […]

MongoDB and UnitOfWork love or hate?

One of the most common source of issues for MongoDB newcomers is the lack of transactions, people have been used to work with transactions for the past 10 years and probably their web frameworks automatically starts, commits and rolls back transactions for them whenever something wrong happens. So we are pretty used to web development environments where the problem of writing only a part of our changes is usually solved out of the box. When people first approach MongoDB I noticed that this behaviour is often took for granted and messed up data might arise from code that crashes while creating or updating entities on the database. No transaction, no party? To showcase the issue, I’ll try to came up […]

TurboGears 2.3 Hidden Gems #3 – Minimal Mode

Recently it has been pointed out to me that the Wikipedia TurboGears page was pretty outdate, so I ended up reading and updating a few sections. As that page reported a “2.x Components” section where Pylons was listed, I wasn’t sure what to do as it is not a component of TurboGears anymore since 2.3. Should I create a new “2.3+ Compoenents” section? Should I explain that it was used before? Then I realized that many people probably don’t even know that it got replaced and that gave me the idea of writing a blog post about the TurboGears2 Minimal Mode which is what replaced most of the Pylons functionalities in 2.3. For people that didn’t follow TurboGears recent evolutions, […]

DEPOT 0.1.0 new Aliases Feature for Easy Storage switch

DEPOT is a file storage and retrieval framework we created to solve the need of switching different storage systems when deploying in different environments. We wanted a unique and cohesive API that made possible to keep storing files the same way independently from where they were actually stored. As systems evolve and change during time we also wanted the ability to switch those storages whenever required without breaking past files or changing any code. That lead to various features of DEPOT that specifically pointed this problem, the last of this list is the new Storage Aliases support. If you used a single storage, myfiles, registered as the default one, when you wanted to switch from Local Storage to S3 that was […]

TurboGears 2.3 Hidden Gems #2 – Application Wrappers

One of the less known features introduced in TurboGears 2.3 are application wrappers. Application wrappers are much like controller wrappers (available since 2.2), but instead of wrapping controllers they actually wrap the whole application providing an easier way to implement what in plain WSGI is done through middlewares. The advantage of application wrappers over middlewares is that they have full access to TurboGears stack, they can access the current request, the database, session and caches as the application itself would do. The great part is that, as they run between TGApp and TGController, they can also replace the TurboGears Context and the TurboGears Response providing a great way to hijack requests, responses or even replace entire features of the framework […]

TurboGears 2.3 Hidden Gems #1 – New Response Management

TurboGears2.3 has been a major improvement for the framework, most of its code got rewritten to achieve less dependencies, cleaner codebase a cleaner API and a faster framework. This resulted in reduction to only 7 dependencies in minimal mode and a 3x faster codebase. While those are the core changes for the release, there are a lot of side effects that users can exploit at their benefit. This is the reason why I decided to start this set of posts to describe some of those hidden gems and explain users how to achieve the best from the new release. The first change I’m going to talk about is how the response management got refactored and simplified. While this has some […]

It’s a Pluggable World

One of the new additions in TG2.1.4 has been the support for the so called pluggable applications, this is a really powerful and convenient feature that probably not enough TurboGears users started embracing. For people that never used them, pluggable applications provide a python package that can be installed and “plugged” inside any existing TurboGears application to add new features. Django has been probably the first framework to bring this feature to Python world and TurboGears implementation tries to be as convenient by making pluggable applications identical to plain TurboGears applications and providing a “quickstart-pluggable” command that creates the application skeleton for you. Pluggable applications can be installed using easy_install or pip and they can off course depend on any […]

What’s new about Sprox 0.8

Today Sprox 0.8 got released, it is the first release to add ToscaWidgets2 support. Depending on which version of ToscaWidgets is available inside your environment Sprox will either use TW1 or TW2 to generate its forms. Being mostly a TW2 oriented release it might seem that not a lot changed since the previous version, but a little gem is hidden between all the TW2 changes as Sprox now supports setting default behavior for models themselves using the __sprox__ attribute inside model declaration. class Parent(DeclarativeBase): __tablename__ = ‘parents’   uid = Column(Integer, primary_key=True) data = Column(String(100))   class Child(DeclarativeBase): __tablename__ = ‘children’   class __sprox__(object): dropdown_field_names = {’owner': [’data’]}   uid = Column(Integer, primary_key=True) data = Column(String(100))   owner_id = Column(Integer, […]

TurboGears future performances comparison

Recently I decided to give a quick benchmark for curiosity to the going to be branches of TurboGears2. I quickstarted a simple genshi based application (plain turbogears2 quckstart) and then I created a plain controller method without template, to avoid counting the template generation overhead. The application has been installed in three virtual environments: one with TG2.1.4, one with the development branch which is going to be TG2.2 and one with the development branch which is going to be TG2.3 The following graph reports the resulting requests/second that my pc has been able to serve on each turbogears version. I have to admit that I’m quite happy with the results, the grow is steady and TG2.3 seems to be three […]

Mastering the TurboGears EasyCrudRestController

One of the key features of TurboGears2 is the great CRUD extension. Mastering the CRUD extension can really make the difference between spending hours or just a few minutes on writing a web app prototype or even a full application. The CRUD extension provides two main features, the CrudRestController which is meant to help creating totally custom CRUDs and the EasyCrudRestController which provides a quick and easy way to create CRUD interfaces. I’ll focus on the EasyCrudRestController as it is the easiest and more productive one, moving forward to the CrudRestController is quite straightforward after you feel confident with the Easy one. The target will be to create, in no more than 40 lines of controller code, a full featured […]