Archive for March, 2010
Redis and MongoDB insertion performance analysis
Tuesday, March 16th, 2010 | Computer Science | Comments
Recently we had to study a software where reads can be slow, but writes need to be as fast as possible. Starting from this requirement we thought about which one between redis and mongodb would better fit the problem. Redis should be the obvious choice as its simpler data structure should make it light-speed fast, and actually that is true, but we found a we interesting things that we would like to share.
This first graph is about MongoDB Insertion vs Redis RPUSH.
Up to 2000 entries the two are quite equivalent, then redis starts to get faster, usually twice as fast as mongodb. I expected this, and I have to say that antirez did a good job in thinking the redis paradigm, in some situations it is the perfect match solution.
Anyway I would expect mongodb to be even slower relating to the features that a mongodb collection has over a simple list.
This second graph is about Redis RPUSH vs Mongo $PUSH vs Mongo insert, and I find this graph to be really interesting.
Up to 5000 entries mongodb $push is faster even when compared to Redis RPUSH, then it becames incredibly slow, probably the mongodb array type has linear insertion time and so it becomes slower and slower. mongodb might gain a bit of performances by exposing a constant time insertion list type, but even with the linear time array type (which can guarantee constant time look-up) it has its applications for small sets of data.
I would like to say that this benchmarks have no real value, as usual, and have been performed just for curiosity
You can find here the three benchmarks snippets
import redis, time MAX_NUMS = 1000 r = redis.Redis(host='localhost', port=6379, db=0) del r['list'] nums = range(0, MAX_NUMS) clock_start = time.clock() time_start = time.time() for i in nums: r.rpush('list', i) time_end = time.time() clock_end = time.clock() print 'TOTAL CLOCK', clock_end-clock_start print 'TOTAL TIME', time_end-time_start
import pymongo, time MAX_NUMS = 1000 con = pymongo.Connection() db = con.test_db db.testcol.remove({}) db.testlist.remove({}) nums = range(0, MAX_NUMS) clock_start = time.clock() time_start = time.time() for i in nums: db.testlist.insert({'v':i}) time_end = time.time() clock_end = time.clock() print 'TOTAL CLOCK', clock_end-clock_start print 'TOTAL TIME', time_end-time_start
import pymongo, time MAX_NUMS = 1000 con = pymongo.Connection() db = con.test_db db.testcol.remove({}) db.testlist.remove({}) oid = db.testcol.insert({'name':'list'}) nums = range(0, MAX_NUMS) clock_start = time.clock() time_start = time.time() for i in nums: db.testcol.update({'_id':oid}, {'$push':{'values':i}}) time_end = time.time() clock_end = time.clock() print 'TOTAL CLOCK', clock_end-clock_start print 'TOTAL TIME', time_end-time_start
Remember me in Turbogears2
Wednesday, March 10th, 2010 | Software Development | Comments
One of the problems with TG2 is that the current version doesn’t support a “standard” way to remember the user after he closes the browser. We have been able to find a quick and dirty solution that we like to share here. Keep in mind that this solution only works with recent versions of repoze.who, this works with TG2.0.3, but might not work with previous releases of TG2.
Inside the login.html we set a cookie for the remember_me option to pass it to the controller and then inside the post_login we change the cookie.
Supposing you have a #remember_me checkbox inside your login.html you can add this to set the cookie:
function set_remember_cookie() { is_checked = jQuery('#remember_me:checked').length; if (is_checked) document.cookie = 'remember_me=1'; else document.cookie = 'remember_me=0'; } jQuery(document).ready(function() { set_remember_cookie(); jQuery('#remember_me').click(set_remember_cookie); }
Then inside your post_login method in the root controller you can place:
remember_me = request.cookies.get('remember_me', 0) try: remember_me = int(remember_me) except: remember_me = 0 if remember_me: request.identity['max_age'] = 2252000 # 30 days request.identity['userdata']= "max_age" # force cookie refresh
This would remember the user for 30 days even if he closes the browser.
Turbogears authentication over mongodb users database
Wednesday, March 3rd, 2010 | Software Development | Comments
As we saw that there isn’t a lot of documentation around about how to perform authentication in turbogears over mongodb we decided to create a simple code snippet and public it here to help people trying to obtain the same thing.
This is mainly a proof of concept and is quick and dirty way to obtain it. You will probably have something like ming as your model, instead of directly accessing mongo.
This code also validates password over the clear text one, you will probably have hashed passwords in your database, so remember to change validate_password method as required
To make it work you will have to place this code inside your config.app_cfg, it also expects you to have you database exposed as db inside your model
from my_app.model import db from zope.interface import implements from repoze.who.interfaces import IAuthenticator, IMetadataProvider from repoze.who.plugins.friendlyform import FriendlyFormPlugin from repoze.who.plugins.auth_tkt import AuthTktCookiePlugin from repoze.who.middleware import PluggableAuthenticationMiddleware def validate_password(user, password): return user['password'] == password class MongoAuthenticatorPlugin(object): implements(IAuthenticator) # IAuthenticator def authenticate(self, environ, identity): if not ('login' in identity and 'password' in identity): return None login = identity.get('login') user = db.users.find_one({'user_name':login}) if user and validate_password(user, identity.get('password')): return identity['login'] class MongoUserMDPlugin(object): implements(IMetadataProvider) def add_metadata(self, environ, identity): user_data = {'user_name':identity['repoze.who.userid']} identity['user'] = db.users.find_one(user_data) class MyAppConfig(AppConfig): auth_backend = 'sqlalchemy' #this is a fake, but it's needed to enable #auth middleware at least on TG2.0 login_url = '/login' login_handler = '/login_handler' post_login_url = None logout_handler = '/logout_handler' post_logout_url = None login_counter_name = None def add_auth_middleware(self, app, skip_authentication): cookie_secret = pylons_config.get('auth_cookie_secret', 'myapp_adsfsdfh3423') cookie_name = pylons_config.get('auth_cookie_name', 'myapp_auth') who_args = {} form_plugin = FriendlyFormPlugin(self.login_url, self.login_handler, self.post_login_url, self.logout_handler, self.post_logout_url, login_counter_name=self.login_counter_name, rememberer_name='cookie') challengers = [('form', form_plugin)] auth = MongoAuthenticatorPlugin() authenticators = [('mongoauth', auth)] cookie = AuthTktCookiePlugin(cookie_secret, cookie_name) identifiers = [('cookie', cookie), ('form', form_plugin)] provider = MongoUserMDPlugin() mdproviders = [('mongoprovider', provider)] from repoze.who.classifiers import default_request_classifier from repoze.who.classifiers import default_challenge_decider log_stream = None app = PluggableAuthenticationMiddleware(app, identifiers, authenticators, challengers, mdproviders, default_request_classifier, default_challenge_decider) return app base_config = MyAppConfig() base_config.renderers = [] base_config.package = my_app #Set the default renderer base_config.default_renderer = 'genshi' base_config.renderers.append('genshi') base_config.renderers.append('json') #Configure the base SQLALchemy Setup base_config.use_sqlalchemy = False base_config.model = my_app.model
ACR gets Slice Templates
Monday, March 1st, 2010 | Web | Comments
ACR is a flexible and quite powerfull CMS library, but users have to learn Slices and Slicegroups to be able to insert more advanced content like photo galleries, videos and a news section. To simplify this process we created “Slice Templates”.
Slice Templates are actually a set of common ways to use slices to insert more advanced content. The first two slice templates implemented are:
- Youtube Videos which make easy to insert a youtube video inside an ACR Page.
- Photo Gallery which makes easy to insert a photo gallery with cool effects inside an ACR Page.
Next template to come will be a news section, for now you can upgrade your libACR and start using the templates feature or take a look at the screenshots of the currently implemented templates
Search
Archives
- January 2012
- November 2011
- October 2011
- July 2011
- June 2011
- May 2011
- April 2011
- March 2011
- February 2011
- January 2011
- December 2010
- November 2010
- September 2010
- August 2010
- July 2010
- May 2010
- April 2010
- March 2010
- February 2010
- January 2010
- December 2009
- November 2009
- October 2009
- August 2009
- July 2009
- June 2009
- May 2009
- April 2009
- March 2009
- February 2009
- December 2008
- November 2008
- October 2008
- August 2008



