Python 3.0

Python 3.0 changes have been defined “arbitrary and academic“, for no real use for real programmers. Before starting to talk about python changes, I would think about the meaning of “real programmers” in the sentence before.

How would we define a “real programmer”?

  • Are hackers “real programmers”?
  • Are professors of language theory “real programmers”?
  • Are experts in algorithm theory “real programmers”?
  • Are people who do coding for a living “real programmers”?

Probably people inside the first definition didn’t mind a lot about how Python 2.x did things and could live with it. They will just be angry because suddenly they tools and scripts will stop working. But those people can also go on with Python 2.x as they don’t tend to have complex software maintainment processes and would probably just be happy having a way to run it.

People inside the second and third definition will probably be happier as Python 3.0 clean up will satisfy their concept of elegance a bit more.

People inside the fourth definition will probably be angry thinking of how many hours they will have to spend porting their code to Python 3.0 as their code probably will have to live for the next 5 years as hundred of customers will have to use it.

This might be right, but I think that we have to go deeper inside the question to really understand the effect of those changes. I actually do code for a living, and inside the majority of our projects we try to refactor often to make long time maintaining of the code as easy as possible. As us, there will probably be hundreds of other companies who often perform refactoring, and all of us do it for one common goal: “Keeping the code as simple as possible to make it clear and obvious even after years”.

Usually this means things like:

  • converting strange behaviours to standard one inside the application domain
  • converting spread values to constants with a meaning
  • removing code duplicates to have only one unit of work
  • change function and classes to make their role as simple to understand as possible
  • Separate mixed code blocks in methods/classes with a clear scope
  • and things like that…

Uhm, wait for a second… isn’t this exactly what Python is doing for the 3.0 release?

  • They changed the print keyword to a standard function conforming it to the behaviour of any other function inside the language.
  • They removed some of the duplicated modules like urllib and urllib2
  • They took care of modules that did the same things trying to localize everything in only one clear place
  • They moved things inside more clear namespaces instead of having them all mixed in the global one
  • They removed old modules superseded by new one that still where available.
  • They renamed some things to make them conformed to python conventions.

In the end they actually forced us to perform refactoring of our own projects refactoring a piece of our projects: “the underlying library and language”. This is just the same thing that happens everyday with self written libraries that you reuse in multiple projects inside your company.

But if thousand of people are performing refactoring everyday on things from which their lives depend on, if Google is accepting Python 3.0 when most of their business uses Python (and they also hired Guido so they could probably influence Python 3.0 development a lot if they wanted) how can it be such a bad thing?

I’m starting to think that people blaming python for its changes are mostly angry because they have been forced to perform refactoring when they didn’t want to or just saw their software stop working. But actually you don’t have to change things, your software didn’t stop working and Python 2.x will be maintained for at least one year. You have the choice to do what you want.

As for me I’ll probably just thank the Python team for refactoring libraries and things that I use everyday without asking me to pay a thing. And I’ll just perform migration to 3.0 of my projects using 2to3 tool and something that I already perform everyday: code refactoring…

LSCube Flux 1.0 Released

After a few months of work and tests the LSCube project released the first component of the imminent rewrite for the ESOF 2010 event.

The component is Flux, it is the RTP Mixer/Stream manipulator of the project. It has born to replace the old felix tool which was the core tool to perform live streaming injecting packets inside the feng server. Now Flux for the first release has the same features of felix and aims to implement more complete and powerful packets manipulation like transcoding, overlaying and mixing.

Also Flux has a more clean and easy to understand architecture making possible for anyone to implement new IO Classes or Parser in an easy and clean manner.

PyHP Sessions support improvements

Good news for PyHP. Latest SVN version got a few bug fixes and a big session management improvement. Now session collision is handled in a more coherent way, it’s a major improvement and I suggest to everyone using it to upgrade to svn revision. Also a few fixes have been implemented to catch programming errors by the developer that caused PyHP to crash making it more stable.

Liskov Substitution Principle Reflection

I have found a quite interesting shakespearean article about Liskov Substitution Principle. It is mainly about C++, but being it the core of Object Oriented paradigm I think that it might be interesting for everyone.

Most of the think is nowadays resolved with mixins and Policy Based Design, but they run away a bit from pure Objects Oriented and so is still an interesting reflection

So if you are interested take a look at this  Blog Post

FlyPDF now has C# support

FlyPDF the multi-language PDF generation library we develop in collaboration with OS3 now has C# (.Net) support bindings. Also improvements have been made to the C bindings implementing callbacks to override default header and footer generation. Also the Write method for flowing text has been made available to all the language bindings including C++ and Python.

Also Fabio Rotondo from OS3 had a speech about FlyPDF at the Italian PyCon which had quite a big success.

PyHP now supports multiple headers

SVN revision 24 of PyHP now supports multiple headers!

This broke code that iterates over the old headers_in dictionary because now every value inside headers_in is a list instead of a string. On the other side code writing inside headers_out should continue to work because it is possible to insert both a string or a list inside headers_out dictionary to set only one header or multiple headers.

You can try it from here: http://pyhp.svn.sourceforge.net/viewvc/pyhp.tar.gz?view=tar

Remember to sync (or what the heck is that RAW C: partition on my Windows on mac?)

Our current computers use MBR to store the table of primary partitions. MBR for convention allows only 4 primary partitions and then uses Logical Partitions inside Extended Boot Records for the successive. Instead Apple computers use the new Extensible Firmware Interface with the GUID Partition Table to describe disk partitions permitting any number of primary partitions to be allocated.

MacOSX uses the GPT partition table while Windows expects to find the MBR one (Actually Windows for Itanium can boot from GPT, but that is not what you are usually installing on your desktop). Fortunately the EFI keeps a “fake” MBR table for backward compatibility and that is why we can install Windows on our MacIntel. But what happens when we change our GPT partitions from OSX/Linux which supports GPT? It happens that our MBR becames out of sync with the GPT.

Recently I had to install a triple boot with OSX+Win+Linux on my MacBookPro. This required 5 partitions (1 for EFI, 1 for OSX, 1 for Linux, 1 for Windows, 1 Linux Swap). After creating them from OSX I synchronized the MBR to GPT with refit tools to have the first four partition available also on MBR (this way I lost only the SWAP partition)  then I installed Windows XP. Everything went well but after booting, Windows ended up installing on D: and there was a C: partition resulting of RAW type. I just thought that obviously this was because there was the un-formatted Linux partition before the Windows one and restarted to install Linux.

Then I formatted my linux partition as EXT3 and installed Gentoo over it. Everything was fine until I had to restart windows to install the Service Pack 3 for XP. My wonderful RAW C: partition was still there… and I wasn’t able to install the service pack because SP3 installer wasn’t able to install on anything else then C: even if my Windows installation resulted on D: and C: didn’t result as a valid partition (off course… It’s my Linux partition!)

Windows should have known that it isn’t a valid partition for him and should hide it, that didn’t happen… Why? After thinking a bit I remembered that the Partition Type is also a field inside the MBR. When we format a partition the system specifies also the partition type inside the MBR to tell everyone that partition has been reserved by some OS. Having initialized my EXT3 partition from Linux had as a result that only the GPT was updated and the MBR still thought that my third partition was un-formatted.

After losing about an hour fighting with the Service Pack installer I just realized that I had to resynchronize the two tables… After doing that the RAW C: partition disappeared from Windows and my service pack installed correctly.

New Lesson: Ok, Microsoft could make a better installer and stop preventing their OS to install from something different from what they call “C:” but sometimes you have to think more before enraging with your OS 😀

PyHP improvements

New features in PyHP this week on the SVN version

  • it is now possible to set sessions timeout with PyHPSessionTimeout config variable
  • The Result object from the Database Layer is now iterable and will work as performing multiple .fetch() calls
  • pyhp.status is now exposed to return a status code different from HTTP_OK (200)
  • pyhp.content_type is now exposed to permit serving images and files different from text/html

Also some performance tests has been performed on mod_pyhp in production mode and the results are here available:

Time taken for tests:   1.419409 seconds
Complete requests:      500
Failed requests:        0
Write errors:           0
Non-2xx responses:      500 (hello_world.pyhp returns 500 on last pyhp)
Total transferred:      1558000 bytes
HTML transferred:       1344000 bytes
Requests per second:    352.26 [#/sec] (mean)
Time per request:       2.839 [ms] (mean)
Time per request:       2.839 [ms] (mean, across all concurrent requests)
Transfer rate:          1071.57 [Kbytes/sec] received

The results reveal to be quite good compared to other popular web development tools like WEBrick, mod_python and mod_php. Still they have a bunch of space for improvement to reach the “standard” ~600req/sec that more mature solutions obtain. Probably .pyhp files caching might be implemented to increase a lot performances.

I will provide a complete comparations to the other major web tools as soon as I have some time to set up a concrete test suite.

The misunderstood one

I’m a usual follower of C++ related blogs and newsgroups and still I’m impressed how much this language is misunderstood after 10 years of a precise standard. I’m actually starting to ponder if we are ready for C++0x while a lot of people yet misunderstand C++98 and C++03.

I often end up discussing with my colleagues about the fact that they prefer a less powerful, but more reliable and less obscure language like C. I know that C++ isn’t one of the easiest languages to master correctly and I also know that it has some problems about compilers often changing their behaviour and with slow templates compilation time. All things that C++0x is trying to make easier or faster with extern templates, move constructors, delegate constructors and type determination.

C++0x will be a quite complex and advanced language, which will give the programmer a lot of control over the compiler, but still after 10 years of existence of the standard C++ and formal STL libraries you can find posts like this which confuse good design practices and theories with runtime optimizations.

Having the stack container with the pop method returning void is exception neutrality not a runtime optimization. Strong exceptions safety means that the operation has either completed successfully or thrown an exception, leaving the program state exactly as it was before the operation started.

What would happen if we have a T pop() method returning the head after removing it? We might have the copy-constructor of T fail throwing an exception and find that after removing our head we also lost it forever. Having it just remove the item means that our code might only have success removing our item or fail to remove our item leaving the state exactly as it was before.

This is one of the first issues raised years ago, an issue discussed in depth since the article of Tom Cargil in 1994. An issues explained by Meyers and Sutter in their books and available in a lot of community blogs like the boost one. And still this isn’t clear even to long time C++ developers.

C++ is a multipurpose language, having all that freedom of movement requires more deep knowledge of the language itself. Shouldn’t we change how C++ is taught and more strictly enforce some practices before moving to a new standard? As a sporadic C++ teacher I often end up with this question…