unicolet.org

I have just completed an upgrade from OpenNMS 1.8.11 to the latest and greatest 1.10. The upgrade in itself is easy and the guides on the OpenNMS wiki will serve you well. Instead in this post I'll describe a couple of other changes that I made which improved very much the overall performance and responsiveness of the system. One is the upgrade from PostgreSQL  8.4 (which came with CentOS) to 9.1 + tuning. The other is switching from apache to nginx. Upgrading postgres is mostly a matter of taking a backup,  pulling in the right repo , running  yum install  and finally importing the database. Tuning postgres I left opennms running on PostgreSQL 9.1 for a while and then I went checking how well postgres was doing. Postgres 9 already performs significantly better that its 8.x predecessors, but I wanted to do better than out-of-the-box. As the postgres user I logged in into the opennms database to install a utility that will help me estimate how muc...

KVO is one of Sproutcore coolest features. At the core of it there is the RunLoop in which events and notifications are efficiently processed, dispatched to their destination(s) and views modify the DOM to reflect the new application state. Sproutcore developers are consequently told not to manipulate directly the DOM. When out-of-band events, like an ajax call returning from a third-party library or a WebSocket receiving a new message happen it is possible to trigger the RunLoop manually by calling  SC.RunLoop.begin() ... SC.RunLoop.end() . Sometimes though it is not only necessary, but recommended, to bypass the RunLoop and manipulate directly the DOM to either provide UI refreshes as fast as the browser allows or avoid the expensive computations implicated in the RunLoop. These concepts were incidentally discussed on IRC just when I needed to implement a progress bar to provide feedback on the loading state of a particularly slow datasource and I am writing them dow...

It happened to me recently that I needed OpenLayers (version 2.11) to parse a GetCapabilities response from Geoserver to present the user a list of layers to pick from. The capabilites request is made through a Sproutcore request, which is basically a jQuery ajax object in disguise. This of course works beautifully in every browser with the notable execption of ... IE (IE9 included). The cause is that IE will helpfully xml-parse a response whose content-type is text/xml , but will refuse to parse a document whose content-type is  application/vnd.ogc.request+xml . To add comedy to the drama the responseXML attribute of response is not null, as one would expect, but is instead set to reference an empty dom. The workaround is to put a giant browser-sniffing if in your javascript to handle IE differently. The code is like the following and please note that it is Sproutcore code, so browser sniffing and other amenities are peculiar to Sproutcore: // God mess IE if(SC.$.bro...

This is just a note-to-self kind of post. Official instruction, available here , do not explain how to setup the development version (which currently is 1.7.1.beta). This is the full list of command (ran and tested on Ubuntu Lucid Lynx 10.04), just need to type the occasional yes . For those cases when bundle fails (happened to me) to create the bin directory with the beta version of sc-* tools just type the longer variant: bundle exec sc-server

I'm writing this post just to share an interesting conversation that popped up on #sproutcore this week. Sproutcore is a huge framework and there is no amount of documentation that will explain it all, that's why I suggest you hang out in IRC . Even if you don't have questions to ask you'll learn a lot from others'. In this case the question was: how do I make my custom view update whenever one of the content object properties change? The perfect case for this, I think, would be if we were writing a custom  SC.ListItemView . In my case, prior to discovering SC.ContentDisplay I used the following configuration in a custom ListItemView: Maps.OpenLayersLayer = SC.View.extend({ content: null, displayProperties: ["content", "content.order", "content.visible", "content.opacity", "content.cql_filter"], [...] While it works (it was, I quote, described as funny  though) this is the recommended way to do it: Maps.O...

Using expendable virtual machines for testing purposes is hardly a new concept. Now  pretty-please-with-sugar-on-top would be automating the whole create-provision-test-destroy cycle. Enter  Vagrant . Vagrant is a tool for building and distributing virtualized development environments. It allows automated creation, provisioning (with Puppet, Chef or shell scripts) and tear down. Installing Vagrant on Ubuntu 10.04 LTS (which I'm still running) requires the following: a 4.1.x VirtualBox distribution (the 3.1 bundled with Ubuntu will not work) a recent version of ruby (you guessed it: the 1.8.x bundled with Ubuntu will not work) To install VirtualBox add the following repo to /etc/apt/sources.list: deb http://download.virtualbox.org/virtualbox/debian lucid contrib non-free then issue the usual apt-get update and install with (remove stock packages first): sudo apt-get update sudo apt-get remove virtualbox-ose virtualbox-ose-dkms sudo apt-get install ...

Sometimes I endeavor in complicate refactorings and, instead of commiting to a branch when at least a part is working, I keep editing until I lose control of the source and wished that I could go back to that previous working situation. The solution is, luckily, not too difficult: save the edits to a patch svn diff > great_mother_of_all_patches.patch revert the project to the last commit: svn revert -R . apply the patch edit by edit (I know this is tedious) using the eclipse " Team Synchronizing " perspective, then click on the Synchronize button and choose the Synchronize with patch option Disclaimer : I don't use the Eclipse subversion plugin because, on Linux at least, it frequently crashes Eclipse.

OTRS is a great ticket-tracking/helpdesk software. It even has cool statistics built in, but sometimes the PHB goes crazy and asks for impossibily detailed statistics. For those situations when the stats module cannot help us there a couple of (Open Source) tricks up our sleeve: install the Eclipse Birt web reporting application (we will need java and tomcat for that) create a custom report using the report designer deploy the report on the server and send the link to the boss To make sure the SQL queries used in the report are the same used by OTRS we can temporarily have OTRS log all queries (remember to disable logging after you're done or your log files will grow out of control) by editing the Kernel/System/DB.pm file as documented here : # 0=off; 1=updates; 2=+selects; 3=+Connects; # $Self->{Debug} = $Param{Debug} || 0; # leave the original around for later $Self->{Debug} = 2;

Grails has great support for storing binary data into any database, Postgres included. It is usually simple to use but today it gave me problems. I have an application originally developed on pg 8.4 and grails 1.3.6 which stores images in a database table and used to work fine until I upgraded the database to 9.1. It turns out that with version 9 Postgres by default will return data to the client using a new hex format , instead of the escape format used in versions < 9. This, of course, confuses the client which returns garbage to the browser, hence the corrupted image. The solution is, luckily, quite simple: just tell postgres to revert to the old behaviour. It seems that this behaviour can be tuned per database, so you could have one database using the new format and another the old one: ALTER DATABASE dbname SET bytea_output='escape'; I should point out that I could also have updated the jdbc driver (now I'm using postgresql-8.3-603.jdbc3.jar ), but that s...

Bash shell scripting is one the things that I miss more on the times I work on Windows. Even as good as shell scripting is, sometimes I wish it was easy to keep track of state across script executions, for instance when a script executes a rsync at short intervals. In that case I don't care if one particula rsync fails, but I definitely care if it fails, say, ten times in a row or for more than a day. To do this I need some kind of way to keep state and record each run exit status. The simplest approach that I could think of is to use SQLite . Copied straight from the SQLite site:  SQLite is a software library that implements a self-contained, serverless, zero-configuration, transactional SQL database engine. SQLite is the most widely deployed SQL database engine in the world. The source code for SQLite is in the public domain. As I said, in this example I'll show how to track the status of a series of rsync operations, all run from a central node and pushed to remote nodes...

Today I was asked to quickly put together a script to replicate files/directories from a central server to a number of XP clients. This would be dead simple if it was not for Windows. The customer originally thought of robocopy/xcopy but I dont' really trust them: error reporting is kinda difficult to read and copying over cifs is slow and unreliable. So I set out to build it on top of rsync , but to do that I had to find a way to get rsync installed as a service on all the clients. Turns out it's quite easy. I started from a clean cygwin install to which I added rsync and cygrunsrv. The latter is the native cygwin tool that can be use to run any tradition *nix daemon as a native Windows service. Even a basic install like this will be in the order of 50MB or so which is too large for quick deployments. I then started trimming all the stuff that was not essential to the rsync service. At the end of the trimming process I was left with a mere 10MB, which after being compres...

Sometimes it is convenient to make certain topological operations available in a web-based gis. For a gis that makes heavy use of Javascript (like OpenLayers-based ones) it might be worth looking at jSTS , a Javascript port of JTS. For all the rest and for those who don't want to load yet another library in the browser you can always write a Grails controller that encapsulates common JTS operations like buffer, intersection, union, etc. Assuming you are familiar with Grails the steps are as follows: drop the jts jar in the lib directory create a controller and define the relevant methods define a url mapping to prettify the calls Step 1 is trivial, so we'll go straight to 2. Create a controller and call it JtsController , then open the source file and paste this code: import com.vividsolutions.jts.geom.* import com.vividsolutions.jts.io.* import com.vividsolutions.jts.operation.overlay.snap.* import grails.converters.JSON import grails.plugins.springsecuri...

Mapserver  is probably the most popular Open Source web mapping platform (even though Geoserver is getting much of the limelight nowadays). One of the advantages that Mapserver has against Geoserver is that its configuration is pretty easy because it consists of a flat text file (Geoserver instead uses a xml-backed repository). Because of this kind of repository managing a Geoserver farm becomes complicated when changes have to be replicated across all hosts and the services restarted to pick up the changes. To address this issue there have been recent efforts to build a multi-master replication mechanism plugged into Geoserver. While this is pretty cool (and it's done by an Italian company of which of course I'm proud of, being an Italian myself) I think it's even cooler to see how easy it is to manage Mapserver configuration files in a similar cluster environment. The Mapserver setup is as follows: a cluster of mapserver servers serving WMS through a number of ma...

In one app I needed to allow the user to CTRL-click on an OpenLayers map to popup a tool menu. Since all the rest of the app is Sproutcore based I also wanted the menu to have the same look&feel. Sproutcore already has a menu view which is called SC.MenuPane . Unfortunately SC.MenuPane can be positioned only relatively to another SC.View, but what if I want to position the menu exactly where I clicked on the map? The solution turns out to be pretty simple and requires us to override only one MenuPane method. The method is positionPane and can be overridden at definition time by adding it to the mixin as done below (gotta love javascript, huh?): Maps.openLayersController.menuPane: SC.MenuPane.create({ layout: {width: 120}, itemHeight: 25, items: [ { title: '_geocode'.loc(), icon: 'http://maps.gstatic.com/favicon.ico', action: "geocode" }, { title: '_streetview'.loc(), icon: 'icon-streetview-16', action: "streetview...

In some cases it is practical in an application to have a single global settings file where all configuration is stored. Most often this configuration file varies between a deployment and another, depending on customer requirements, licensed features, backend technology, etc. When developing an application with Sproutcore 1.x (1.6 is assumed) it is not immediately clear how to do it, not because it is not possible but just because documentation on Abbot , SC's build tool is, to use an euphemism, well, scarce. So what do we want to do exactly? We want to have a single config.js file that is included before any other application file and, most importantly, must not be minified and packed so that on-the-fly editing or inspection is still conveniently possible. The file should be kept small anyway so that packing and minifying it is not going to be necessary. To do this we need to create a new .js file and place it wherever we like in the application tree. A sensible choice wou...

I have an OpenNMS server (a PC really) sitting under a desk at a company that I don't hear from often, but I still get weekly backup mail-recaps. This week the recap brought bad news: postgres stopped accepting connections until the db is vacuumed to prevent transaction id wraparound. I have know of this issue of postgres for some time and planned accordingly: a cron job every month would run a full vacuum. Unfortunately in one month the db grew so much that vacuum never reached completion because the customer simply rebooted the server thinking that it was 'stuck'. This vacuum-stuck-reboot had gone on for months until today. As I knew from prior experience this was going to be a painful experience: a quick du on the database folder reported 60GB of data. Vacuum would probably take days. Luckily for me I had heard from the great Postgresql 9.0 high performance book that there is another tool in the toolbox called cluster . So I decided to execute this plan this ...

I personally like/use GrailsUI a lot and I'm sorry that it's been discontinued. Just today while I was upgrading an app from Grails 1.1 to 1.3.6 I found out that my previously working menubar is now spitting out ugly errors (like this one ). The problem lies in the unsafe use of instance variables by the tag library. Luckily for us the solution is pretty simple, just replace all references to items and group with a request scope variable like request.items and request.group. In my case I went a little further and tried to come up with a variable name that would not clsh with other variables actually used by me. My (working, yup) copy of the menu tag library is available at  https://gist.github.com/1358135 . To use it just copy it into your project's taglib folder and then build a menubar as you would do with GrailsUI, only using the g: prefix instead of gui: .

When using OpenLayers embedded in a Sproutcore app you will want to set the following option on the OpenLayers.Map constructor: fallThrough  : false to avoid double handling events (once in OpenLayers and once in Sproutcore). Map dragging should improve and feel much smoother. Although deprecated this wiki page explains very clearly Sproutcore event handling.

Dennis Ritchie died last week. While Steve Jobs had a great influence in my past years (I have only recently become an Apple customer/fan) my first experiences with Ritchie's (and Kernigan) go back  to my university days and are forever linked to my first baby steps with Slackware linux, fvwm and tcp-ip. I miss those days as anyone misses his/hers youth. So long Dennis and thanks for all the segfaults!

Warning : this post is a work in progress. Full source code for both Sproutcore and Grails  parts can be found at my github account  https://github.com/unicolet . All the Sproutcore examples that I've found so far only show how to fetch data from different backends, but there is unfortunately very little material on authentication. In my case I'm hooking up to a Java backend written in  Grails  with authentication being provided by by Spring Security . The grails spring-security-core plugin documentation already incorporates  Ajax logins so the server-side code is already in place. We only have to apply one small tweak to make it work with Sproutcore. Prerequisites: Grails, Sproutcore > 1.6 with Statecharts. The statechart in the figure illustrates the states and transitions we plan to handle. Authentication Let's start with authenticating the user. We will deal with reconnecting the session later. Assuming you have a basic Grails ...