Posts

SVN Revert (but only *some* changes)

Sometimes I endeavor in complicate refactorings and, instead of commiting to a branch when at least a part is working, I keep editing until I lose control of the source and wished that I could go back to that previous working situation. The solution is, luckily, not too difficult: save the edits to a patch svn diff > great_mother_of_all_patches.patch revert the project to the last commit: svn revert -R . apply the patch edit by edit (I know this is tedious) using the eclipse " Team Synchronizing " perspective, then click on the Synchronize button and choose the Synchronize with patch option Disclaimer : I don't use the Eclipse subversion plugin because, on Linux at least, it frequently crashes Eclipse.

OTRS-hacking: log all sql queries for use in external reports

OTRS is a great ticket-tracking/helpdesk software. It even has cool statistics built in, but sometimes the PHB goes crazy and asks for impossibily detailed statistics. For those situations when the stats module cannot help us there a couple of (Open Source) tricks up our sleeve: install the Eclipse Birt web reporting application (we will need java and tomcat for that) create a custom report using the report designer deploy the report on the server and send the link to the boss To make sure the SQL queries used in the report are the same used by OTRS we can temporarily have OTRS log all queries (remember to disable logging after you're done or your log files will grow out of control) by editing the Kernel/System/DB.pm file as documented here : # 0=off; 1=updates; 2=+selects; 3=+Connects; # $Self->{Debug} = $Param{Debug} || 0; # leave the original around for later $Self->{Debug} = 2;

Grails, blobs and postgres 'bytea_output'

Grails has great support for storing binary data into any database, Postgres included. It is usually simple to use but today it gave me problems. I have an application originally developed on pg 8.4 and grails 1.3.6 which stores images in a database table and used to work fine until I upgraded the database to 9.1. It turns out that with version 9 Postgres by default will return data to the client using a new hex format , instead of the escape format used in versions < 9. This, of course, confuses the client which returns garbage to the browser, hence the corrupted image. The solution is, luckily, quite simple: just tell postgres to revert to the old behaviour. It seems that this behaviour can be tuned per database, so you could have one database using the new format and another the old one: ALTER DATABASE dbname SET bytea_output='escape'; I should point out that I could also have updated the jdbc driver (now I'm using postgresql-8.3-603.jdbc3.jar ), but that s...

Using SQLite to keep state in shell scripts

Bash shell scripting is one the things that I miss more on the times I work on Windows. Even as good as shell scripting is, sometimes I wish it was easy to keep track of state across script executions, for instance when a script executes a rsync at short intervals. In that case I don't care if one particula rsync fails, but I definitely care if it fails, say, ten times in a row or for more than a day. To do this I need some kind of way to keep state and record each run exit status. The simplest approach that I could think of is to use SQLite . Copied straight from the SQLite site:  SQLite is a software library that implements a self-contained, serverless, zero-configuration, transactional SQL database engine. SQLite is the most widely deployed SQL database engine in the world. The source code for SQLite is in the public domain. As I said, in this example I'll show how to track the status of a series of rsync operations, all run from a central node and pushed to remote nodes...

Rsync (or any other cygwin daemon) as a Windows service

Today I was asked to quickly put together a script to replicate files/directories from a central server to a number of XP clients. This would be dead simple if it was not for Windows. The customer originally thought of robocopy/xcopy but I dont' really trust them: error reporting is kinda difficult to read and copying over cifs is slow and unreliable. So I set out to build it on top of rsync , but to do that I had to find a way to get rsync installed as a service on all the clients. Turns out it's quite easy. I started from a clean cygwin install to which I added rsync and cygrunsrv. The latter is the native cygwin tool that can be use to run any tradition *nix daemon as a native Windows service. Even a basic install like this will be in the order of 50MB or so which is too large for quick deployments. I then started trimming all the stuff that was not essential to the rsync service. At the end of the trimming process I was left with a mere 10MB, which after being compres...

JTS processing Grails servlet

Sometimes it is convenient to make certain topological operations available in a web-based gis. For a gis that makes heavy use of Javascript (like OpenLayers-based ones) it might be worth looking at jSTS , a Javascript port of JTS. For all the rest and for those who don't want to load yet another library in the browser you can always write a Grails controller that encapsulates common JTS operations like buffer, intersection, union, etc. Assuming you are familiar with Grails the steps are as follows: drop the jts jar in the lib directory create a controller and define the relevant methods define a url mapping to prettify the calls Step 1 is trivial, so we'll go straight to 2. Create a controller and call it JtsController , then open the source file and paste this code: import com.vividsolutions.jts.geom.* import com.vividsolutions.jts.io.* import com.vividsolutions.jts.operation.overlay.snap.* import grails.converters.JSON import grails.plugins.springsecuri...

Managing a Mapserver farm with Salt

Mapserver  is probably the most popular Open Source web mapping platform (even though Geoserver is getting much of the limelight nowadays). One of the advantages that Mapserver has against Geoserver is that its configuration is pretty easy because it consists of a flat text file (Geoserver instead uses a xml-backed repository). Because of this kind of repository managing a Geoserver farm becomes complicated when changes have to be replicated across all hosts and the services restarted to pick up the changes. To address this issue there have been recent efforts to build a multi-master replication mechanism plugged into Geoserver. While this is pretty cool (and it's done by an Italian company of which of course I'm proud of, being an Italian myself) I think it's even cooler to see how easy it is to manage Mapserver configuration files in a similar cluster environment. The Mapserver setup is as follows: a cluster of mapserver servers serving WMS through a number of ma...

SC.Menu dynamic positioning with OpenLayers

In one app I needed to allow the user to CTRL-click on an OpenLayers map to popup a tool menu. Since all the rest of the app is Sproutcore based I also wanted the menu to have the same look&feel. Sproutcore already has a menu view which is called SC.MenuPane . Unfortunately SC.MenuPane can be positioned only relatively to another SC.View, but what if I want to position the menu exactly where I clicked on the map? The solution turns out to be pretty simple and requires us to override only one MenuPane method. The method is positionPane and can be overridden at definition time by adding it to the mixin as done below (gotta love javascript, huh?): Maps.openLayersController.menuPane: SC.MenuPane.create({ layout: {width: 120}, itemHeight: 25, items: [ { title: '_geocode'.loc(), icon: 'http://maps.gstatic.com/favicon.ico', action: "geocode" }, { title: '_streetview'.loc(), icon: 'icon-streetview-16', action: "streetview...

Using a separate settings file with Abbot and Sproutcore 1.6

In some cases it is practical in an application to have a single global settings file where all configuration is stored. Most often this configuration file varies between a deployment and another, depending on customer requirements, licensed features, backend technology, etc. When developing an application with Sproutcore 1.x (1.6 is assumed) it is not immediately clear how to do it, not because it is not possible but just because documentation on Abbot , SC's build tool is, to use an euphemism, well, scarce. So what do we want to do exactly? We want to have a single config.js file that is included before any other application file and, most importantly, must not be minified and packed so that on-the-fly editing or inspection is still conveniently possible. The file should be kept small anyway so that packing and minifying it is not going to be necessary. To do this we need to create a new .js file and place it wherever we like in the application tree. A sensible choice wou...

Postgres maintenance: vacuum hell, cluster to the rescue

I have an OpenNMS server (a PC really) sitting under a desk at a company that I don't hear from often, but I still get weekly backup mail-recaps. This week the recap brought bad news: postgres stopped accepting connections until the db is vacuumed to prevent transaction id wraparound. I have know of this issue of postgres for some time and planned accordingly: a cron job every month would run a full vacuum. Unfortunately in one month the db grew so much that vacuum never reached completion because the customer simply rebooted the server thinking that it was 'stuck'. This vacuum-stuck-reboot had gone on for months until today. As I knew from prior experience this was going to be a painful experience: a quick du on the database folder reported 60GB of data. Vacuum would probably take days. Luckily for me I had heard from the great Postgresql 9.0 high performance book that there is another tool in the toolbox called cluster . So I decided to execute this plan this ...

Fixing GrailsUI infamous 'Cannot create a menubar within a menu'

I personally like/use GrailsUI a lot and I'm sorry that it's been discontinued. Just today while I was upgrading an app from Grails 1.1 to 1.3.6 I found out that my previously working menubar is now spitting out ugly errors (like this one ). The problem lies in the unsafe use of instance variables by the tag library. Luckily for us the solution is pretty simple, just replace all references to items and group with a request scope variable like request.items and request.group. In my case I went a little further and tried to come up with a variable name that would not clsh with other variables actually used by me. My (working, yup) copy of the menu tag library is available at  https://gist.github.com/1358135 . To use it just copy it into your project's taglib folder and then build a menubar as you would do with GrailsUI, only using the g: prefix instead of gui: .

Improve OpenLayers performance in a Sproutcore app

When using OpenLayers embedded in a Sproutcore app you will want to set the following option on the OpenLayers.Map constructor: fallThrough  : false to avoid double handling events (once in OpenLayers and once in Sproutcore). Map dragging should improve and feel much smoother. Although deprecated this wiki page explains very clearly Sproutcore event handling.

RIP Dennis Ritchie

Image
Dennis Ritchie died last week. While Steve Jobs had a great influence in my past years (I have only recently become an Apple customer/fan) my first experiences with Ritchie's (and Kernigan) go back  to my university days and are forever linked to my first baby steps with Slackware linux, fvwm and tcp-ip. I miss those days as anyone misses his/hers youth. So long Dennis and thanks for all the segfaults!

Grails ajax login from a Sproutcore app

Image
Warning : this post is a work in progress. Full source code for both Sproutcore and Grails  parts can be found at my github account  https://github.com/unicolet . All the Sproutcore examples that I've found so far only show how to fetch data from different backends, but there is unfortunately very little material on authentication. In my case I'm hooking up to a Java backend written in  Grails  with authentication being provided by by Spring Security . The grails spring-security-core plugin documentation already incorporates  Ajax logins so the server-side code is already in place. We only have to apply one small tweak to make it work with Sproutcore. Prerequisites: Grails, Sproutcore > 1.6 with Statecharts. The statechart in the figure illustrates the states and transitions we plan to handle. Authentication Let's start with authenticating the user. We will deal with reconnecting the session later. Assuming you have a basic Grails ...

Adding Google Web Fonts to a Sproutcore app

Yesterday I was playing with the Hubbub   Sproutcore app that is just plain beautiful (and useful, if anyone used it over here in stodgy Italy). Anyway I noticed that the beautiful texts on the home page are not images, but actual text with fonts. I did not know I had such great fonts on my computer and in fact I don't. Those fonts come from Google Web Fonts  and you can use them too, right away in any of your web apps or sites. Since I am working on a Sproutcore app that could use some beauty I decided to style the app name with this font. Now, if only I knew how to add the css link to the html generated by abbot... [/me is gtfw & source browsing for a good 30 mins...] The solution turns out to be pretty simple and elegant (as everything in SC world, wink wink). Open the Buildfile in your favourite editor and add a stylesheet_libs option to the project config as follows: config :all do |c| c[:javascript_libs] = ['http://maps.google.com/maps/api/js?v=...

Adding Sproutcore KVO to OpenLayers - More Automation - Part 3

I have showed in the previous post how to add KVO to OpenLayers objects and then connect these objects to inputs, etc. The approach was unfortunately troublesome because each OpenLayers object had to be programmatically augmented through a SC.mixin call. In this post I will show how the OpenLayers source can be patched to make the augmentation process built-in. The only modification that needs be done is in Class.js where the OpenLayers.Class function must be modified as follows: OpenLayers.Class = function() {     var len = arguments.length;     var P = arguments[0];     var F = arguments[len-1];     var C=null;     if(typeof F.initialize == "function") {         if (!F.__my_init) {             F.__my_init = F.initialize             F.initialize = function() {                       ...

Adding Sproutcore KVO to OpenLayers - Part 2

Image
You can also  Read Part 1   or   Read Part 3 In the previous post of this series I have showed how easily an OpenLayers object can be extended with Sproutcore KVO . The main driver behind integrating KVO into OL is that of beauty and simplicity. When KVO is available texfields, checkboxes, layers, features, you name it can all have their state transparently and continuosly synchronized without writing any code at all. The first post was more of a proof of concept. In this post instead I am going to show a more practical use and rewrite one the OpenLayers examples (from the dev examples). The original example shows how to create and add features to a map from javascript. With the help of KVO I am going to expand the original and also make the features editable. As we will see the changes will propagate immediately into the map. The full source is available on github . To run the example simply download/clone/fetch the repo and open index.html with a browser. The...

Adding Sproutcore KVO to OpenLayers - Part 1

The truth is that after you start using Sproutcore KVO you become addicted to its beauty and simplicity. Where in other frameworks you spent most time keeping each part of the app in sync, in Sproutcore you get it for free. Sure Sproutcore isn't perfect and is not for all, but KVO is simply automagical! In this post I will show how easy it is to implement KVO on top of OpenLayers and take advantage of KVO to bind features, wfs queries and UI controls to otherwise stubborn Openlayers objects. In this example we will create a Vector layer, add KVo to it and then verify KVO bindings by manually adding an observer. I have run all this code from Google Chrome console and you can do it too. First thing to do is to create a scratch SC project: sc-init kvool then let's edit the Buildfile and add the following lines at the end so that the OpenLayers library is loaded with the app: config :all do |c| c[:javascript_libs] = ['http://www.openlayers.org/api/OpenLaye...

Automated conversion of MS-Access database to Postgres

Thanks to mdbtools it is now a little bit easier to migrate MS-Access databases to a newtork database like Postgres. This is a script that will bulk convert all the tables in a MS-Access database into a set of ready-to-use sql scripts (one per table).

Simple VCB: a tool For Vmware Consolidated Backup

I have written and would like to share with the world a groovy script that can be used to drive VCB backups without additional third party software. One of the most common gripes with VCB is that it doesn't integrate naturally into the VMWare management solution. Of course one can always buy a license for the integration module of his backup software, but that costs additional money. Some people then resort to script VCB so that additional licenses are not necessary but that, of course, requires scripting skills. Which also means reinventing the wheel just another time. This has happened to me too recently and I decided that ok, I would reinvent the damn wheel, but this time I would also make sure that the blueprint goes public! So I wrote this groovy script which I creatively named Simple VCB which does some nice things like: handles more than one esx, vcenter or vsphere server so that you don't have to split your configuration and schedules in more than one scrip...