unicolet.org

Subtitle: broken by impress.js I've been using Linux as the primary OS on all my laptops since a loooong time ago and until recently I always chose computers fitted with nvidia graphics over any other brand because their drivers probably have the best quality and performance. My last (and current) laptop, an HP ProBook, came with ATI graphics, but since in the last years ATI seemed to have caught up, I decided to give it a try. For over two yeas the laptop performed extremely well with Ubuntu 10.04 LTS as everything worked right out of the box with no customization whatsoever. Suspend/resume worked, performance was great and boot time pretty good too (for a conventional hard drive at 7200rpm, at least). The problems came with the upgrade to Ubuntu 12.04.

It's that time of the year again and, since it appears that the Maya spared us, I want to share with you a couple of gists that I came up with recently that could be generally useful. Btw, there are lots of other gists on my gist.github.com profile, check them out. If you find these script useful star them on github, drop me a comment or just share them. Once again, Merry Christmas everyone! The first one is for Java people and is a HttpServletRequestWrapper that supports: injection of the principal: for those cases when you use trust authentication and you are rolling your own SSO solution and/or you need to integrate with an existing SSO solution (I used it with for CAS ) supports reading of the InputStream multiple times. We all know that in a POST the request input stream can only be read once, so this will definitely help you if you need to access the post body or a request parameter in a Filter and make sure the upstream servlets/filters still work

To improve the quality of both my sleep and Mappu development I have started using Selenium IDE for automated testing of Mappu's UI. It was kind of hard at the beginning, being the UI based on Sproutcore which has the annoying habit of changing the controls ids with every page load, but after a while I was able to get it rolling quite nicely. Then I hit a major roadblock: the OpenLayers-based map control wouldn't react to clicks, mousedown, mouseup, fireEvent or anything else I threw at it.

Grails has had a liquibase plugin for a long time (note: I am talking about the old liquibase plugin based on 1.9). One functionality that (I feel) it's missing is a proper Bootstrap hook so that database changes can be deployed right when the application starts. The newest liquibase plugin called  grails database migration  has it built in.

In enterprise environments VMware vSphere is the most common virtualization choice. Hence when it comes to provisioning most rely on vm templates to accelerate and standardize the creation of new virtual machines. While templates work well for both Windows and Linux vms it still leaves a lot out and administrators may have additional work that has to be carried out manually (like configure networking, optionally move the vm in the right network/resource group, create application accounts, set up IIS, tomcat, java, etc, etc). True, most of this post-bootstrap setup can be taken care of with Puppet, Chef or, in my case, Saltstack , but what I wanted was a push-button vm provisioning that does all of the following automagically: create a new vm (only Linux vms are supported as of now) assign a pre-determined static ip address/name move vm into appropriate network configure it with Saltstack register the vm into out monitoring system (currently OpenNMS )

Disclaimer: this is just a shameless post to get myself a place at the great Rome event ;-). Oh well, this does not mean this post is not interesting to read. When I went to the Extreme Programming conference in Alghero (Sardinia) in 2001 I was consulting mostly as a Systems Administrator. So I felt a little bit like a fish out of water and actually one of the participants asked me: do you think there are aspects of XP that can be applied to systems administration? I think I said yes, but at that time it was kind of hard for me to find points of contact between the two. Maybe unit testing could be associated with putting a pervasive monitoring in place so that when I refactored a configuration I would know if it worked before clients did. Or coding standards could be associated with using automated installers for deploying servers, but what about keeping the configuration in sync after, when the systems went into production? And what about the rest of the rules? Last bu...

See all my Salt-related posts Welcome back! In our quest to simplify the configuration and automate our systems we have  installed  Salt on all our servers and then moved on to some  basic state management . We want of course to do more sophisticated stuff with salt and we'll get to that too. But first we want to make sure that all minions are aligned to the same salt version (the latest in this case). To do that we will add another state to our configuration which we will call (very much unimaginatively) salt.sls. The content is below: salt-minion: pkg: - latest service: - running - watch: - pkg: salt-minion This instructs minions to upgrade the salt-minion package on the node and, if upgraded, restart the service. To activate this state we'll edit the top.sls state file as follows: base: '*': - ntp - salt We are now ready to apply the changes. Let's start with a guinea-pig minion: [prompt]# salt 'expendab...

Welcome to the third episode of the series ! In the previous posts we have installed salt on CentOS machines and then moved on with a basic state configuration (we will cover more in the coming postst). Now it's time to handle those pesky SLES hosts for which there are no pre built binaries. Therefore we'll have to install salt using pip . I'll cover SLES11 in this post as that's the only variant I have. Hopefully other versions should require only minor changes. Note: active subscription to Novell update service is required as the following packages can only be found on SLES 11 SDK (it's an iso, and a large one, so if you don't have it around start downloading it before you start):  python-devel libopenssl-devel zlib-devel swig Installation Add the SDK iso in the Software Management sources. Then, as root, run the following commands (answer yes when required): zypper in gcc-c++ python-devel libopenssl-devel zlib-devel swig zypper -p http://downlo...

I was recently asked to recover a mirth instance whose embedded database had grown to fill all available space so this is just a note-to-self kind of post. Btw: the recovery, depending on db size and disk speed, is going to take long. The problem A 1.8 Mirth Connect instance was started, then forgotten (well neglected, actually). The user also forgot to setup pruning so the messages filled the embedded Derby database until it grew to fill all the available space on the disk. The SO is linux. The solution First of all: free some disk space so that the database can be started in embedded mode from the cli. You can also copy the whole mirth install to another server if you cannot free space. Depending on db size you will need a corresponding amount of space: in my case a 5GB db required around 2GB to start, process logs and then store the temp files during shrinking. Then open a shell as the user that mirth runs as (you're not running it as root, are you?) and cd in...

After part 1  of this series I had Salt running properly on all minions. It's now time to get some work done with it. We will start with something simple like making sure that ntp is installed and running on all minions. In order to to do that we will use the Salt  states enforcement feature. The default salt states configuration requires that: state definitions be kept in  /srv/salt the default state be named top.sls We will probably need to create both the directory and the files, which we can do with the following command (check that you are not overwriting your own state, needs to be done on the master only!): mkdir -p /srv/salt cat <<EOF >/srv/salt/top.sls base: '*': - ntp EOF What this state definition means is that the base state requires all nodes (as selected by '*') to apply the ntp state. Since we have not yet defined an ntp state we are going to do it right away: cat <<EOF >/srv/salt/ntp.sls ntp: pkg: - ins...

This post is the first in a series documenting the deployment of Salt on a small network ( ~ 100 hosts, initially targeting only linux-based ones which account for roughly half of it). Due to the low number of hosts I have gone for a single master layout. The linux hosts are for the greatest part running Centos 5.[4,5] in both x86 and x64 favors, and just a couple running SLES. Installing salt master The easiest way to install salt on Centos is to pull in the epel repository  : rpm -Uvh http://download.fedoraproject.org/pub/epel/5/i386/epel-release-5-4.noarch.rpm then install salt with yum: yum install -y salt-master Since minions by default will attempt to connect to the salt master by resolving an host named  salt  I configured a salt cname record for the salt master host in the dns server. At this point the master can be started with: /etc/init.d/salt-master start Note : I don't have firewall or SELinux enabled. In particular SE...

One liner shell command to find and rename to lowercase all SHP, DBF, and SHX file extensions starting from current directory: EXTENSIONS="SHP SHX DBF" for E in $EXTENSIONS ; do export E; export e=`echo $E | tr '[:upper:]' '[:lower:]'`; for f in `find . -name "*.$E"` ; do mv $f `echo $f | sed "s/$E/$e/"`; done ; done Useful when you get your shapefiles over from Windows and mapserver won't cooperate.

When developing applications for the cloud everybody knows (or should know) that a host, network or disk (in short any resource) failure is not an exceptional event but a rather common one. A resource failure becomes a 'normal', common event in the application lifecycle like the occasional bug. The Amazon approach is that some services (like databases) come with a certain degree of resiliency built-in while others (i.e. EC2 instances) are expected to fail relatively frequently and it is left to the developer to install backup, redundancy and availability countermeasures. My understanding is that other providers, like Rackspace , have instead a more traditional approach and will automatically restart failed virtual servers in case of host failure. If the failed cloud server image cannot be recovered then it will be bootstrapped from the most recent backup. This means that, depending on the requirements, one could move a traditional application to the cloud without having to...

QNAP devices have snmp support out-of-the-box, unfortunately the agent they ship with is almost unusable. At first it seems it supports lots of cool features (like fans, temperature, smart, etc) but if you take a little time to dig deeper you will notice that almost all key entries are, what?!, octetString s. Capacity reported as a String, QNAP what were you thinking? So good luck estimating disk usage when it is reported as a string: '1.8TB'  (I quote exactly as it is shown by mibbrowser) . Without wasting any more of your (and mine) time let's fix that by installing the Optware QPKG and then installing net-snmp from the ipkg web console. The whole process is traightforward, just make sure to have a recent firmware: log in to the QNAP admin web interface open Applications servers and then select QPKG Center from the Available tab install Optware and the enable it from the Installed tab now access the Optware web interface and search for net-snmp, then click th...

Now, I'm not a fan of Nagios and I always recommend OpenNMS over Nagios, but when a client is fixated with Nagios I take a deep breath and get work done with it too. Now it just happened that I couldn't convince a customer to use OpenNMS so I decided that if I really had to use Nagios I would do it in a way as innovative as possibile. The first phase in this kind of projects is usually gathering requirements, that is hosts/appliances to be monitored. So I opened up a Google Document spreadsheet and started typing. At a certain point it hit me, what if I could make this doc the source for all configuration and just be done with it? My spider sense were tingling and I knew I had just found a way to make a dull project an interesting and blog-worthy one. I created a spreadsheet like the one in the picture with only 4 columns: ip, name/description, location, groups. After that I shared the spreadsheet as csv and grabbed the url. From the shell I could now f...

In some applications it is useful, and in some case required, that the application audits some or all database operations: for instance to track when a user updates, deletes or inserts a record. In Grails this is often handled by a plugin, like the Audit Logging plugin, which hooks into the Hibernate events and then logs the details to an audit table, including a timestamp, the user (if available) who initiated the operation and the type of operation. Unfortunately the Audit Logging plugin suffers from some issues  and if you, like me, have to ship today then you can only roll your own solution. In this post I will lay out a solution which is simple, efficient and should work in most scenarios. The general requirements are: only one datasource the underlying dbms must have triggers and some kind of session variables (most do, but I'll focus on Oracle) quartz jobs, background operations and other activities not necessarily initiated by an http request must properly initi...

Bright style after changing the background color. More from my adventures in TileDrawer : the bright style comes with the wrong color for the map background (blue?), so I decided to replace it with a more sober shade of pale brown like most slippy maps seem to use. The change is a one liner: go to /usr/local/tiledrawer/gunicorn , edit the second line of style.xml and change the bgcolor attribute to whatever you like. I used rgb(241,238,232). Diff: Other TileDrawer posts .

I have been playing with TileDrawer recently. After a first rapid successful test on a throw-away EC2 instance I decided to deploy one on a vm for intranet use. The installation process is pretty straightforward, just run the script copied from the TileDrawer page as root. FYI make sure you have installed curl and python-cssutils before launching the script. After the script completed I started browsing the map and noticed that tiles took a looong time to render. A look at top from the server console showed that postgres was hogging the cpu. Memory was fine with no signs of swapping (it is a 1GB instance which I promptly upgraded to 2, running on server class hardware with Xeon CPU). Even after the memory upgrade tile rendering was so slow that the browser would sometime give up and show a white tile. I decided to look into it a little further and started by using the technique I have already described in another article . The database cache looked fine though as mo...

Whenever in a Sproutcore app a controller or a state needs to reference a view which is deeply nested in the ui tree there is a good case to use SC.outlet . The outlet creates a layer of indirection between the view and the other layers leaving to the view the responsibility to provide shortcuts to its inner objects, decoupling the layout from the application logic. An example of SC.outlet usage is shown in the todos application described in the Getting Started guides. Without outlet: ... TodosThree.mainPage = SC.Page.design({ // conventional design // https://github.com/sproutcore/getting-started/blob/master/apps/todos_three/resources/main_page.js#L4 ... TodosThree.SHOWING_APP = SC.State.design({ enterState: function() { TodosThree.mainPage.get('mainPane').append(); TodosThree.mainPage.mainPane.newTodoField.field.becomeFirstResponder(); }, Notice how the view design trickles down to the state layer (a part of the controller layer, which should be pre...

Make no mistake: Sproutcore IS a huge framework. Also not all of it is documented equally well and that's the reason why a casual observer or someone picking it up for the first time might find it difficult to perceive its full power (and extent). My first acquaintance with Sproutcore goes back to fall 2009 when a series of articles began to appear on the web about this new framework created at Apple by Charles Jolley. Since then a lot of things have changed and it's not always been clear what was driving those changes. This post is an attempt to write down and share my experiences and the resources that I used in learning Sproutcore in the hope that they may be useful to others. A little bit of history First of all a little bit of history: Sproutcore was started by Charles Jolley while working on the MobileMe platform at Apple. From there the framework spun off into an Open Source MIT licensed version (originally hosted on GitHub under the sproutit account). Tha...

Most Sproutcore developers working on Windows are used to the installer, which is also the default choice on the download page. Since lately the installer seems to be a little unreliable I decided to try the manual way and screenshot the whole process so that it might be of help to others. The first thing to do is install Ruby for Windows, which can be conveniently fetched from  http://rubyinstaller.org/ . I recommend you pick version 1.9.2 which, at the moment, I believe is the version used by most of the Sproutcore developers. Installing is a breeze, just remember to tick the 'Add Ruby to your path'  box so that you don't have to do it yourself later. If you have other ruby versions installed and you DO care about which version is selected by default than it's up to you whether or not to tick the box. If unsure just tick the box. When the installer is done open a command prompt (just type cmd in the run box on the Start menu and select the black icon ...

As much as I like to delve in cutting edge tech, to make a living I also provide systems administration services on old-school environments, where ahem Windows rules. One problem that's been biting me (daily!) for some time now is that on a Win2k3 Storage Server Event ID 333 will often surface, requiring a reboot. After checking all software, disabling antivirus, analyzing memory pools and all the usual (and unusual) stuff it turned out that the cause of the problem was the NFS server. I moved all shares to an EMC NAS, modified mount points, disabled the NFS service and the issue has been gone for the last two weeks.

I have just completed an upgrade from OpenNMS 1.8.11 to the latest and greatest 1.10. The upgrade in itself is easy and the guides on the OpenNMS wiki will serve you well. Instead in this post I'll describe a couple of other changes that I made which improved very much the overall performance and responsiveness of the system. One is the upgrade from PostgreSQL  8.4 (which came with CentOS) to 9.1 + tuning. The other is switching from apache to nginx. Upgrading postgres is mostly a matter of taking a backup,  pulling in the right repo , running  yum install  and finally importing the database. Tuning postgres I left opennms running on PostgreSQL 9.1 for a while and then I went checking how well postgres was doing. Postgres 9 already performs significantly better that its 8.x predecessors, but I wanted to do better than out-of-the-box. As the postgres user I logged in into the opennms database to install a utility that will help me estimate how muc...

KVO is one of Sproutcore coolest features. At the core of it there is the RunLoop in which events and notifications are efficiently processed, dispatched to their destination(s) and views modify the DOM to reflect the new application state. Sproutcore developers are consequently told not to manipulate directly the DOM. When out-of-band events, like an ajax call returning from a third-party library or a WebSocket receiving a new message happen it is possible to trigger the RunLoop manually by calling  SC.RunLoop.begin() ... SC.RunLoop.end() . Sometimes though it is not only necessary, but recommended, to bypass the RunLoop and manipulate directly the DOM to either provide UI refreshes as fast as the browser allows or avoid the expensive computations implicated in the RunLoop. These concepts were incidentally discussed on IRC just when I needed to implement a progress bar to provide feedback on the loading state of a particularly slow datasource and I am writing them dow...

It happened to me recently that I needed OpenLayers (version 2.11) to parse a GetCapabilities response from Geoserver to present the user a list of layers to pick from. The capabilites request is made through a Sproutcore request, which is basically a jQuery ajax object in disguise. This of course works beautifully in every browser with the notable execption of ... IE (IE9 included). The cause is that IE will helpfully xml-parse a response whose content-type is text/xml , but will refuse to parse a document whose content-type is  application/vnd.ogc.request+xml . To add comedy to the drama the responseXML attribute of response is not null, as one would expect, but is instead set to reference an empty dom. The workaround is to put a giant browser-sniffing if in your javascript to handle IE differently. The code is like the following and please note that it is Sproutcore code, so browser sniffing and other amenities are peculiar to Sproutcore: // God mess IE if(SC.$.bro...

This is just a note-to-self kind of post. Official instruction, available here , do not explain how to setup the development version (which currently is 1.7.1.beta). This is the full list of command (ran and tested on Ubuntu Lucid Lynx 10.04), just need to type the occasional yes . For those cases when bundle fails (happened to me) to create the bin directory with the beta version of sc-* tools just type the longer variant: bundle exec sc-server

I'm writing this post just to share an interesting conversation that popped up on #sproutcore this week. Sproutcore is a huge framework and there is no amount of documentation that will explain it all, that's why I suggest you hang out in IRC . Even if you don't have questions to ask you'll learn a lot from others'. In this case the question was: how do I make my custom view update whenever one of the content object properties change? The perfect case for this, I think, would be if we were writing a custom  SC.ListItemView . In my case, prior to discovering SC.ContentDisplay I used the following configuration in a custom ListItemView: Maps.OpenLayersLayer = SC.View.extend({ content: null, displayProperties: ["content", "content.order", "content.visible", "content.opacity", "content.cql_filter"], [...] While it works (it was, I quote, described as funny  though) this is the recommended way to do it: Maps.O...

Using expendable virtual machines for testing purposes is hardly a new concept. Now  pretty-please-with-sugar-on-top would be automating the whole create-provision-test-destroy cycle. Enter  Vagrant . Vagrant is a tool for building and distributing virtualized development environments. It allows automated creation, provisioning (with Puppet, Chef or shell scripts) and tear down. Installing Vagrant on Ubuntu 10.04 LTS (which I'm still running) requires the following: a 4.1.x VirtualBox distribution (the 3.1 bundled with Ubuntu will not work) a recent version of ruby (you guessed it: the 1.8.x bundled with Ubuntu will not work) To install VirtualBox add the following repo to /etc/apt/sources.list: deb http://download.virtualbox.org/virtualbox/debian lucid contrib non-free then issue the usual apt-get update and install with (remove stock packages first): sudo apt-get update sudo apt-get remove virtualbox-ose virtualbox-ose-dkms sudo apt-get install ...

Sometimes I endeavor in complicate refactorings and, instead of commiting to a branch when at least a part is working, I keep editing until I lose control of the source and wished that I could go back to that previous working situation. The solution is, luckily, not too difficult: save the edits to a patch svn diff > great_mother_of_all_patches.patch revert the project to the last commit: svn revert -R . apply the patch edit by edit (I know this is tedious) using the eclipse " Team Synchronizing " perspective, then click on the Synchronize button and choose the Synchronize with patch option Disclaimer : I don't use the Eclipse subversion plugin because, on Linux at least, it frequently crashes Eclipse.

OTRS is a great ticket-tracking/helpdesk software. It even has cool statistics built in, but sometimes the PHB goes crazy and asks for impossibily detailed statistics. For those situations when the stats module cannot help us there a couple of (Open Source) tricks up our sleeve: install the Eclipse Birt web reporting application (we will need java and tomcat for that) create a custom report using the report designer deploy the report on the server and send the link to the boss To make sure the SQL queries used in the report are the same used by OTRS we can temporarily have OTRS log all queries (remember to disable logging after you're done or your log files will grow out of control) by editing the Kernel/System/DB.pm file as documented here : # 0=off; 1=updates; 2=+selects; 3=+Connects; # $Self->{Debug} = $Param{Debug} || 0; # leave the original around for later $Self->{Debug} = 2;

Grails has great support for storing binary data into any database, Postgres included. It is usually simple to use but today it gave me problems. I have an application originally developed on pg 8.4 and grails 1.3.6 which stores images in a database table and used to work fine until I upgraded the database to 9.1. It turns out that with version 9 Postgres by default will return data to the client using a new hex format , instead of the escape format used in versions < 9. This, of course, confuses the client which returns garbage to the browser, hence the corrupted image. The solution is, luckily, quite simple: just tell postgres to revert to the old behaviour. It seems that this behaviour can be tuned per database, so you could have one database using the new format and another the old one: ALTER DATABASE dbname SET bytea_output='escape'; I should point out that I could also have updated the jdbc driver (now I'm using postgresql-8.3-603.jdbc3.jar ), but that s...

Bash shell scripting is one the things that I miss more on the times I work on Windows. Even as good as shell scripting is, sometimes I wish it was easy to keep track of state across script executions, for instance when a script executes a rsync at short intervals. In that case I don't care if one particula rsync fails, but I definitely care if it fails, say, ten times in a row or for more than a day. To do this I need some kind of way to keep state and record each run exit status. The simplest approach that I could think of is to use SQLite . Copied straight from the SQLite site:  SQLite is a software library that implements a self-contained, serverless, zero-configuration, transactional SQL database engine. SQLite is the most widely deployed SQL database engine in the world. The source code for SQLite is in the public domain. As I said, in this example I'll show how to track the status of a series of rsync operations, all run from a central node and pushed to remote nodes...

Today I was asked to quickly put together a script to replicate files/directories from a central server to a number of XP clients. This would be dead simple if it was not for Windows. The customer originally thought of robocopy/xcopy but I dont' really trust them: error reporting is kinda difficult to read and copying over cifs is slow and unreliable. So I set out to build it on top of rsync , but to do that I had to find a way to get rsync installed as a service on all the clients. Turns out it's quite easy. I started from a clean cygwin install to which I added rsync and cygrunsrv. The latter is the native cygwin tool that can be use to run any tradition *nix daemon as a native Windows service. Even a basic install like this will be in the order of 50MB or so which is too large for quick deployments. I then started trimming all the stuff that was not essential to the rsync service. At the end of the trimming process I was left with a mere 10MB, which after being compres...