Tag Archives: javascript

Visualizing Slovenian IT tax spending

My latest released project is focusing on Visualizing Slovenian IT tax spending (139 million euros), the idea here is to take otherwise meaningless numbers and display them visually in a way that tells a story of who is spending how much and on what. The data set comes directly from the government in semi-clean XLS file. Visualization technique I’ve decided on is treemap visualization to represent the data with different box sizes relative to each other.

Give it a try for yourself:

Launch the interactive Slovenian IT tax (in Slovenian)

Visualizing Slovenian IT tax spending

While visualization itself is nice, there are a two points that you have to be careful about when releasing such visualizations to the public:

Transparency of data and data transformations

In my case, the data set came directly from the government. In order to make sure that everyone can check my calculations I’ve included links to their file as well as provided a local copy in case their version changes or disappears.

You’re loosing and reinterpreting data with every visualization. That is why it’s important to also include transformation scripts so that others can check your work and possibly build on top of it or at least make sure that you didn’t do anything tricky with the data.

I’ve opted for a github repository where I’ve pushed all the associated files: http://github.com/gandalfar/itproracun. It’s a bit chaotic but it should be pretty self-explanatory to any python and JavaScript developer.

Telling the story
Every data visualization is trying to tell a story. It might not be obvious to the visualization author but it helps to identify this early in the process.

I started with just a simple breakdown based on the institutions:

It’s very noisy and it’s hard to compare different institutions to each other. Initial comments to this were that it’s not shiny enough. Cleaning the interface up I came to the following revision:

It’s much cleaner and what basically showed that I need to find an angle to this data. I decided to focus on the ratio between software and services vs. hardware and network equipment. Final version now tells a story of how police is spending a lot of their IT money on network and hardware equipment, while Tax Office is spending much more money on software and services.

Agenda of this last version of visualization should be clear to anyone who takes a few moments to study it.

Other lessons learned

Visualization toolkit should be powerful on one hand, but offer first results without too much work. JavaScript InfoVis Toolkit does this job very well. There are some interesting tidbits that are not entirely clear from the documentation, but become obvious once you start thinking how the rendering works.

The biggest time sink is parsing and cleaning up the data. Don’t expect that the .xls file will make any sense from the programmatic point of view, even though it mostly looks fine when viewed manually. Small parsing errors, moved cells and strange line breaks made parsing this data the biggest challenge.

Big thanks go the community of Slo-Tech and my brother that gave valuable feedback during the development.

I hope you’ve enjoyed this visualization. Let me know in comments what other points of view you’d like to see as well as your ideas how to further improve it.

Marko Samastur – Easy deployment of site-extensions with a browser plugin [firefox3 launch party notes]


Description unavailable

Image by ccmerino via Flickr

Marko Samastur – Easy deployment of site-extensions with a browser plugin.

Starting with Zemanta presentation, since it’s relevant for the presentation. Zemanta is a tool for contextual matching for blogging platforms and has to support a wide variety of platforms.

The limitations of XHR was a problem because of limitations of security-model that doesn’t allow cross-domain POST requests. In case of Zemanta, we can use custom-build JavaScript function.

When to use extension?

When web-site as such, doesn’t provide integration points (e.g., Blogger.com platform), the other reason is if you want to use functionality that is not available within the normal browser window.

Process

First thing that you need to do, is that you need to find a place for widget. In case of Zemanta, we used right side widget. In this case, we used CSS and made it a bit narrow.

Blog better using Zemanta
Creative Commons License photo credit: chucks

But wherever you go, CSS selectors that you use, will most probably match styles of the platform. So if you use a widget on different platforms.

You can conquer this within CSS by writing selectors that are more binding than the default way. The one that we did (and is not the best way), we use a platfora of id’s and classes and then we use these names and classes that get injected into web-page and are more binding than the platform ones. This one works great if you use two files: general one and platform specific one.

Second way is to use nesting and “id envelops” – encapsulation and while it’s ugly, it’s the best way:

#zem1 #zem2 #zem3 h3 {

}

the second thing that you should try to use is microformats. After you put your widget into the page, you usually want that widget to do something, and for that we normally use Javascript.

Javascript problems

The variety of problems is large, but a short selection:

Extending Object and Array – it sounds a good idea in a start, but it’s a major problem down the road. It’s usually a bad idea that many JavaScript libraries (like JQuery), use Objects as configuration buckets and if you extend it, it means that all objects will have this and bad things will happen.

The second thing is, that if you have a web-site and you want to have it “extendable”, it’s a good idea not to step all over their feet and potential code.

Private name-spaces – either through anonymous functions or through object hierarchy. The problem is that sometimes you want to change behaviour of methods that are hidden behind namespaces (e.g., changing how dragging boxes works in 2.3 WordPress).

One thread implication: what comes later can rely on what came before. It’s not always true, because JavaScript API’s like DOM functions doesn’t always run in the same thread.

While you can do is to test your JavaScript on slow browser and computers, and on different connections. But this is often not enough, so you should actively protect from this and check to make sure the code and functions are there before you call them.

Useful tools for extension creation

Firefox extension Wizard – http://ted.mielczarek.org/code/mozilla/extensionwiz/

FUEL – http://developer.mozilla.org/en/docs/FUEL

Besttoolbars – http://www.besttoolbars.net/

Zemanta Pixie

Gorazd Golob – Firefox extensions and what we have to do with them [firefox 3 release party notes]

20080422_TiledWall

Image by alx_chief via Flickr

Gorazd Golob - Firefox extensions and what we have to do with them.

Who are we

  • Developing local search engines
  • The most visited website in Slovenia
  • Present with Pogodak, local search sites in Croatia, Serbia, Bosnia & Herzegovina, Macedoni

Their servers are runinng 99.5% open source software. They are mostly Linux users in-house, so they use Firefox as the only real option.

Firefox usage numbers for Interseek:

  • June 2005  – 5.1%
  • June 2007 – 15.2%
  • June 2008 – 26.9%

Third one is always Opera (not Safari). In June 2008, Opera has 1.2% usage base.

We have Firefox users

We have a lot of Firefox users, and we must offer them a full service experience. We have to give them goodies like toolbars and Firefox search plugins. Publishing these features is good for us but also helps to expand the general Firefox user base.

Our story

in the first half of 2007 we launched Firefox toolbar and search plugin for Najdi.si and Pogodak (.hr, .ba., rs.) search engines.

What is so cool about Firefox extensions?

It’s mostly cool because XUL is cool. If you know how to make a dynmaic client side web application then you will probably know how to do XUL application. If you know how to program using XML, JavaScript and DOM you know XUL. XUL is supported by the Gecko engine.

Only XUL?

XPCOM components written in C++. Integration of other libraries into Mozilla.

Why would be ever need this? Things we can’t do with Javascript, we can do with an XPCOM component (e.g., usage of tcp socekts, etc.) and then use new functions from Javascript.

XUL extension

If you are planning an extension like toolbar or an ultra cool addon for uploading pictures to your new social network web site..

If you don’t have an idea what is possible – check http://addons.mozilla.org

Building UI applications based on XUL. Two popular applications/examples are Songbird and Komodo.

Is your plugin working in Firefox 3?

Secure updates of your application with files hosted on secure http servers or signed xpi files. If you don’t have https server, it’s also possible to sign them with help of McCoy to create keys and fingerprints.

Resources for you extensions

Zemanta Pixie

TextMate tip: Making single apostrophes double

imageComing from Python world where most of text is in double apostrophes like:


"Hello world"

to JavaScript where, e.g. jQuery , expects you to write:


$('#foobar')

with single quotes, it can be quite annoying to always have to type it twice, since for double quotes (“), TextMate it always does it automatically. This feature is called Auto-Paired Characters.

Here is a procedure how to extend your Auto-Paired characters to also include ‘, in the default list (or any other character for that matter).

Go to Bundle Editor, and locate Text bundle (you might have to turn it on using Filter List in bottom left corner). Now locate Text -> Miscellaneous with (p) next to it. Inside it you should see a snippet of text that says:


...
smartTypingPairs = (
( '"', '"' ),
( '(', ')' ),
( '{', '}' ),
( '[', ']' ),
( '“', '”' ),
( "', '" ),
);
...

edit that snippet to look like this:


...
smartTypingPairs = (
( '"', '"' ),
( '(', ')' ),
( '{', '}' ),
( '[', ']' ),
( '“', '”' ),
( "', '" ),
( "'", "'" ), <--- insert this line
);
...

And that’s it! Just restart TextMate for Bundle to reload and try it out by typing ‘ and observe its new auto-pairing powers.