Culture successfully raises funds from the EU

Today  Cultural Contact Point Slovenia based at SCCA-Ljubljana, Media Desk Slovenia, and Culture.si/Ljudmila Art and Science Laboratory released visualisations of last decade of successful fundraising data from EU programmes.

culture-eu-funding-interactive

kultura_klicaJ_TISK64x44_ENG_2013-10-15.indd

Process

This is first public release of such data in Slovenia and is result of 6 months of intensive work in data reconciliation, methodology and finally – visualisation.

I helped mostly in regards with data reconciliation and can speak about the tools we used. Basic tool was Google Spreadsheet and was used as a database that everyone could contribute to and it helped us sync the data together. It also allowed for basic pivot table based visualisations. It worked mostly ok and ability to write scripts for it also helped a lot. Finally the data was moved into Semantic media wiki and visualised using d3.js.

Lessons learned

  • Google Spreadsheets don’t scale. After you reach about 1000 rows with 30 columns, it becomes almost unusable slow.
  • This dataset is complex enough that it would benefit from automatic checks – automated reimporting into real database and basic reports – unique institution, basic pivot tables. This would help with encoding, whitespace issues that Spreadsheet doesn’t handle.
  • Google Spreadsheet got really good tools for pivot tables, but they’re a pain to manage if data ranges change. It can probably be further automated but I haven’t yet figured out how.

Notes on Open Data from OKCon 2013

It’s popular today to work in Open Data, Big Data or similar space. It feels just like the times of Web 2.0 mashups, but instead of simple Google Maps based tools we’re now creating powerful visualisations that often feel like an end by itself.

In a way, I expected OKCon, OpenKnowledge Foundation’s conference, to be about makers – people that build useful part of OpenData ecosystem and to provide in-depth case studies. Instead we’ve got a mix of representatives from government, large international NGOs, small NGOs and various developers and software providers. To me these worlds felt just too far apart.

Data Portals

Data portals are here and while we haven’t seen a large scale deployment from government in Slovenia, it’s likely that they we will have something in this regard in the next few years. Everyone else has already deployed their first version and more advanced (public) institutions are already on their second or third attempt. Just as with social media a few years ago, we’re now also seeing first case studies that show economical and political advantage of providing such sites.

Self hosted CKAN platform seems a popular tool for such efforts (or it just might be conference bias as it’s developed by OKFN).

Budgets and contracts

A lot of effort is expanded in area of representing budgets, tenders, company ownerships and similar. In this regard, Slovenia’s Supervizor looks like something that’s from far future compared to what other countries or projects achieved at the moment. We could contribute a lot back to the international community, if we can produce case studies on benefits (or lack of) of such system.

At the same time, building visualisations around local budgets is something that doesn’t feel productive anymore. I think we should just upload sanitised data to a portal like OpenSpending and focus our efforts on projects with more impact.

Maps

Just as with mashups, everyone loves geospatial representation. The more colours and points of interest, the better. The only problem is, that it’s often useless for people that actually need to use it. While not openly expressed during presentations, discussions during the break often revolved around how bloated and useless were these representations and that just having a good text/table based report would work so much better.

At some point, community will have to embrace modern product development methodology – stories, user testing, iterative development and similar.  Right now it feels like a lot of these tools are either too generic or sub-contracted and developed through water-fall model.

Having said that, I’ve seen great examples of how to do things right: landmatrix.org

Tools

While NGOs might be building things the old fashioned way, their developers certainly aren’t. Tools and platforms are openly licensed and published on GitHub and often tied into different continuous integration environments.

  •  https://github.com/ostap/comp – automatically exposes your local CSV, XML and similar files as JSON endpoint through standalone Go based server. Developed by mingle.io team.
  • Drake – for building workflows around data
  • Pandas – python based data analysis

 

Conclusion

Software development is already hard for teams of seasoned veterans that work on projects inside the tech industry. It’s almost impossibly hard for both large and small NGOs since there just isn’t enough talent available. Additionally, it seems that these organisations often don’t want to coordinate efforts (even basic sharing of data) with each other or even internally, making projects even less likely to succeed.

I think that  we’ll continue to see a lot of badly executed projects in this area until modern, tech-driven groups like OKFN and Sunlight Foundation manage to raise the bar.

Visualizing Slovenian coalition agreement

With the election of new Slovenian prime minister we also got formal release of a Coalition agreement. Since it’s a 72 page document, I was wondering what keywords would stand out. Here is the result:

Pogodba za Slovenijo 2012 - 2015 - word cloud (top 80 words)

While we’re at it, we can also take a look at the coalition agreement that Pozitivna Slovenija prepared. As we run them through the same process, we get:

Koalicijska pogodba - Pozitivna Slovenija - 2012 (top 80 words)

 

A few words on how to reproduce this:

  • Grab your favorite OCR software and convert scanned PDF into .docx
  • From Word save it into .txt file
  • Lemmatize the words so you normalize all the grammar rules
  • Apply stop-words (in this case mostly: ministrstvo*, vlada, slovenija*, ..)
  • Drop the resulting text into wordle.net

 

WebCampLjubljana Autumn 2011

Last weekend we finished latest addition to WebCampLjubljana series with another sold out event and with 160+ in attendance. We’ve had more than 20 talks and finished the day with a short series of lightning talks.

For this event I tried not enforcing any specific language of the talks, which in the end meant that we had most of the talks in Slovenian and Croatian and a few in English.

We’ve also switched to all digital cameras this time around and so we could release a nice archive of videos in just a week after the event: http://video.webcamp.si. There’s a surprising amount of work required to get such archive online.

Finishing off with a few pictures from Peter’s gallery and I hope to see you at the future events.

Rebooting the community

This year Kiberpipa will be 10+ years old and its’ event lineup is incredible. Over 350 events a year involving various talks, meetups, screening and everything. During this time a bunch of new technologies showed and demand for jobs as well as our interest shifted and I/we stared creating bigger and more exclusive events (as in: you can only attend if you speak).

Django Meet Ljubljana, 6. julij 2011

It mostly worked, with limited success. What happens is that while the core members Kiberpipa and related communities are used to speaking and working out in the open, not everyone is. And we didn’t give new members of community a chance to slowly build up their involvement.

With this in mind, I’m hoping that we can get a few more meetups and events going where minimum level of involvement is a 15-min talk instead of a full 45-min lecture with all eyes on you.