Robotic moderation duties

Every now and then, the discussion about why it takes too long for messages to get approved when posted to some of the PostgreSQL mailinglists comes up, and it goes around a couple of laps. Maybe we get a new moderator. But eventually it comes back down to not enough people. Personally, I've usually just found myself not having the time to keep up with moderation. So when I needed a project to learn some Android development on, I figured this could be an interesting (probably) and useful (maybe) topic.

So, meet Mailinglist Moderator. A tiny android application that helps with the daily moderation chores for anybody moderating Mailman or Majordomo2 mailinglists (should be easy enough to add more list managers if there is any that people actually use).

The application will simply enumerate all unmoderated items and let you set them to accept or reject either one by one or in batch. Personally, I've found it makes it a lot more likely I will do moderation - it's literally down to 30 seconds while waiting for a bus or train, or something like that. And once the hurdle is gone, it's a lot more likely I'll end up actually moderating. I found it useful - hopefully others will as well.

Here are a couple of screenshots showing what the application looks like.

The app is available as an APK for download on my github page. It hasn't been published to the market now, but I'll do that if enough people find it useful and ask me for it...

And of course, this is all BSD licensed open source, so any contributions are welcome!

PostgreSQL Europe Marchandise Store

We've finally opened the merchandise store for PostgreSQL Europe. It's a chance for everybody who haven't had the chance to attend one of the many PostgreSQL events where we've been selling mugs and shirts for a long time, as well as a chance to get some stuff that we haven't previously had available.

There's close to zero earnings for PostgreSQL Europe off these purchases - we're trying to make it as cheap as we can for everybody. You are of course most welcome to donate some extra to the project, should you wish.

Planet integration update

This post is one of those seriously annoying ones that's just here to verify that the updates I've made to the twitter integration of planet works. Since Twitter are terminating the type of authentication we were using, I had to change it. It is a change to a better method, but still somewhat annoying.

If you're interested in looking at the code, it's up on github and on git.postgresql.org.

PostgreSQL Europe election results

The elections for PostgreSQL Europe are now closed, and the full results are published on http://www.postgresql.eu/elections/2/.

The PostgreSQL Europe Board would like to welcome our new board members Dave Page and Guillaume Lelarge, as well as welcome back Andreas Scherbaum who was re-elected.

I would also like to personally thank Gabriele Bartolini, member of our original board and one of the initiators for PostgreSQL Europe, who will now be leaving the board. Gabriele has been instrumental in getting PostgreSQL Europe off the ground, and I'm sure we will see much of him within our community in the future as well, even if he is not serving on the board.

A total of 20 persons voted in the elections, out of the 39 that were eligible.

Thanks to all who participated in the elections!

If you have any further questions around this, please feel free to contact me or any of the other board members (old or new) directly.

PostgreSQL Europe - get your nominations in

We have now opened the nominations period for the upcoming elections to the board of PostgreSQL Europe. It's simple - anybody who is a member of PostgreSQL Europe (if you're in Europe and doing PostgreSQL stuff, as a developer, consultant or just a user, you really should be. It's easy!) can be a candidate to be elected. You just need to be nominated by one member (who can be yourself, just to let people know you are interested in being a candidate) and seconded by one other member - that's all.

So if you're interested in this, or know somebody who should be, post your nominations to the pgeu-general mailinglist. For full details about the procedure, see this email.

PostgreSQL infrastructure updates

At PG-east^H^H^H^H^H^H^HJDCon here in Philly yesterday, Dave announced the infrastructure changes that we decided on late last year, and have been working on and off to get the groundwork done for over the past months (mostly off, or we'd be done already). Since I've had a number of questions around it, I'm going to post a summary here.

The bottom line is, we're changing - over time - how we're going to run and manage the infrastructure behind the postgresql.org services. Things like website, source code repositories, etc. The reason is simple - the way we are doing it now has become too hard to maintain, and just requires too much manual work.

There are several parts to these changes. For one, we are building automated methods for a lot of things that previously required a little manual work. But even a little manual work rapidly becomes very time-consuming when you have a lot of machines - and the number of machines we have have been increasing fast over the past couple of years.

The biggest change is that we will be moving our services off FreeBSD, that we use now, onto Debian GNU/Linux. At the same time, we will switch virtualization solution from FreeBSD Jails to KVM. FreeBSD jails have served us very well over time, but now that we have access to a well working KVM product that uses hardware virtualization, the gains of having full virtualization are much easier to get at.

One of the main drivers for the change is that maintaining ports-based installs are just taking way too much time. The new system will be based heavily around using Debian packages and the apt system, to the point that everything being installed will always be done using "meta-packages" that will ensure that all our machines look the same way. This also plugs into our monitoring systems very well, making applying (security) updates across the many machines much easier. In particular, it should get rid of what's sometimes a multi-day operation to get everything security patched, due to dependencies and the slowness of updating ports.

It is quite possible - in fact, I'd say probable - that there are perfectly fine ways of doing this on FreeBSD. But this outlines another big reason for this change - it's simply a lot easier to find people who can do these things on a Linux based system today. Our efforts are entirely volunteer based, and in the end we just don't have people who know enough FreeBSD volunteering to do this work. In fact, we have had several people retire from the sysadmin team over the years specifically because they refuse to work with FreeBSD. We don't expect this change to magically create more volunteers, but it will make it easier to recruit in the future.

The system will also automate a lot of the configuration work that needs to be done manually today, as well as automatically configure our monitoring systems (primarily Nagios and Munin) and backups. And finally, the system will automatically provision (and un-provision) users and access permissions across all machines from a central repository.

The code is not entirely done yet, but we do have enough in place to automatically provision and configure basic virtual machines in just minutes, as well as much of the configuration. We have prototype deployments online, but we haven't started moving production services yet. We'll hope to get started on that shortly. It will obviously take a long time before we have migrated all services - in fact, we may never migrate all services. The focus will obviously be on the most manpower-intensive ones first.

As usual the code that's used to build these services will be published under the PostgreSQL licence once it's finished. Some security sensitive parts may be removed, but the bulk of it is very generic and should be re-usable as a whole or in parts.

Another issue that's going to delay some changes is the fact that a few of our infrastructure servers are currently not VT-capable, and thus cannot run KVM. In the short term these machines will continue to run FreeBSD Jails, but eventually we want to replace these machines with more modern ones. So if you are sitting on one or more VT-capable machines with enough RAM to run a number of virtual machines, hosted in a datacenter somewhere that can provide us with multiple IP addresses and a decent availability of network and services, that you would like donate to the PostgreSQL project, please get in touch with our sysadmins team to see if this is something we can work with!

Heading west, going east, better schedule, and more

(public service announcement further down!)

I'm starting to get ready for the most mis-named conference so far this year. I mean, seriously. It's called East 2010, and yet it's located approximately 6000 km (that's almost 4000 miles for you Americans) to the west of the prime meridian. That's not even close. Sure, it's to the east of where the West conference is usually held, but really, this reminds me of POSIX timezones...

This will be a somewhat different conference than previous PostgreSQL conferences. It's the first big commercial conference. This has enabled it to change venue from "local university or college where rooms are cheap or free" to a conference hotel in downtown Philadelphia. Whether this is actually good for the talks is yet to be seen, but it's likely going to make some things around the conference easier and more integrated. There will also be a exhibition area - something we have tried for pgday.eu without much interest, but it will hopefully be more successful in this surrounding.

The contents of the conference are also somewhat different, since there is now a clear focus also on "decision makers", something that many PostgreSQL conferences have been lacking in. We may all want it to be true that the decision makers are the people who are actually going to use the product, but we know that's not true. This gives a conference schedule that contains a broader range of talks than we're used to - this can only be good.

Myself, I will be giving an updated version of my security talk, focusing on authentication and SSL. The updates are mainly around the new and changed authentication methods in 9.0, and some minor updates on the SSL part. If you've seen it before it may be an interesting refresher, but you might be better off going to see Greg Smith's benchmarking talk if you haven't...

Now for the second part, which is the public service announcement...

If you're like me, you find the official schedule for East very hard to read and basically useless to get an overview. It's also horrible to use from a device like the iPhone, something I'm quite likely to use at the conference. But we solved this problem for the PostgreSQL Europe conferences, and that code is pretty simple. So some copy/paste of a couple of hundred lines of python, some glue code, and voilà, a much more (IMHO) readable schedule. As a bonus, it also generates an aggregate iCalendar feed, that you can plug directly into the calendar application on the iPhone (or I assume most other phones). Google calendar may be very nice to use to work on the schedule, but I find this a lot more user friendly for those reading it - particularly in the ability to get an overview.

The page and the feed will both update once per hour by pulling from the official calendar feeds. They will also adjust for the fact that all the official feeds are 3 hours off due to the calendars being set in the wrong timezone. So should they suddenly jump 3 hours that's because the official ones were fixed - just remind me and I'll take out the adjustment. Anybody is free to use them, but of course, usual disclaimers apply, double check with the official one, etc, etc, etc.

Finally, then mandatory before

and after shots.

If you're interested in the code for the aggregator itself, it's up on my github page. And of course, any patches for cool features or just making it look better are always appreciated - it's open source after all.

FOSS STHML #1

Yesterday was the first meeting for the FOSS STHLM "group" - a (very) loose group of FOSS interested people in the Stockholm region. We met in a lecture hall at the Stockholm University in Kista north of Stockholm, for a couple of hours of short presentations. The lineup was very nice: cool embedded stuff? yup, rockbox. General linux? Yup, upstart. Debian-specific? Of course. OpenSource Sweden? yeah. Curl? What else did you expect with Daniel as one of the organizers? PostgreSQL? Yeah, why else would I be blogging this?

Our allocated time was short, so the setup was many short talks. It actually worked a lot better than I thought it would, but it's still very hard to convey something useful in just 20 minutes.

I had a hard time figuring out what I should focus on, so I did a split into two parts (perfect strategy - if you have too little time to do one thing good, do two things in the same time instead...). I started with a section about "things to think about if you're switching to PostgreSQL or trying it out for the first time" - things like the very most basic config parameters that you always have to touch. And of course the classic - "ident authentication failed" issue that hits everybody on RedHat or Debian platforms at least (which is most of the users - definitely in this crowd). It's hard to do much in 12 minutes, hopefully it got some people interested.

I followed that with a very very very short version of "what's coming up in 9.0". Once again, I focused on one of my personal favorite features, which is Exclusion Constraints. While this is often listed as one of the cool things in 9.0, sometimes I feel that too much focus is on streaming replication and hot standby. Don't get me wrong, these are very good and very much needed features. But Exclusion Constraints is a real eye-opener. All databases (including PostgreSQL, of course) have replication - this is "just another way to do it". Yes, a very important and good way to do it, but it's still not something brand new. Exclusion Constraints is something that's fundamentally new. And it's a brilliant example of how PostgreSQL is moving the goalposts forward. Oh, and it's really useful and cool, of course! (and it'll be even better when we have the period datatype, or something similar, in 9.1!)

In summary, I think it was a great event. Big thanks to all those who helped make it happen! Hopefully we can follow it up with many more in similar ways - perhaps mixing these many-short-talks with some more focused discussions on specific areas or technologies. Time will tell...

Update: For those who asked, there were somewhere around 150 people in the sessions.

Summary of feedback from FOSDEM devroom

I forgot completely to close up the feedback system for our talks at FOSDEM and post the feedback to the speakers. My apologies to all speakers and others who were interested! Thankfully, Marc Balmer reminded me today and the individual ratings for different speakers have been sent off to them.

When it comes to the aggregated statistics, they look fairly similar to what we saw at PGDay. The number of speakers were much lower, and unfortunately so were the number of people giving feedback. But in the end, the results were fairly clear. I am especially happy with the level of knowledge in the speakers we attract. Let's start with the graphs:

Breaking down the details, I think this shows a great quality of our speakers:

Name | Speaker Quality | Votes | Deviation Simon Riggs | 4.8 | 6 | 0.4 Magnus Hagander | 4.6 | 5 | 0.5 David Fetter | 4.5 | 4 | 1 Heikki Linnakangas | 4.3 | 9 | 0.9 Dave Page | 4.2 | 10 | 1.0 Marc Balmer | 3.4 | 5 | 1.7 Peter Eisentraut | 3.3 | 7 | 1.0

And the level of their knowledge is even higher:

Name | Speaker Knowledge | Votes | Deviation Simon Riggs | 5 | 6 | 0 Magnus Hagander | 5 | 5 | 0 Heikki Linnakangas | 5 | 9 | 0 David Fetter | 4.8 | 4 | 0.5 Dave Page | 4.3 | 10 | 1.1 Peter Eisentraut | 4.3 | 7 | 1.1 || Marc Balmer || 3.6 || 5 || 1.7 ||

Even at the bottom of this list, the numbers are very good! Unfortunately, the fairly low number of votes give a high deviation at the bottom of the chart.

For next time, we'd really like to get more people to leave their feedback. Does anybody have a good idea on how to make that happen?

Important updates for PostgreSQL Europe

Those of you who were at FOSDEM, heard my bold announcement that we will finally get off the ground and hold proper elections fort he board of PostgreSQL Europe, per a decision made the day before. Now, when we got home and read up on the statutes that govern the organization, it turns out we actually can't do that. The way forward that we've found around this, is to actually change the statutes to make these things possible.

To change the statutes, we need a majority of the General Assembly, made out of the active members, to agree on a change. It turns out that given how the statues are formed, there are currently only four members in PostgreSQL Europe, and these four members are the original board. Because of this, it was pretty easy to change the statutes (we only needed 3 of these people to agree, which we all did). And this we have done. The reason for changing them is to make sure we can get a working system going where we have more people involved. But in order to make that work, we first had to make this small-group decision.

The changes have just been published to www.postgresql.eu, and should show up any minute there. The changes that have been made are:

  • We have removed the definition of "Benefactor members", to keep things simple. We are keeping Active and Honorary members.
  • Created a membership fee, to control who is a member.
  • Change so that any resignations are handled by the Board, not the President, to make things more flexible.
  • Add that a membership is terminated if the fee isn't paid.
  • Added mailinglist as an approved way to do meetings. Since this is how we do most of our communicating, it made no sense not to have it on there.
  • Moved the budget out of the GA, it's just a tool for the executive board. Keeps down the paperwork.
  • Add a defined way for the GA to vote on issues, using a web based system (to be developed).
  • Since we will be voting over the internet, remove the ability to vote by proxy.

As you can see, the main change is the creation of a membership fee. This is the most controversial change. The idea here is that it will be a nominal fee (€10 for two years of membership - though that exact amount isn't part of the statutes, to keep flexibility). This will make it possible for us to have an actual membership list, which will be the grounds for who can vote.

The current method for handling membership requires the signing and sending around of forms on paper (yes! Actual dead trees!). Given how our organization works, this simply does not work. That is the main driver behind us changing this.

We did evaluate several other ways to deal with membership, ranging from things like requiring physical meetings (not really a good idea for a pan-European organization) to fancy PGP signing schemes (which really is way too complex for a lot of people). Our conclusion was that this was at least the least bad way to do it.

We are going to build on this shortly with more information about exactly how the elections are going to be held. The plan is still to have completed this years election before the summer, and we are well on track to make that goal.

The changes to the English statutes are of course available in our git repository. The binding french changes will be made shortly.

If you have any changes or comments, please direct them at the pgeu-general@postgresql.org mailinglist, or directly contact the board.

Conferences

I speak at and organize conferences around Open Source in general and PostgreSQL in particular.

Upcoming

FOSDEM+PGDay 2019
Feb 1-3, 2019
Brussels, Belgium
Nordic PGDay 2019
Mar 19, 2019
Copenhagen, Denmark

Past

PGConf.Asia 2018
Dec 10-12, 2018
Tokyo, Japan
DC PostgreSQL Users Group
Nov 14, 2018
Washington DC, USA
New York City PostgreSQL User Group
Nov 13, 2018
New York City, NY, USA
Driving IT 2018
Nov 2, 2018
Copenhagen, Denmark
PGConf.EU 2018
Oct 23-26, 2018
Lisbon, Portugal
More past conferences