Back to conferencing - PGWest

Today is the first day of PG-West, also known as JDCon-west. After having about a week off to visit places and visit friends, I'm now back up in San Francisco for this conference, which will cover most of this week. It's a bigger conference than JDCon has been before - in most measures. It as more sessions than ever before - but you have to wonder who thought it was a good idea to have five parallel sessions. That's almost a guarantee that there will be more than one session you really want to go do. I'd rather have seen it in fewer tracks and spread over more time.

It's also bigger in attendees than before. Last I heard it was at 203 or something like that - just over 200. That means that for the first time, JDCon is actually larger than a PGDay.EU (that had just over 190 last year) - I'm sure being in a great location in central San Francisco helps with that, along with the fact that the economy is in a better place now than a year ago. We're still in the lead over time (we were well over 200 a couple of years back), but we're also both well beaten by the Brazilians. It' sets a good target for us to work towards!

The set of sessions looks really good, but as usual the hallway track is the one where much of the really good things happen. I missed this mornings tutorial sessions completely due to very interesting discussions outside. Hopefully the slides and notes and/or video will be available to look over once we're done. If you're tracking this from away, the twitter stream has some interesting comments - and will hopefully have more!

Speaking of conferences - if you haven't already, now's a good time to register for pgday.eu. Particularly if you are planning to attend one of the training sessions - at least one of the sessions is already more than half sold out!

PGDay Europe 2010 Registration Open

It's finally time - we've opened up for registrations for PGDay Europe 2010.

We are not finished with the schedule yet, so if you are looking for a specific talk, you'll have to wait a while longer. Work is in progress though - we've already notified some of our speakers that they are approved. However, if you submitted a talk and have not heard from us yet, it's not yet time to panic. The reason we haven't published a schedule yet is that we're working on ways to include more talks!

So why would you want to go register now, even though the schedule isn't posted yet? Well, first of all, the schedule is looking like it'll be at least as good as last year. We have several well known good speakers from the community showing up again, and also some fresh faces with interesting topics!

But more importantly, this year, we've added training for the first time. Training will run on the wednesday (the main conference being monday and tuesday). This training is limited availability (25 seats per session), and extra cost. You pay this at registration. And the seats are handed out on a first come/first serve basis. So if you want to attend the training, now is the time to register! The training schedule is final, so be sure not to pick two training sessions that run at the same time.

The conference this year will be held at the Millennium Hotel in Stuttgart. We do recommend that you reserve a room with that hotel, as we have a group rate there, and it's conveniently located (hint: no need to go outside to get from A to B). But using this hotel is not mandatory - you can book your room anywhere you like. However, it should be noted that wireless internet is only included if you booked a room using our group rate. If you don't, you can pre-purchase the access when you register, or you can solve it yourself for example using 3G data. We will not have the ability to provide or sell you wireless access unless you pre-purchase it!

With all that said, head off and register!

Monitoring streaming replication lag

Once you've set up the great new Streaming Replication with Hot Standby in PostgreSQL 9.0, you need to somehow monitor it. I've created a simple Munin plugin to graph the lag between the master and the slave, and also the lag between receiving and applying on the slave. It's available on my github page, and will likely also be included in the next Munin version. If you are using SR and Munin (or just SR and want to graph it), please try it out and let me know if there are issues with it - it could certainly do with some more testing.

PostgreSQL - now on git!

So it finally happened. The official PostgreSQL master source tree is now managed in git, instead of cvs. This means, amongst other things, that the worlds most advanced open source database now has a version control system with.. eh. atomic commits!

Like the first run, this one had some issues with it, but it was smaller and resolved in time not to have to roll back. This time, it turned out that the cvs version that ships in Debian GNU/Linux comes with patches that change the default date format to the ISO standard. But since one of our main requirements on the conversion was to be able to faithfully represent the old versions of the code, this broke every single file - since we used CVS keyword expansion in the old tree. Once we found this, it was a simple case of adding the DateFormat=old parameter to the CVS config file and re-run the whole conversion - which took several hours.

A lot of work went into making the repository conversion correct. Some of this was due to issues in the toolchain used - many thanks to Michael Haggerty and Max Bowsher for getting those fixed and explaining some of the behaviors of the software for us. In the end, a number of things needed to be changed in our existing CVS repository to make it migrate properly. Tom Lane provided a big patch to apply to the CVS repository itself prior to the conversion that cleaned most of those up - you can find a copy of it my github page if you're interested.

With this patch applied, we managed a conversion that was very close to the original repository. I personally think this is only because the PostgreSQL project has been very careful about how it deals with it's CVS repository - using it in a fairly simple way. And even with that, we had a number of issues - such as tags moved "after the fact", and branches created off partial checkouts. A fair number of the issues were simply because CVS doesn't have ways to represent everything in a reasonable way, such as issues when a file was deleted, re-added, deleted again, and mix this over different branches.

Git obviously deals with this better, and hopefully we'll have no such issues creeping into the new repository. However, the PostgreSQL project will be sticking with our "conservative approach" to source control - at least for the time being. For this reason, we are restricting what committers can use within git. We still allow any developers (and committers) to use whatever parts of git they want as they develop, but for commits going into the main tree, we are making a number of restrictions:

• We will not allow merge commits. The PostgreSQL project doesn't follow the "git workflow" - we generally develop our patches on the master branch, and then back-patch to released stable branches for important bugs. We will continue doing this as separate commits and not using merges, thus keeping history linear.
• We will not use the author field in git to tag it with the patches original author (even in the few cases when the patch is actually authored by a single person). Instead, we will require that author and committer are always set to the same thing, and we will then credit the author(s) (along with the reviewer(s)) in the commit message, just like we've done before.
• As a follow-on to that requirement, we will require that all committers are the ones registered with the project, using the same name and address on all commits. So even if a patch is developed on a topic branch on say github, it will get collapsed into a single commit (or maybe a couple, depending on size) tagged with the committers name on that.

There has been a lot of discussion around this, and this is how the PostgreSQL project has worked and wants to continue working. We may change this sometime in the future, but not now - we are only changing the tool, and not the workflow.

To enforce these requirements, I've developed a policy hook for our git server that makes sure we don't make the mistake. It's up on my github page, along with the script we use to generate commit mails to the pgsql-committers list that look just the way we want them to.

What does this mean for you as a PostgreSQL user? Really, nothing at all.

What does this mean for you as a PostgreSQL patch developer? Not much. If you did your work off the cvs-to-git mirror, you need to do a new clone. This repository is converted from scratch, so the old one is not valid anymore. We still encourage you to use for example github if you want to do your development there, but the patch submission process remains the same - send a context style diff to the pgsql-hackers mailinglist.

What does this mean for you as a buildfarm-animal maintainer? You need to reconfigure it to use git. I expect Andrew to post instructions on exactly what to do, and keep track of who hasn't done it ;)

Thanks and Well done to all the people involved in making this happen!

A busy week for PostgreSQL - hello 9.0!

In case you missed it (I'm certainly not the first on Planet PostgreSQL to blog about this, but there are supposedly other aggregators), PostgreSQL 9.0 has been released. Comes with spiffy things like Streaming Replication, Hot Standby and Exclusion Constraints! If you aren't up to speed on all the news already, go check out the release notes.

Second, I yesterday committed Thom Browns changes to the stylesheets for the documentation. So not only do you get the documentation for the new version, it also looks a lot better than the old one - nicer formatting for tables and highlighting examples and code in a better way. This is also changed in the old version of the docs.

Finally, tonight we start the second attempt to move the authoritative PostgreSQL source tree to git. It didn't end well last time and we reverted back to cvs, but given the large amount of work put into it by many people, I have much higher hopes this time. Stay tuned...

PGDay.EU announced and call for papers

PGDay.EU 2010 has finally been announced. It will be in Stuttgart, Germany, on December 6th to 8th. More details available on the conference website.

We have also sent out our call for papers. If you have done something interesting with PostgreSQL, please go ahead and submit a talk! We are currently looking for talks in both English and German!

Robotic moderation duties

Every now and then, the discussion about why it takes too long for messages to get approved when posted to some of the PostgreSQL mailinglists comes up, and it goes around a couple of laps. Maybe we get a new moderator. But eventually it comes back down to not enough people. Personally, I've usually just found myself not having the time to keep up with moderation. So when I needed a project to learn some Android development on, I figured this could be an interesting (probably) and useful (maybe) topic.

So, meet Mailinglist Moderator. A tiny android application that helps with the daily moderation chores for anybody moderating Mailman or Majordomo2 mailinglists (should be easy enough to add more list managers if there is any that people actually use).

The application will simply enumerate all unmoderated items and let you set them to accept or reject either one by one or in batch. Personally, I've found it makes it a lot more likely I will do moderation - it's literally down to 30 seconds while waiting for a bus or train, or something like that. And once the hurdle is gone, it's a lot more likely I'll end up actually moderating. I found it useful - hopefully others will as well.

Here are a couple of screenshots showing what the application looks like.

The app is available as an APK for download on my github page. It hasn't been published to the market now, but I'll do that if enough people find it useful and ask me for it...

And of course, this is all BSD licensed open source, so any contributions are welcome!

PostgreSQL Europe Marchandise Store

We've finally opened the merchandise store for PostgreSQL Europe. It's a chance for everybody who haven't had the chance to attend one of the many PostgreSQL events where we've been selling mugs and shirts for a long time, as well as a chance to get some stuff that we haven't previously had available.

There's close to zero earnings for PostgreSQL Europe off these purchases - we're trying to make it as cheap as we can for everybody. You are of course most welcome to donate some extra to the project, should you wish.

Planet integration update

This post is one of those seriously annoying ones that's just here to verify that the updates I've made to the twitter integration of planet works. Since Twitter are terminating the type of authentication we were using, I had to change it. It is a change to a better method, but still somewhat annoying.

If you're interested in looking at the code, it's up on github and on git.postgresql.org.

PostgreSQL Europe election results

The elections for PostgreSQL Europe are now closed, and the full results are published on http://www.postgresql.eu/elections/2/.

The PostgreSQL Europe Board would like to welcome our new board members Dave Page and Guillaume Lelarge, as well as welcome back Andreas Scherbaum who was re-elected.

I would also like to personally thank Gabriele Bartolini, member of our original board and one of the initiators for PostgreSQL Europe, who will now be leaving the board. Gabriele has been instrumental in getting PostgreSQL Europe off the ground, and I'm sure we will see much of him within our community in the future as well, even if he is not serving on the board.

A total of 20 persons voted in the elections, out of the 39 that were eligible.

Thanks to all who participated in the elections!

If you have any further questions around this, please feel free to contact me or any of the other board members (old or new) directly.