OSCON Europe day one - first impressions

So I got here, so some quick first impressions:

• It's a bit smaller than I expected. But it's probably going to pick up tomorrow.
• Not surprisingly, a lot of familiar faces - if only familiar from webpages...
• So far, good quality talks!
• Oh yeah, "they" deliver talks about LISP...

No PostgreSQL talks scheduled until wednesday, but there's at least Bruce to chat about pg stuff with. And if there are others around, do show yourselves if you can figure out where we are :-)

/mha/uploads/oscon-dead-languages.jpg alt="" align="right" Now for the mandatory first day picture. Daiman Conway delivering his talk on dead languages, and actually showing LISP code (and that wasn't the worst!) I thought I'd never have to live through watching a presentation about LISP again... (pardon the layout - the wireless connection here (hotel) is very flaky, so I have to click things quickly before it goes offline)

Batch encoding conversion

Today I found out (the hard way - customer call!) that I had created a whole bunch of databases on a server in UNICODE encoding, when the locale picked only supported LATIN1. There were also a bunch of other dbs that were actually in LATIN1 that had no problems.

To solve this, I wrote a quick-n-dirty-n-hacky script to batch-convert all the databases from UNICODE to LATIN1. Luckily none of them were very large :-) I'll post it here in the hope that it'll help someone - but consider it mainly something to build from. Specifically the errorchecking should perhaps be better than just an echo of the exitcode.

It solved my problem. YMMV.

#!/bin/sh

/usr/local/pgsql/bin/psql template1 postgres -A -t -c "SELECT datname FROM pg_database WHERE datallowconn AND encoding=6 AND NOT datname ~ '.*_s'" | while read D ; do

  echo "Converting $D"
  echo "-- start" > $D.dump
  echo "ALTER DATABASE $D RENAME TO ${D}_s;" >> $D.dump
  echo "CREATE DATABASE $D ENCODING='LATIN1' TEMPLATE=template0;" >> $D.dump
  echo "\\connect $D" >> $D.dump
  /usr/local/pgsql/bin/pg_dump -U postgres $D >> $D.dump
  /usr/local/pgsql/bin/psql -v ON_ERROR_STOP=1 -U postgres template1 -f $D.dump
  echo Exitcode $?
done

ODBC and Kerberos

Finally got around to testing the new ODBC version for Kerberos support, and I'm happy to report it works perfectly and requires no hacks to install.

To use it, make sure you have a libpq with Kerberos support compiled in (yeah, this means manual build - we're looking into this for 8.1, but no promises yet). Then just set up your data source as you normally would, and skip specifying the password (you must specify a username though). As long as your pg_hba.conf is properly set up, that's all you need.

Synchronizing Apache passwords

I have an application with the need to synchronize apache passwords for basic auth (using DBM files) from a postgresql table. Using mod_auth_pgsql simply not an option - there may be other cases where it's an option but not wanted, in this case there it was out from the beginning.

So. pl/perl once again to the rescue, along with the CPAN module Apache::Htpasswd. Pretty easy to do with a trigger. Hit the link to see the code.

CREATE TABLE users (
  userid character varying(32) NOT NULL PRIMARY KEY,
  "password" character varying(32) DEFAULT randpwd() NOT NULL
);

CREATE FUNCTION sync_user_list() RETURNS "trigger"
  AS $_$

use Apache::Htpasswd;
use strict;

my %25userlist;

open(HT,"/data/webusers/users") || elog(ERROR,"Could not read password file: $!");

while (<HT>) {
  my ($uid,$pwd) = split /:/;
  $userlist{$uid}=1;
}

close(HT);

my $ht = new Apache::Htpasswd("/data/webusers/users") || elog(ERROR,"Cannot open password file: $!");

my $r = spi_exec_query("SELECT userid,password FROM users");

my %25users;

for (my $i = 0; $i < $r->{processed}; $i++) {
  my $u = $r->{rows}[$i]->{userid};
  my $p = $r->{rows}[$i]->{password};
  $users{$u} = 1;
  if (!$ht->fetchPass($u)) {
    $ht->htpasswd($u,$p) || elog(ERROR,"Could not add user $u: " . $ht->error());
  }
  elsif (!$ht->htCheckPassword($u,$p)) {
    $ht->htpasswd($u,$p,1) || elog(ERROR,"Could not add/set password for $u: " . $ht->error());
  }

}

foreach my $u (keys %25userlist) {
  if ($users{$u} != 1) {
    $ht->htDelete($u);
  }

}

return;
$_$
   LANGUAGE plperlu;



CREATE TRIGGER trg_sync_user_list
  AFTER INSERT OR DELETE OR UPDATE ON users
  FOR EACH STATEMENT
  EXECUTE PROCEDURE sync_user_list();

pginstaller progress

Code has now been committed to pginstaller to move user creation to the execute sequence. This means it can be used from silent installs, and also that the user won't be created until you have filled in all dialogs - in 8.0 we create it as soon as you leave the service config screen.

If you need this, or really anybody else as well, please grab latest cvs, test and let us know how it works. If you can't compile yourself, wait for the next beta which hopefully won't be too far off. But as this is a fairly major change, we'd appreciate a lot of testing.

A couple of minor fixes have also gone in, as we're nearing release. Not a lot of bug/feature-requests still pending now.

Hectic autumn

Wow, this autumn is turning out really hectic. Which can also be read as very interesting /mha/templates/default/img/emoticons/smile.png alt=":-)" style="display: inline; vertical-align: bottom;" class="emoticon" /

Apart from helping out with the final touches of PostgreSQL 8.1 (now in beta! Download and test!) and finishing off the 8.1 installer, I'm also involved in the Slony on win32 project Dave has been referring to.

Apart from coding up the win32 port of Slony, this also means I'll be going to Tokyo to deliver a talk on it /mha/templates/default/img/emoticons/laugh.png alt=":-D" style="display: inline; vertical-align: bottom;" class="emoticon" / Which is going to be really interesting - a big thanks to Hiroshi for setting this up!

And finally, I'm also a co-speaker with Simon Riggs on Euro OSCON in Amsterdam, on the topic of PostgreSQL in the Enterprise. My part of the talk will deal with win32 issues, such as Active Directory integration. If you're a PostgreSQL user or developer, I'll look forward to meeting you there!

Oh, and on top of this, all the normal stuff at work of course /mha/templates/default/img/emoticons/smile.png alt=":-)" style="display: inline; vertical-align: bottom;" class="emoticon" / So there will certainly not be many boring moments.

dblink-ldap on win32

Just a quick note that I now have dblink-ldap running fine on win32. I have some performance issues in my lab environment, but I think I've tracked them down to a broken DNS setup. Will need to try it again in a better environment in the next couple of days.

Let me know if you want to test it and I can send you a binary of what I have now.

Not caught up

I'm back for my around-two-weeks of vacation - much needed, and turned out very well.

Meanwhile, Dave has done a lot of work on the installer, and even put out a beta release.

Still leaves the break-out-account-creation part for me to do, but a lot of the other TODO items have been ticked off. Great work Dave!

Dave Page joins core

Dave Page has been added to PostgreSQL core, so now the win32 team has a representative there! Not to mention the website team. For these reasons and others I believe Dave is a great choice for the core team.

Congratulations, Dave!

pginstaller for 8.1

Got together with Dave this monday as I was visiting the UK. Didn't have much time to do anything productive, but we did get some discussion going on what to do about pginstaller for 8.1. Some highlights:

• User creation has to be moved to the execute sequence. This probably shouldn't be too hard now. It was when we first added it, but things have changed since. This would also mean it could easily be exposed to silent installs, which is a much requested feature.
• We need to move libpq.dll out of SYSTEM32. The current working model for this is to just remove the ability to move binaries between different directories, and force them into the same directory. Then we just stick libpq.dll there, and the problem is solved. We still need to do some testing on how this will effect the ODBC driver (the new one uses libpq), especially if you have 8.0 installed with libpq.dll in SYSTEM32.
• Slony will definitly be included, assuming the win32 version is stable and released. But that's the hope.

Of course, we're going to try to tick off anything else that is on the feature tracker on pgfoundry. If anybody wants to help out, now's a great time to start feeding patches :-)

And Dave - nice meeting you! Looking forward to the next time - most likely in Tokyo for the Slony talk!