I’m going to start regularly writing daily posts about what I did today and the things I found interesting, I’ll at least try anyway.

First up for today is a personal thing, I completed a rather largish Uni assignment today which reminded me of all of the pains that come with C++, but to follow that I returned to working on my filesystem in C, which is just more pain. I got a quick response back, and almost full marks (96%) so I’m happy for all of the time I put in to get it done and how its probably far more complicated than anything else that will be submitted (it used Boost Signals and a whole heap of other things that I don’t think will ever be taught in the subject for a long time). But hey, thats just Uni!

Today I finally managed to get Pentaho, some business intelligence (BI) software, to play nicely with Novell eDirectory’s LDAP interface. I must have missed the option, but Pentaho doesn’t seem to accept anonymous binding to the LDAP server, which means I need to bind as a user. By default our users funnily enough have less access than the anonymous account (which is actually a proxy account with full browse permissions). The solution was simple enough: we shunted our dummy Pentaho user into the same group as the anonymous proxy account and everything worked. So I’ve now got Pentaho using LDAP for authentication (yay!) and a MySQL database to get its role/group permissions. Funnily enough when its all said and done the documentation is pretty close to the mark.

But once I had that I don’t have an ability to manage the groups/roles within Pentaho, so I end up having to write some small PHP to manage that. Luckily I worked on a project a while back that I called “Joomla! Central Management for Users” which basically connected directly to MySQL databases of Joomla! installs and altered the users. I had originally built it with a plugin infrastructure in mind so that I could plug other stuff into it later. Starting this morning it only had a ‘connector’ for Joomla! 1.0 via MySQL and LDAP, now it has one for the Pentaho security tables. This means I can easily copy users from LDAP or Joomla! into Pentaho without too much issues and has a debugged user interface already. But wait theres more!

When I was originally developing the tool I wrote a query language for it. See, SQL is a great language for databases, but its a bit hard to apply in situations where you don’t quite need all of that power. So I wrote my own query language. Its quite simple it can validate simple attributes and allows for set operations within “Sites” (a site is a container for users and groups). So for example I want to see all of the users who are on our web site but not in our LDAP directory:
existsin “Web Sites” and not existsin “LDAP”

Primitive sure, but it because writing a large SQL expression for something simple. I hope to expand on it, but it already does what it needs to do for the time being.

So I’ve covered query languages, LDAP and BI! All I need now is the filesystem news. Today there was a whole heap of fan fare on Slashdot about the ZFS news from Apple, whilst thats cool and all (especially since I don’t mind Apple’s UI), I personally have my own filesystem that I’ve gotten back into to do some work on. It also happens to be a Uni assignment due on Friday! So I’ll be back to working on that and hopefully I’ll have it to a nice stage that I can do some lightening talks at linux.conf.au!

I’m presently over in Taiwan presenting at ICOS, and one of the things I’m looking at is one of Joomla!’s 2007 Summer of Code student’s final project. Its an email interface for publishing into Joomla!, and it raises what I find is the interesting concept: building a simple interface into a more complicated thing and allowing that simple interface to provide the basis of a more advanced system.

When I consider this more and more I look at the ability for systems to build up more advanced using simpler tools. It is actually the basis of the UNIX philosophy, lots of small tools doing their own thing really well and then tying them all together to build a more advanced product, perhaps a GUI interface or something that automates things.

When you step back and think about it, the internet is based on a similar concept. If you look at a really common protocol like HTTP or SMTP you can easily write and send valid protocol responses without too much effort by hand. And then you look at what is built on top of that, we have products like Mozilla Firefox which provide a nice GUI interface into the system and allow the user to graphically browse the web and saves them from having to see the raw HTTP that is being transmitted with each and every request.

So how does this relate to my part of the world? Well at present I’m writing a metadata file system (the sort of thing you do in your spare time…no really!) and part of its interface is utilising the file system to provide the system interface instead of writing a full API to manipulate the metadata part of the filesystem. What I’m using is a combination of the POSIX listxattr, getxattr, setxattr and other related functions. The benefit of this will mean that the system will work in locations where this is supported and will appear as a ‘natural’ part of the filesystem. This means that applications can start to take advantage of the extended attributes within the file system without having to worry about compatibility from one file system to the next (you can enable extended attributes on ext3 though they’re not structured like they are with the MDFS project). 

So starting off simple and making a flexible interface allows for the system to deploy more advanced systems at a later point without compromising on short term functionality. The development and debugging of a GUI application is harder than testing concrete scriptable interfaces such as checking that completing certain operations behaves the same way in two different circumstances, however it is easy to move from the simple interface into building the more advanced one.

As part of my final year of studies at university I’m working on building a metadata filesystem, called “MDFS” or “MetaData FileSystem”. Creative naming don’t you think? The project is using FUSE to create the filesystem interface and PostgreSQL is used to store the data in the back end. Its in its early phases, but you can check it out here at a Trac instance I’ve created: http://mdfs.pasamio.com/.