Currently ePrints lists authors by using the creator field of an ePrint. This is usually entered as the cited name of the author. If you have a small number of authors with different names this works reasonably well as the system can differentiate between people without much issues. The problem comes in when people are cited differently or have very similar names but are distinct people. Then it becomes bad.

One way of solving the problem is really quite ugly: you go through and change the cited author to make it look “neater”. In fact one of our library staff decided that they would do that greatly damaging the value of the data stored with in the system, tainting it horribly and creating more work for themselves. The cynic in me wonders that they’re just creating more work for themselves later to go back through and fix it all up. So there is a solution, non technical and pretty ordinary.

So our solution was to turn authors into a first class data set within ePrints. They’re not some unidentifiable free text field with no great purpose (there was a creator ID field but email address was suggested to be used for it) but in fact they are now their own entity which is cool.

We’ve done some different stuff with the author ID project at USQ that I’d like to share:

So an author has a name, they can be linked back to a user ID, they have an email address (both primary email address which should be used for contact and alternate addresses for aiding search), a biography and external identifiers. A cool feature of the primary email address is that when it is updated it automatically gets copied to the alternate email address list,  so you can easily update an email address and still retain the old one (you can still remove them from the alternate email address easily). The external identifiers section provides the ability to list useful identifiers for the author that mean something outside of ePrints. This could be stuff like a staff or student number that you record against the author to help with recognising your users. This is also a new field type developed called ‘Name-Value’ to represent name value pairs of data and provide an editing interface.

But the problem with authors is that they don’t stay the same: they’re dynamic. So they might move around an organisation, shift faculties, join a research centre or even leave the organisation. Recording this in the author record is more than possible but it starts to get messy – we start talking about ‘versions’ of the author but versions imply the wrong concept. A researcher might be working on papers in two different faculties, each paying for his time and each deserving of recognition. In our case we also need to use that information for reporting so that we can properly allocate funds based on research outcomes as well – for us it is important to know who paid for the research.

So to solve this problem I created a system called “author instances”. Author instances provide a subrecord of the authors that permits instances of specific data to be created and associated with the author. Then when an author creates a paper a link is created to this instance and not directly to the author. This allows the author record to contain information that is relatively static and data that might periodically change (such as department or even which institution someone is at) is then located in the author instance. As this isn’t ‘versioning’ per se it also works well for when a researcher is concurrently producing papers for different parts of the institution (or even for bodies external to the institution).

This screen depicts the edit view for an author instance.

The author instance is very light by default, it only contains the author that it is linked to, its display name and a preferred flag. The preferred flag is mutually exclusive within author instances – if one instance for an author sets the flag then it is cleared for all other authors. In the screen shot you will notice an untranslated field called “deptid”. This is another new metafield that has been developed called “Externalitemref” which is similar to the built in ItemRef field however it works on tables managed outside of ePrints. ExternalItemRef takes the params of a table name, a key field and a name field to operate in a similar manner to the ItemRef and link data back in.

The observant will notice that both the ExternalItemRef and the built-in ItemRef field also have a find entry button. This button has also been added to the ePrint item editing screen as well and triggers a popup window which allows easy searching for the particular item (user, author, author instance) that you are interested in.

The standard ePrints creator fields have a "Find Entry" button as well.

So as we can see there is now a “Find Entry” button that pops up with a new window allowing us to select an author instance for this author easily and not have to worry that we’re going to mistype their email address or similar.

This is just a quick introduction into some of the changes that the Author ID system which USQ has developed provides. Ideally we’d love to feed this back to the ePrints core so that everyone can use this, or a similar system, in a great way. If you want to read more you can check out the “Author ID Notes” document on my knowledge base download site.

My solution was to use token login to handle the actual request. The process is a tad convoluted, but the diagram should explain how the system works:

So the user logs into a remote service, which proxies a username to the Joomla! which is picked up by SSO HTTP in a custom component which generates a token using the detected username, redirects the user to the token login component (not through the proxy) which then lets the user log in and continue through the site.

So what we need to create is a new component to handle detecting the user, creating the token and them redirecting them. We can use the SSO HTTP plugin to handle detecting the remote user perfectly fine (this has already been tested) and the Token Login component can handle validating tokens and redirecting users.

Step 1: Prerequisites
I’m going to use the JAuthTools packages available from Joomla!Code. To install these, you will need to install the Advanced Tools package first before you get started. The latest version is Advanced Tools 1.5.1 available from JoomlaCode.

Once we’ve got the Advanced Tools installed, we’re going to need the JAuthTools 1.5.4 Core package. This will give us the SSO library that we’re going to use later on. We will also need the SSO plugins package (for the SSO HTTP plugin in our case) and the Token Login package. As we’re not using the user source plugins for this case (both LDAP and Session are provided by default) we don’t need to install them and since we’re going to write our own SSO detection component we don’t need to use the options offered by the SSO package.

Once we’ve got all of the prerequisites installed we can start building the component

Step 2: Building the component
So we’ll start building our component with the basic few lines all components require:

This basic check ensures that we’re within Joomla!’s confines or kills the execution if we’re not. From here, lets set up some basic variables:

These are two variables we’re going to use later. The landing page is used for token login to redirect the user to a page once they’ve been authenticated successfully. The autocreate option is utilised by the SSO system to determine if it should automatically create the user. Keep these in mind as we’ll use them later on.

In this next phase we’re going to take a copy of the current user and then use the SSO system to attempt to do the login. If its successful, the user’s ID will change and we can then create a token for them and redirect so they can login, if not we’ll redirect them with an error message:

So we get a reference to the user object, get a copy of the user ID, import the SSO library, create a new SSO authentication object and handle authentication. We see the $autocreate variable we initialised before used here. We could load the plugins ourselves and handle everything but the doSSOAuth call will handle this for us, create the users session and if requested attempt to create the user if they don’t exist. We can then check if the user ID’s are different and then create a token to direct the user back:

A sizable chunk of code. So we check if the user ID has changed (typically from unauthenticated to authenticated, we could also just check if the user ID != 0 as well). We then import the token login library, grab a copy of the database object and then create a new token.

So with the token, we set the username to be that of the currently authenticated user (which we know is correct), set the expiry to be the present time plus an hour. Since this data is only ‘temporary’ we don’t bother with JDate and just use the server time. If the server transitions to or from daylight savings there could be an issue but since the user should be redirected rather quickly that seems like an unlikely possibility. We set the number of logins that the token is valid for, which for this case is only one and then we set the token’s landing page to be the value of $landingpage that we set before. Once we’ve set all of the data we call the ‘store’ function to save all of this to the database which will also trigger the generation of the login token.

Since the session we’re currently in is invalid, we destroy it for a bit of extra security and then we redirect the user to the token login component. Since this request will get proxied as a redirect to the real site (hopefully the proxy doesn’t rewrite the redirect header coming back otherwise this will fail), token login can then authenticate the user (again) and create a session for them without the proxies’ interference. There is however one last part of our component, which is the else case for if the user doesn’t get detected:

So if we don’t detect anything, redirect the user to index.php with the message ‘Invalid SSO Request’. Too easy almost!

So for reference below is the final completed component file, don’t forget to update the values of $landingpage and $autocreate:

Now the original design of Experts Exchange wasn’t too bad. You could ask questions if you had enough points. You could also assign points to different questions increasing in value for importance I guess. You acquired points by either paying or by successfully answering questions. The thing that annoyed me was that if you weren’t the person that was nominated as the one who answered it you got no tangible credit for your contribution even if it helps or even if the correct answer was actually wrong or perhaps not the best response.

But obviously at this point they feel that they have enough knowledge to justify not only spamming their pages with tonnes of ads but also starting to force people to pay for even more. And be aide they’ve been around for a while and have had a good reputation they’re using this plus close keyword matches on the question to continue to drive traffic.

So now with Google’s Search Wiki, we can fight back against Experts Exchange and it’s pointless entries in Google’s index. All you need to do is be logged in and when you see an Experts Exchange result in your Google search make sure you delete it from your results. My belief is that if we get enough people to blacklist and delete those entries, Google will take note and eventually lower the rank of the entries and we’ll stop seeing their results.

The Kerberos keys that I had asked to be remade were ready for me by the time I got there. It took a bit of time to rebuild the different keytab files to support the vhost environment (need to merge the respective keytab files) but once that was done everything was working. Well, mostly working. Firefox on my Mac worked fine, Firefox on the Windows desktops I tried worked when they were configured (see http://grolmsnet.de/kerbtut/firefox.html for information on what you need to do to get Firefox to do negotiate), IE on most of the desktops worked fine however some installations weren’t getting SSO, all of the Citrix servers seem not to pass through authentication (they end up going in a weird loop where IE appears to keep loading the page) and Safari on my Mac doesn’t seem to want to play the game either. Perhaps I’ll sort that out over the next week or so but that consumed a reasonable amount of time going through and checking different IE versions and if they worked. The only machine not to play the game seems to be Firefox on my Linux desktop (it should be working) so I’ll have a look at the ones that don’t work and why they don’t want to work. For the Windows boxes I have the feeling that the Netware client is causing issues (which would explain Citrix) so hopefully when our network eradicates Novell we’ll be fine.

And that leads us to the afternoon’s fun of building Joomla! 1.6′s backup system. I’ve managed to get the system to export the sample database, reimport it and then delete the files afterwards so I’ve moved onto much larger goals. I’ve taken one of our internal websites and I’m trying to get it to important. Suffice to say that it has enough data to cause an issue with the system. For data loading I’m using a heavily modified version of Alexey Ozerov’s “BigDump” script, which has been used in the past in a less modified form for the Joomla! 1.5 migrator. It is slowly being converted to use the new Tasks system in 1.6 which is another concept borrowed from the 1.0 migrator. The Tasks system in 1.6 has two items: a task set which is a container for individual tasks. So considering backups, one task set might be a full backup run of the site with individual tasks being an SQL backup, a file backup (tar archive perhaps?) and maybe copying that to a remote FTP site or similar. So the one task set would have an “SQL backup” task and a “file backup” task. Extension package installation may do a similar item as well splitting the install into different parts.

A new part of this is the data load system that provides functionality to read and load data files, at the moment only supporting SQL but I’m hoping I’ll be able to create a CSV one as well some luck, again probably reusing Alexey’s code in part here as well. I’m mostly through building parts of this system though I’m experiencing a strange issue with my sample data (hence why the updates haven’t been committed to J!’s SVN repository today) where it loads the file up through to almost 2000 queries and seems to stop suddenly. I’m not quite sure whats going on but I’m happy enough that the task system is picking up and storing values for it to progress as far as it does.

Another successful day spent on my Mac as well, NetBeans doesn’t seem to want to look at my project any more crashing instead of loading it which is disappointing but I’ll work that out another day. And now its time to enjoy some Dawn of War.

What triggered a lot of this is a set of rather strange bash scripts that form the build tool for Joomla!. It handles building the packages, exporting the packages and building the patch packages (diff of new files and updated files from the old release). Wilco always says that we should use Phing instead because it has all of these features. So this project called JAuthTools that I work on started becoming big and it was a pain to build things for it by hand so I decided that now was the time to get into Phing.

Phing
Phing (http://phing.info/) is as stated before a PHP based build tool. Its web site says it can do anything you can do with a traditional build system, which is really kind of your base expectation when you think about it. Why would you replace your build system with something that provides you less? Phing comes with by default all of the nice things you’d want from a build tool, tasks to do things, output things, make directories, move and copy things, call other tasks and a few other nice things.

Problem is that what I’m after is something to build tarballs, and these are marked as optional tasks for the system. This has some cool things like DbDeploy, which connects to a special DB and builds SQL delta files (version control for your DB), coverage analysis, PDO SQL executor, PHPUnit and my personal favourites, tasks for Zip, Tar and Subversion.

So I went to install Phing on my work box. Its a Linux machine so I decided that I wanted to avoid the PEAR path because that is just as good as installing things all over my system. Phing lived happily in my personal bin directory until I realised that to get SVN working I needed to get this thing from PEAR. I decided that I might as well fight PEAR (and my work proxy for that matter) and install it that way. It only took a few attempts to get PEAR to work through my work proxy, it appears that between the last time I was fighting it they fixed it up which is nice. CPAN is another one that is a pain as well because it wants to pull things left and right and my proxy decides that half of the files it wants are too big and it should ask for them after 5pm instead or use a special download request system. But I digress, Phing ends up being mostly unhappily installed via PEAR with a few complaints (I said, “yes download your dependencies” which bombed out when it figured out that its dependency was still marked unstable and refused to install it (so why err bother? why not ask me! I want to install it, I even said so, that is what “alldpes” means right?)). So now I have a SVN capable Phing! Shiny!

So I go off and build all sorts of things and suddenly the installer rejects the package I just built with Phing complaining its an invalid zip file. So I ended up by working around this with a Bash script that basically rebuilt the ZIP archive after Phing was done with its business. So I progress from Bash to…err…Bash via Phing! This is on a PHP 5.2.5 latest build so I’m not sure where the fault for this lays, but if I end up using the old tool then as far as I’m concerned it isn’t a replacement for my build system.

The next thing that I ended up finding is that the nested dependencies appeared not to be fully supported by the tool, which didn’t particularly bother me however as you’ll read later Make supported this perfectly fine. So I ended up having to alter my build file to handle Subversion check outs better because it would check it out for each sub dependencies. So I would have a top level one and then sub-dependencies so that each individual item wouldn’t have to do it themselves, especially for the targets that matched the same repository. I ended up just moving this into a common parent dependency and going from there.

And then I wanted to do some work on my Mac at home. I have a slightly out of date PHP, 5.2.0, which appears to have caused issues. This time I went straight to using PEAR to install things and again this caused issues, but given I was expecting some hassles I managed to get things working. I also don’t have a nasty proxy or firewall so that makes life easier for me as well when installing these things. I tried out my first build script and it spewed out a horrible amount of errors. For some reason on this combination SVN doesn’t want to work which dies silently which then causes the rest of the script to fail because it expects the SVN task to succeed and given I’m using a build tool to ensure that errors get properly trapped it doesn’t bail out or check that directories exist like I could do with a shell script.

Whilst this all sounds bad I did however like one feature that allowed me to control what the zip task (or selected other tasks) included and excluded which made things easier for building packages and controlling operations that are slightly harder in bash or make.

Make
Today I started off making a few new scripts on my Mac to replace the non functional Phing scripts. It wasn’t too hard to go from Phing to Make but I ended up re-expanding out my SVN scripts now that I have a system where the dependencies work better. Strangely enough my zip files also started building properly which was nice to have. But this got me thinking: why do we reinvent the wheel?

Sure it is nice to have a fancy XML file that takes a lot of control away, but at the end of the day when I compare what I wrote to get SVN to export in Phing and what I have to do with Make or BASH, on sheer number of characters I end up finding that Phing loses again.

But here is the really simple thing: Phing’s SVN support is a wrapper around the regular command line SVN client. The advantage of using Make or BASH is their portability for myself. My primary environments are Mac and Linux, and when I have been on Windows for extended periods of time I ended up installing Cygwin. So at the end of the day my build system is portable between systems. Additionally the “Gotcha’s” page for Phing shows that it isn’t really a walk in the park to get it to work on Windows either. The question is that do you really want to play with your build system each time you want to change platforms or do you want to go and get up and go without too many issues.

Make and BASH are both staples of the Unix environment and have had years of testing. Whilst it is nice to have something with a whole heap of new features, the fact is that some of the basic functionality doesn’t work properly or is a pain to get working across platforms: I ended up replicating these features anyway.