To be honest I’ve seen this tactic before where vendors lock you into a given platform. And realistically Apple is already there in a sense. They limit your ability to distribute iOS applications and limit what you can include in their store. Anyone expecting Apple to behave any differently in this particular situation is delusional.

Fundamentally you need to think about what iBooks Author really is providing: it’s a typesetting tool. It’s not targeted towards print like InDesign or Quark, it’s a tool designed to highlight the capabilities of the iPad platform. But essentially it is a typesetting tool in a world where you’re not limited to static content but can build interactive content. And at the moment no other tool really exists to provide this functionality because the platform really hasn’t existed yet. But that doesn’t mean someone can’t make a tool that does do all of this just like when the iPhone was first released the smart phone market didn’t quite have what the iPhone offered. There were products that had portions of the system: many platforms had installable apps, many platforms had web browsers, a few platforms had large screens for input and many had phone and data capabilities. None of them quite had everything that the iPhone offered which is what made it successful.

Now what is being missed here is that effectively the results of what you typeset in iBooks Author is limited in where you can sell it. What Apple suggest is that you can develop your content in another tool such as either Pages or Word and then finalise your typesetting in iBooks Author. You can obviously produce your content entirely within iBooks Author but you don’t necessarily have to do this.

However at the end of the day it is this strange sense of entitlement, this sense of how dare Apple seek to limit what you could potentially do with a bit of software they have given away for free. This reminds me of people lamenting that you couldn’t write iOS apps on Windows and that Apple should port it.

At the end of the day it is their ecosystem, if you don’t like it then leave. Certainly this isn’t the first time Apple have taken draconian steps to limit freedom and it likely won’t be the last. It also reminds us that we need to read these agreements we agree to on the sides of the Apple Mac App Store. All of the Apple developed apps I’ve seen on their store have all had an accompanying “App License Agreement” so I wasn’t surprised to see this one.

Audacity in putting the terms there perhaps, however it is hardly unprecedented. Perhaps the audacity is the expectation that a for-profit company known for their tendency for draconian restrictions on their software and platforms would behave any differently when creating a new free tool to create content aimed towards their platform. What is really more surprising that they didn’t create an entirely new proprietary format…they just added some stuff to an existing one (not quite as evil but certainly up there).

To use the bundle, download it from my consulting knowledge base, extract the zip and then double click the bundle to install it using Sequel Pro.

Once you get into Sequel Pro go to a query view or a content view for a table to get a result list. In the result list select your rows, right click and select “Bundles” then “Copy” and “Copy as DB Unit”. The generated data will be put on your clipboard and you can paste it into your favourite application.

The output looks roughly like the following:

<table name="articles">
	<column>id</column>
	<column>fileid</column>
	<column>tag</column>
	<row>
			<value>1</value>
			<value>1</value>
			<value><![CDATA[testing]]></value>
	</row>
</table>

Text values are automatically wrapped in CDATA tags and numbers are straight outputted. I haven’t done a lot of testing with this but for quick and simple creation of the XML files it’s coming in handy for myself. So I’m sharing this and hopefully it will come in handy for you!

And the story begins around December 2010 with a defective iPhone 4 and Exchange. I had attempted to forward an email out of my work’s Exchange mailbox to a system administrator I was sitting next to from my iPhone. For some reason at the time this didn’t work quickly so I tried to forward it a few more times from my iPhone. It didn’t turn up at the other end and I forgot about it. Over the next few days I noticed that my phone was warmer than usual however I couldn’t work out what was going on. I made sure any application I could think of that would use data was terminated and I even tried to restart my phone at one point to make sure it was clear. I noticed that it was continually transmitting data but it continued to do that even after a restart of the phone. What’s going on?

I ended up in the mail application and noticed that it had a messaging saying there was one unsent message. I jumped into the outbox on the phone and saw that there was an error against the message that I’d been trying to send earlier in the week. I nuked it and watched my data usage stop increasing. By this time I’d received a wonderful SMS from Telstra letting me know that I’d gone over my limit. Was frustrating to be told I’d gone over my limit without knowing why it had happened.

Having figured out what was the root cause of the issue, I contacted Telstra. Unfortunately the fellow on the other end of the phone informed me that until I had a bill they couldn’t do anything for me. They did however give me a reference number and said that when the bill came that they would waive the data charges. Suffice to say that when the bill came, I called back and the person said the data charge had been waived. It turns out that they hadn’t waived the charge and a month later I called back and actually got some progress this time. It is disappointing that I had to ring three times to get it sorted however at all times the experience was positive and supportive.

In the mean time I had filed a bug report with Apple (actually before I contacted Telstra and I provided that bug report number to Telstra so that they could verify it if they needed it). I worked with the World Wide Developer Relations team to triage the issue and provide log files. I conclusively managed to nail it down to an issue with MobileMail’s background thread that would continue to resend the mail even though it was failing. The error seems to be a part of the ActiveSync protocol and there are some funky results returned.

As an aside, a month later or so I noted an article complaining about Windows Phone 7 chewing data from CNet (http://news.cnet.com/8301-10805_3-20030159-75.html). The article for the most part blamed Yahoo! but I also noticed something very interesting at the bottom:

Microsoft has also identified an issue with our Exchange ActiveSync [EAS] email synchronization protocol that may affect a very small percentage of users.

• This issue can be caused by email that exceeds the mail server email size setting or is sent to an invalid email address
• This issue can potentially affect email services that utilize the EAS protocol – Outlook and Gmail for example
• The immediate mitigation is for users to delete mail that appears “stuck” in the outbox
• This is currently scheduled to be fixed in a near-term end user update from Microsoft

Now the email in question that I was using was nearly 8MB with a 6MB PDF file as an attachment. I think that fits into some of the mail size settings. The service in question was the Exchange Active Sync protocol as implemented on the iPhone. The mail did appear to be “stuck” in the outbox and the iPhone kept sending it and apparently there will be an update from Microsoft. I’ve gone through again today and reproduced the issue on my iPhone4 running iOS 4.3.5 which is Apple’s latest stable release. In doing this I noticed that the phone was sending nearly 1MB of data each minute.

So now this happened another time and given my last experience I wasn’t worried about it. The last time it happened while it took three calls the matter was resolved relatively painlessly. I didn’t call when it happened because last time it didn’t make a difference – and that wasn’t necessarily Telstra’s fault the first time. However eventually the bill came and curiously this time Telstra sent me an SMS informing me of what I’d already known – I’d gone over my data limit. This time it gave me a message suggesting that I call a particular number.

I’ll be blunt and say that I was actually really excited and impressed. Telstra was actually making a positive move to get an issue resolved pre-emptively. I presume this was set up as an automated feature however someone put some thought into it somewhere. That, tied with the last experience, had me hopeful that I’d have a great experience. Oh how I ended up being wrong.

I rang the number, got through and explained what I wanted to happen and was told “You’ve rung the wrong number, I don’t actually handle that.” Wait, what? I rang the number your SMS’d me to suggest to ring for this particular issue. And the first contact I have informs me I’ve rung the wrong number. Someone hasn’t communicated properly internally. Anyway, the fellow gives it a go and is out of his depth. He tells me there isn’t any problems with the iPhone. I have two open issues with Apple for the iPhone (one of them being this one) as well as a few other that are marked as duplicates. I’m well and truly certain there are faults with the device, so someone on Telstra’s Sales team has drunk more of the cool-aid than I have. But I digress, I tried to explain that this had happened before and that I had an issue report with Apple. The fellow has a chat with his supervisor concerned he’s not going to get a commission from this call then transfer me to the iPhone department assuring me that he’d communicated my issue to them.

So the iPhone person gets on the phone and starts explaining to me how to turn off the cellular data on my iPhone. I try hard to treat Telstra’s customer service representatives with respect, inform them of relevant bug reports and even reproduce the issue and what I get treated like an idiot. At this point I felt that I’d met my obligation to attempt to contact the provider to get the issue resolved and they wanted me to disable cellular data. Go figure.

At this point I put through my complaint to the TIO. After a bit of a false start I got a contact with Telstra’s customer service representative and we discussed the issue. It started get back towards the sort of positive customer relationship that I’d like to have with my telephone provider. Unfortunately Telstra’s billing system continued to pretend that nothing had happened and even managed to send me a suspension notice for my account after the overdue notice. So kind.

Anyway, fast forwards another month I’d been wondering that I should have a bill from Telstra and it arrived earlier this week. I got back on Wednesday and opened up the bill to find that it noted that I had an overdue amount of $66. I went through my records and couldn’t find an invoice matching that amount. I did find that overdue notice and suspension notice so I almost wonder if Telstra’s billing system missed sending me a bill or if it got lost in the post. In any case I rang Telstra to ask about it and got them to send me a second invoice. They informed me that my last bill was apparently $220 or so and my last payment was a couple of days before the end of the billing period which doesn’t make sense. Today I get a wonderful overdue notice letter again demanding I pay the uninvoiced outstanding amount. Great. How about including the outstanding invoice as well when you do that Telstra plus any other additional invoices that might have been sent. Let me say I’m less than impressed, particularly when I saw that this bill was the first to note a charge reversal. My TIO complaint was filed on June 8th. That means that its taken them two billing cycles to get this correct (again). I don’t think I should be surprised. However I do find it rather arrogant that when it involves a debt to myself it takes Telstra two months to resolve the issue but when they want to extract money from me they don’t let it exceed more than a month before they’re sending nastygrams.

After all of this I’m curious if Windows Phone 7 or other Microsoft based operating systems that utilise ActiveSync have similar problems. I note that my Outlook desktop client doesn’t have this issue and successfully forwarded my test email. Android would also be another interesting test case to see how it behaves to see if the issue can be replicated there or if it is truly a bug with iPhone.

Sooner or later I’ll get all of this resolved though I’m soon to be a happy non-customer of Telstra. Not that Optus is better and Vodafone certainly isn’t an option but fortunately I’ll have an entirely new set of providers to be upset by in a weeks time.

In my many searches for a Mac OS X Lion compatible driver, I managed to find the following from mobile provider “3″ in UK. The link, http://ask3.three.co.uk/mbbdocs/drivers/apple_drivers.html, contains a link to “dongle drivers” for “Huawei dongles” that work for Mac OS X Lion (10.7). Sounds like a winner!

So I download the zip file and it extracts out to leave me with a “3Connect Installer”. Not quite what I want however all is not lost. You see applications on Mac OS X are actually folders in disguise commonly referred to as either “bundles” or “packages”. So right click on the “3Connect Installer” and select “Show Package Contents”. This opens up a view with only “Contents” listed, foiled? Don’t worry, double click into that directory and then into the “Resources” directory. Here we find a “mobilemanager.mpkg” file. Now “mpkg” is a package which means it has more goodies in it.

So we right click on this and select “Show Package Contents” again. We’ve got our “Contents” folder and inside that we want to go to the “Packages” folder. In there two packages exist: “mobile_manager_application.pkg” and “huawei_universal_425_05.pkg”. Now if you’re like me, everything is setup already – you just need the driver. So I skipped the mobile manager application and just double clicked on the “huawei_universal_425_05.pkg” file and installed it.

After installing the Huawei Universal package, the next time I plugged in my Huawei E160 into my Mac it was detected and worked perfectly fine.

So if you’ve just upgraded to Lion and are trying to work out how to get your Huawei modem working properly with it, those instructions worked for me and I’m using it to write this post.

Platform

So to begin with Titanium Appcelerator is a platform for using JavaScript to build native applications that run on iOS, Android, Blackberry and a few other platforms. If you’ve ever written any code for Java, then the style of coding that you use here is very familiar except the language is JavaScript. JavaScript is a curious language complete with it’s own wonderful set of quirks.

JavaScript is one of those languages that is everywhere. JavaScript is the magic sauce that makes the web go and it isn’t without it’s set of warts. Perhaps the most annoying part for me is the fact that scope is excessively awkwardly defined where you find these situations where a variable is in scope when you really didn’t expect it. This has lead to all sorts of curious patterns in JavaScript to work around these features.

So Titanium uses JavaScript to build a native UI for Android, iOS and BlackBerry. Unfortunately (fortunately?) I didn’t have access to the BlackBerry compatible SDK though with that platform doing it’s best to shoot itself in the foot I don’t think it is too much of a concern. This leaves Android and iOS which were the two platforms that I developed and tested upon for this particular project.

Being a Mac person I have a bit of a bias towards iOS. This is also borne out by me owning an iPhone which means the platform is natural to me. Android ends up frustrating me no end with its flawed and missing support for items. A great example is the constant battle to connect to my university wifi network. I had managed to get it working last week with our testing Samsung Galaxy S but this week it stopped working. Apparently the network is broken however I haven’t had any problems with my iPhone which still appears to connect and work like normal. But I digress, Android seems user unfriendly – or at least unfriendly to me.

So I set up Titanium Developer on my Mac. I have various versions of XCode and the iOS SDK installed already so I had to acquire the Android SDK. That was relatively easy where I downloaded the base applications and then installed all of the SDKs. The important thing, that I found out later, is that the normal API’s aren’t sufficient: you need all of the Google APIs. It turns out to be a lot of data so you might want to be selective for a particular platform (Android 2.2 + Google 2.2 API’s). So that all gets installed and for Android you tell Titanium where it is (XCode and iOS lives in /Developer and is easy to find).

From here you create a new project. This is relatively easy though the pitfall is that it automatically creates a directory for you and if it doesn’t feel you have a particular platform installed it’ll not set up the project to handle those platforms. It turned out at one stage to be easier to recreate the project and replace the JavaScript files.

The next challenge is to get your program up and running in the Android emulator or iOS simulator. This turns out to be easier said than done for Android and often less than 10 seconds for iOS. In my timing Android’s emulator almost always took minutes to start the emulator. Some times the emulator wouldn’t load properly and other times Titanium Developer wouldn’t recognise that the emulator had successfully launched. In these cases you ended up terminating the emulator and starting again. This ends up taking minutes to launch. Eventually the emulator gets itself up after a few minutes (or many more if it fails) and then the application deploys. You find a bug or make a change then want to re-deploy and test on the device. So fortunately for Android you don’t have to restart the emulator, it simply redeploys the application however even this process takes over 45 seconds! This means there is over half a minute of deployment difference between Android and iOS.

Because of this I started heavily developing on iOS and not bothering to test on Android. I didn’t feel that I had the time to waste waiting for Android to launch and be available to me as a developer where as iOS in some cases would be there in a few seconds ready to use (10 seconds at the most). It even kills the simulator and starts from scratch with a clean launch.

The problem that you then end up with the situation where I developed for iOS but didn’t test it on Android. I had faith that if it worked properly on iOS then it would still work fine on Android presuming I didn’t use any “platform specific” functionality. The problem is that it very quickly gets to a situation where everything doesn’t work properly on Android and it requires lots of time to fix the issue. Cross-platform? Not really.

Some of these issues arise from basic differences between the platforms. Android features a lot more hardware buttons including a “menu” button which can popup a menu over the screen to click. The menu is a great way of hiding stuff away and removing it from view. This is important when you don’t have much screen space to begin with however coming from an iPhone perspective where everything is visible on screen some how (even if it is a menu option that hints at more possibilities) this turns out to be rather frustrating as you click through almost every screen until you get to the point where you need to click the menu to progress. iPhone presents everything on screen with hardware buttons to change the volume, a mute/orientation lock toggle, a lock/unlock button and a home screen button. In iOS5 the volume control will also be a shutter button via software control. The flip side is almost anything you want to do is available on screen. There are a few exceptions around pressing down and holding parts of the UI (particularly with text, text boxes, the keyboard and links) which can trigger an extra popup/popover to appear however the norm is very much to have it visible on screen. This means that iOS has a strong UI interaction based around either tabs (bottom of the screen), the “navbar” with back and forwards buttons (top of the screen) or both.

When using Titanium we run into complications. On iOS you could use a tab group and the navbar is free. It automatically puts in the back buttons and the next button (“rightNavButton”) needs to be delivered but all in all not too bad. On Android there are tab groups but no navbar which means you’re using menus. Menus aren’t available on Android but give more UI flexibility than on iOS through having more options. The trap however is that menus and tab groups don’t work together on Android. I’m not sure if this is a Titanium issue but all I know is that it doesn’t work properly when using Titanium. The extra trap is that tab groups and the navbar works fine for iOS applications launched from Titanium’s main context (“app.js”) however on Android you can’t use a menu here. So if you want a menu on the first screen in the app you need to open another window first to create an Android activity to then associate the menu with the window.

And this in itself reveals another slight difference. There is the concept of “heavy weight” and “light weight” windows. iOS doesn’t appear to make any distinction but it very much matters on Android. Another Android feature is the hardware “back” button which permits the user to go back through the interface like a stack. On iOS this is implemented as a top left button on the navbar as a convention however it complicates Android because you end up with the two types of windows. Heavy weight windows create a new context and can be an anchor while light weight windows are ignored. If you open a heavyweight window, then a lightweight window and then click the “back” button the app doesn’t go back to the heavyweight window but to the one BEFORE that one. This may even result in the app quitting. The work around ends up being that all windows have to be heavyweight to prevent a click of the back button going somewhere it shouldn’t.

So once you start working around these platform specific quirks and features you end up with the bugs. One bug I encountered was that setting a top and bottom value for an item would work great on iOS but on Android those values appeared to be added to the parent view or window pushing it down. This perhaps became most obvious when I added a multiple text area option which caused the following undesirable result:

Clearly this isn’t what was intended however the same code works beautifully on iOS devices. I’m not sure if it is a case that I’m doing it wrong or what but fundamentally it is a mismatch between the two platforms. In some cases the problems were that I was using the API wrong and each respectively platform is more tolerant of the others though this only trapped me once or twice. In one situation I was using a platform specific call that wasn’t guarded so it wasn’t run on the other platform which resulting in a weird error that I didn’t fully understand until I took a closer look at the documentation.

And reviewing the documentation is important because on some pages you see a little Apple and little Android logo at the top of the page to signify that it is available on either however you also end up with situations where stuff is only partially available on either. This is distinguished by the text “iOS only”, “Android only” or rough equivalents (yes it isn’t uniform) after the name of the function. In some cases there are also significant behavioural differences. The Titanium.Geolocation.getCurrentPosition call will NEVER (emphasis from Titanium documentation) activate radios on Android instead returning a potentially very stale result. The documentation notes that iOS may activate radios depending if it considers the location data to be “stale”. This means that the most logical and obvious function call doesn’t necessarily make sense to use. The aim of using an abstraction platform like Titanium is that those platform specific differences are abstracted away. Perhaps adding a “maximum age” parameter to the function and then have Titanium automatically do the work needed to update the location details (it is possible by setting ANOTHER handler on the location event to get the position which is the work around I ended up using but that obviates the benefit of having the data cached and not causing a battery drain). Documentation in general is dodgy with this great example from the Titanium.UI.IPad page:

Yes, the page literally just trails off.

It is clear from using Titanium on both platforms that it was originally written with iOS in mind and Android has been tacked on as an afterthought. I’d hate to see what the Blackberry work is like though given the inconsistency between iOS and Android I’m not expecting the Blackberry experience to be much better again.

To make my life easier I constrained my platform to HVGA Android and iPhone. For what I was doing I deliberately excluded working on different platforms and screen resolutions instead hard coding values. Some of this was due to iOS bugs where setting “width: auto” or “width:100%” resulted in child items not receiving touch events on iPhone. This meant that shifting it to a larger screen (e.g. iPad) or changing the orientation presented much white space. However more worryingly I tried out the application on a Samsung Galaxy S running Android 2.1 and the app looks completely different. So this is a screenshot of the emulator running the application:

The layout looks reasonable on Android but here is how it renders on the Samsung Galaxy S:

The interface is almost unreadable shifting from the emulator to the Galaxy S. Fortunately clicking on a textbox and rotating the device yields a full screen edit interface which conveniently provides you with a next button but no indication of what you’re typing into (this isn’t a Titanium issue, but an issue with the Galaxy S or Android; I had a similar problem where I accidentally typed a password into the wrong text box in the WiFi dialog when it was asking for a second username not the password as I was expecting). Perhaps I need to do something more however on the iPad the application renders without issue beyond the immense white space:

This leads me to one final issue with the platform: there appears to be no way to flexibly build a layout using Titanium Appcelerator for multiple screen resolutions that works across platforms. For iOS this isn’t so much of an issue because there is only two resolutions at two orientations (four potential combinations for layout). This means that programmatically setting the values requires at most a combination of four options. Everything else is consistent across the platforms which is useful. However that isn’t the case for Android. Android supports multiple device resolutions at increasing amounts, particularly with the introduction of the various tablet devices. This can cause a pain and perhaps I’m using their layout system wrong with the offsets and sizing however it almost feels like a requirement given the cross section of bugs that I see. The simple vertical style layout work nicely however as soon as you need to mix up the layouts you’re out of luck. Again, perhaps I’m doing it wrong but the most obvious way of doing things seemed to work well on iOS but not so well on Android.

Tools

When I started this project I was using Titanium Developer, a relatively simple stand alone tool which created the scaffolding for a new mobile project, handled compilation and launching the simulator/emulator, deploying it to physical devices or packaging it up for release into the App Store or similar. Titanium Developer doesn’t provide an editor, it is flexible with what ever you use. I tried to use Apple’s DashCode however it kept getting confused and files apparently were being deleted as a part of the process which quickly upset DashCode. DashCode’s package layout also made it impractical to use in reality. I then switched to using Aptana Studio as an editor because of it’s strong JavaScript support. Unfortunately I wasn’t able to leverage much of this but at least there was some support for code formatting and syntax highlighting. I did try some mechanism to include the Titanium API but this didn’t work so well in practice.

The majority of the project was delivered using Titanium Developer and Aptana Studio. The combination worked reasonably well though not as smooth as perhaps the tool integration options that Microsoft or Apple provide – or even to a lesser extent Google with the Java for Android. There is a distinct lack of debugging support so you’re left with logging stuff as your primary method of debugging any buggy application. This means you don’t have a rich interface like XCode to profile and debug an application which is a great loss. The logging worked nicely however iOS 4.3 gave me grief and didn’t appear to work, targeting 4.2 got it back in action though. Logging consistently worked on Android even if I had to sometimes launch the emulator multiple times before Titanium would properly attach and deploy to it. I never had this problem with the iOS Simulator even if the application itself didn’t work. This leads also means that once the Android emulator is up and running you don’t want to quit it however periodically restarting the emulator can fix things. One case in point was when I was working on the location API code it would refuse to say it had an internet connection. I restarted it and everything worked fine there after but that didn’t save me spending an hour trying to work out why my code was broken.

Appcelerator have released Titanium Studio which is an updated copy of Aptana Studio (they bought Aptana) and includes support for their new 1.7 SDK. I didn’t see much of a difference in the SDK for what I was doing and all of the issues I saw with the 1.6 SDK was still present so I’m not sure what they changed however it didn’t break any code so that is a positive. Titanium Studio does feature autocomplete that works, continues their formatter. They’ve dropped support for JSDoc or similar automatic formatting for functions which is a disappointment (Aptana Studio would automatically populate the doc block for a function with the parameters for the function which was good).

For me Titanium Studio was a great improvement. I never once had an issue launching the Android emulator (it was still slow however) like I would periodically have when using Titanium Developer. Using iOS Simulator was still smooth to use as well. Apparently the pro version of these tools (e.g. not the community free download) offer debug functionality. The “Indie” plan is USD$50/month and the “Professional” plan is USD$200/month. XCode 4 with the latest iOS SDK from Apple is USD$5 – however you’ve already paid this as it is a requirement to test iOS applications. So you get a full blown IDE with a graphical layout tool, debugger, profiler and a whole host of tools already then Appcelerator want you to pay another $50/month to get debugging for their platform? You have to pay to get access to API’s like GameKit and StoreKit that you would already have access to normally through the standard iOS SDK.

Conclusions

The benefit for Titanium is to build cross platform applications however the reality is that this is tricky enough as it is. You end up in the valley between the two peaks where you have to deal with both platforms enough times that it isn’t smooth and you lose any advantages. iPad is an interesting case where you have another UI metaphor again with a split view that you would then need to develop an interface to handle the metaphor popular on that platform. This is supported by Titanium (Titanium.UI.IPad.SplitWindow) however we’re now up to a third platform specific UI that needs to be handled. Where does it stop? Blackberry? Windows Phone 7? WebOS? Eventually the application is held together by checks for each platform.

I can’t count the amount of time I spent doing circles on something stupid trying to Google for answers, pouring over documentation and waiting for Android applications to take nearly a minute to redeploy. It was frustrating to have stuff work and not work or then have to rebuild the application because one platform needed something one way and another platform wanted it a different way. Android is even more frustrating because even though I had a 2011 MBP, it still like it is slogging along as slow as anything. Building this app was time consuming as I would spend days in some cases trying to work out why something was mysteriously failing – particularly if it did it on one platform but not the other. The only other time I felt so unproductive was when I was building a Java GWT+GAE application where I ended up with boat loads of classes just to have everything work well (mind you that was more Java’s excessive verbosity than anything).

I believe that while the use of a cross platform toolkit is useful for having the same language regardless of platform (JavaScript), to get the best results the UI layer of an application needs to be platform specific. Even on the simple application I was building it was starting to get to the point where to push it much further would require a large amount of code to just handle displaying an appropriate UI on multiple Android devices not to mention integrating the UI split view and any of the code changes required for that (yet another if statement…). You might be able to get some code re-use here and there but I don’t like being forced to pay to have one set of tools (XCode) to then have to pay 10 times the cost each month (keeping in mind that you only need to pay once for XCode so far, not every month). Once the iOS5 SDK is out of beta, I’m going to spend some time to play with that instead and see where I end up.

Extra Resources

It turns out the best documentation on Titanium are the forum posts asking how to do things and if something is broken. I found a great number of undocumented “features” and “bugs” through reviewing the forums. Google is often your friend though sometimes a trip to the documentation can be equally as enlightening when you read that you’re trying to use an iOS specific feature on Android.

• Quick Tutorial on Navigation Groups (an informative blog in general)
• Use TableView Tips
• Concise example of how to do a menu on Android
• Enabling fake GPS on Android
• Basics of Layout – This is a great comparison of the layout system
• List of Titanium Mobile Events – a concise list of each event for each part of the system in one place. Also listed on individual pages but not always easy to check
• How to use maps on Android – my app worked without this on the emulator and dev environments, I have a feeling it only matters to packaged apps
• Example of how to use a tab group
• Example of Android menu with notes about windows, activities and menus
• Blog about handling CRUD operations
• Example of creating a nav group manually

Everyone pans Apple for this as their option. There is very little blog posts that appear to defend and support Apple in their stance of not having a native SDK and only providing a web “SDK”. Almost every bit of out spoken opinion out there seemed to point negativity towards the direction that Apple had taken. To be fair the initial release of the iPhone and it’s integration is primitive compared to what we have today with items like launcher integration being unavailable initially. However if you spend some time to review what Steve Jobs actually says it becomes more interesting. He states “we have been trying to come up with a solution to expand the capabilities of iPhone by letting developers write great apps for it and yet keep the iPhone reliable and secure”. It is this phrase that informs the future direction of the app store.

In 2008 Apple releases at first a beta in March with the announcement of the App Store with the first hint of “approval” for application. At WWDC2008 later in the year it’s finally released and the App Store is scheduled for it’s release in September of that year. In line with what Steve has said back in 2007, the App Store approval process and some rather strict guidelines for what they will accept. While some of the guidelines are arbitrary and awkward, they do appear to have successfully managed to limit the amount of malware and devious applications that have managed to make their way onto end user devices unlike Google’s Android Marketplace. Amazon with their App Store for Android have similar approval guidelines as well which seems to support Apple’s approach in part.

Since then Apple have increased their restrictions and adding requirements such as requiring in app purchases to at least be available via Apple’s own mechanism. Depending on what happens with the Lodsys case, it may turn out to be a case where you either use the supplied API or you have to deal with the patent issues.

However it appears that the Financial Times aren’t interested in trying to fight Apple at their App Store limitations and have gone back to 2007 and have developed a web app. It utilises local storage to make things faster on the device and it features a lot polish.

But for all of the noise generated about how evil Apple is, the Financial Times are the first group I have seen to actually put their money where their mouth is and released a HTML5 application that looms really good as the number of devices with Android on them explode, I am surprised that more people aren’t realising that web apps are the future of mobile development unless you war to loco yourself inside of apples iCloud. Web based applications also have the ability to target Android without requiring a rewrite and also any other platform that ships a decent web browser (e.g. WebOS or perhaps Windows Phone 7).

With over 300 different smart phone devices out there with a wide range of functionality, display sizes and resolutions. With native applications needing to target iOS, Android, WebOS, Playbook and Windows Phone 7 each with different code that is a significant development burden. With Windows 8 looking like it will implement HTML powered web apps then increasingly it is looming like HTML is not a bad place to put some skill development or professional development.

Perhaps it is back to Apple’s original much derided original direction – even if it was because their native SDK wasn’t available at the launch time of the first iPhone. It was even announced today that iOS5 will bring Nitro JavaScript, one of the speed ups to make web pages in Safari load faster, to the “homepage” apps of which the new Financial Times web app is now one. This means that with iOS5, the Financial Times will run even faster again.

But it begs the question, why would Apple make a mechanism that competes directly with their App Store run even faster?

VuFind UConnect SSO
VuFind is the library catalogue system and UConnect is USQ’s custom SSO system that was developed. I’ve gone through and built an integration between the two which looks like the following:

I talk a bit more about VuFind, SSO and authentication in an earlier blog post.

Book Finder
This is primarily the work of Ken MacAlpine who is working on mobile technologies at USQ. Ken developed an SVG powered map of the top floor of the library to aide in the location of books. He put together a database of roughly where the catalogue numbers stop and start for each shelf on each side. It has been integrated into the catalogue and shows up as a “Show on Map” link for Toowoomba Main Collection items.

At some point in the future we will work to expand it to all campuses and collections. When it is clicked it appears in a new window like this:

It is aimed to be the same size as an iPhone display so that it fits neatly there. It also works in all browsers that support SVG. Fallback support is provided for IE since it doesn’t support SVG yet.

ePrints, RPCS and data conversion!
As an interesting exercise I also spent some time converting data from an internal reporting system, RPCS, into our open access repository – ePrints. This was a fun project which required connecting to RPCS’ MSSQL database and outputting an XML file for ePrints to eat. To do this I figured I’d pull out Web Matrix, Microsoft’s cut down editor for web developers. It feels on par with Geany though it has a built in web server, request tracking and a database explorer. It also integrates with the Web Platform Installer product which means I can easily install other applications. I used Web Matrix to kick start the Joomla! install for the Access All Databases research project as well.

The RPCS data is loaded into our ePrints system along side the Author ID project that existed there. We’re also pulling across the data from their system which is going to result in duplicate authors until we merge them down to uniques however doing so in an automated manner isn’t easily possible. There is a whole heap of work to get things together but all in all we’re doing well.

EQUELLA Moodle module
Last but not least I’ve been playing with the EQUELLA Moodle module. It adds an extra option to the Moodle resource drop down and enables access to select items from the EQUELLA repository software.

I’ve used Web Matrix here to deploy out a quick Moodle site utilising the same Web Platform Installer technology and managed to get up and running quickly with everything. The deployment of the module to our main Moodle environment would enable lecturers to directly select course readings to embed in relevant modules on their Moodle course pages – exciting!

So this tells me that at least Android 2.1 level applications do not respect the proxy settings I had set up. I’ve obviously got something set up properly because the web browser users them. Looking around there seems to be suggestion that I should root the device to set proxies properly or something. For something that “just works” on an iPhone and it is something of a no brainer I would have thought it would have worked fine on Android, at least with Google’s own applications.

However the real nail in the coffin is the lack of complete 802.1x authentication, which while improving it appears that USQ’s current configuration makes it fail. USQ has started to deploy of 802.1x wireless networks to reduce the requirement to log into the wifi network, log into the Cisco access points and then from there gain access. This meant that the earlier wireless network testing was possible albeit a little tedious. Using it from an iPhone was just as tedious however there were apps to get around it. As of this week they’ve switched that network off and now Android doesn’t want to play the game.

The first problem with the 802.1x network is that USQ uses a certificate signed by AusCERT which doesn’t appear to be in the normal trust path for either Android or iOS. iOS gets around this by prompting the user to accept the certificate like what normally happens on a desktop web browser when an untrusted certificate is presented. One has to go through a rather complicated set of hoops to get this working on Android from what I’m told including putting the certificate on an SD card and navigating through a few layers of menu. Having the user download the certificates independently, find an SD card writer, put it on the SD card and then going through a process to import the certificate isn’t particularly user friendly. When you consider that USQ has something like 5000 people on campus so this sort of task doesn’t particularly scale well as opposed to clicking “accept”. Once we’ve gotten over that hurdle the second problem appears to be getting the Android device working properly with the AES encryption used. Apparently this is a barrier that an USQ ICT member with a masters degree is currently tackling though I believe that Android doesn’t support AES only PKIP so I don’t think he’ll have much luck.

Of course on iOS its rather easy, select network, enter credentials, accept certificate and then add proxy settings to the network. And you’re done. This was enough to convince one of our faculty librarians that perhaps the Sony Xperia X10 that they bought in the last few months was really not worth fighting against and now wants an iPhone. The X10 was sold with Android 1.6 which really just makes you wonder about fragmentation in the Android platform.

UPDATE: USQ’s certificates are signed by AusCERT, Australia’s version of CERT (Computer Emergency Response Team), not self signed as I had earlier suggested. AusCERT’s CA doesn’t appear to be in the normal certificate bundle for either platform. This means it came up unverified which is why I had assumed we’d been using self-signed certificates. The secondary issue appears to be the more secure AES being used for the authentication instead of the less secure PKIP. Android doesn’t support the more secure AES and thus fails to work even when the certificate is trusted or the signing CA is added. Additionally I’ve been told that you don’t need to root the device to install a certificate but you can put it on an SD card. These changes have been merged into the main document where relevant.

USQ also developed the Virtua driver for VuFind. Virtua, the library’s integrated library system (ILS), doesn’t provide a driver themselves but sells a reasonably expensive layer which integrates with Drupal these days. Originally such a layer didn’t exist and the system operates by a complicated mess of reading directly from database tables and posting stuff to arcane undocumented web interfaces through an even older web application known as “iPortal” with a binary named “chameleon”. Much fun!

So one of the things that has been desired for a while is the ability to integrate with the corporate directory service and corporate single sign on service. Unfortunately the vendor product is an all or nothing: you either have everyone authenticating from the directory or you create accounts inside the ILS which can authenticate. Due to the nature of our Library and the non-USQ patrons (e.g. people who pay to join the library or reciprocal borrowers) we have we can’t easily just pick the LDAP option. That leaves us in a position where we have to have our own passwords distinct from everything else which leaves a bad user experience.

Fortunately VuFind has a pluggable authentication layer. This means that we can supply our own authentication to handle things. Great! Except that the list of authentication plugins was hard coded. Well not so bad, but easy enough to fix by altering the system to have more flexible support for authentication handlers.

So by default we have an ILS authentication handler which works fine. And for those non-USQ users we still want to use that handler because it works great right now. But we also want to have another one that hooks into our LDAP directory and integrates there. Now I could build a new magical plugin that handles both the ILS authentication and the LDAP authentication however I figured that this was a bit much and decided instead to build an authentication handler which allows chaining of multiple authentication handlers, creatively called “MultiAuth“. This allows chaining and processing of the fields to ensure that they are consistently used by all subsequent child plugins. Nifty!

So now that we have better pluggable authentication support and the ability to chain multiple plugins we’re getting close to everything we need. The USQ situation is unique in that students are issued with a barcode identical to their student number. This makes them easy to process because we can map them directly from the LDAP interface into the ILS system. Staff however present a much trickier problem. Staff are created with a custom barcode and identifier which is distinct from their staff username and unique from the staff payroll number. This means that we need to dereference their username back to their ILS identity. Fortunately this was already done: an alternate barcode in a particular format stored their payroll number for cross verification. Sweet! In the not so lucky bucket was the fact that the custom barcode format meant that I could do a direct mapping still and needed to write some custom code. So with some modifications to the Virtua driver to pull out the alternate barcode and grab their real barcode to log the user in.

So at this point a user can login using the LDAP credentials or the standard ILS credentials. For relevant staff and students this means they’ve got two options but thats better than nothing. This is great but it isn’t single sign on yet because they have to log into the library catalogue specifically and that isn’t fun! So bring on the single sign on!

USQ has its own SSO system that was developed which means the standard mechanisms don’t work (bummer!). Fortunately our ICT department has some smart people and they’re also PHP friendly which means that there was already a library and sample code sitting just begging to be integrated. Surprisingly with the rest of the LDAP stuff in place this was relatively easy. The library did the heavy lifting of authenticating the user and returning their details so we just have to replicate what we had before. Students go straight through since their usernames match their barcode and we do a lookup with the staff payroll number to backtrack to finding their actual barcode and then logging them into the system and sending them on their way.

Fortunately this is all now in place on the live system which is up and running.

Until recently my mum who lives in a rural area didn’t have a unique rural property number for her new house. As such she used the one for the old house and the post man knew which one to deliver too. At one point she got a new number and I was trying to send her something. Curiously all of the systems attempted to autocorrect me to the wrong address – a now vacant house. In some cases (Queensland Motorways) I had to fight the system to let me have the address I knew to be correct.

My present situation is more interesting. I have two problems: I live in a new suburb and I live in a new unit complex. Some systems will complain that the suburb I’m in doesn’t exist and will flat out refuse to let me have it as a valid address. I find this a joke, I mean you’re saying that because you don’t know where I live you won’t let me set that as the address and would rather I set the “wrong” suburb just to suffice validation? In other situations they will let me have the suburb (yay!) but then refuse to acknowledge I live in a unit. In these cases the unit qualifier is dropped or suggested to be wrong.

The depressing thing is that my experiences ranges from three different levels:

• They suggest I have the wrong address but let me keep it as one of their suggestions.
• They suggest I have the wrong address and make me go through multiple levels of confirmation before they let me keep it.
• They don’t let me enter my correct address and refuse it.

What worries me is that this happens all together. The user entered their address and while having an AJAX powered suggestion box would be good as insulting your users by suggesting they don’t know their own address doesn’t help very much. I could see value in suggesting a better suburb match however in my case when universally the application is broken until it next gets an update from whoever it is that hands out addresses – straight out suggesting I can’t enter my address is your failure not mine.

This time they’ve changed tactic. Instead of guessing where you are they ask you what your post code is straight up. I punched in “4350″ knowing there isn’t a Bunnings within a good 50km or more. It then asks me which one of 44 matches I’m actually in. Your next nearest store is more than an hours drive and 80km away. The one after that is an hour and a half with 100km! IT DOESN’T MATTER WHERE IN 4350 I LIVE BECAUSE ITS ALL THE SAME! STOP WASTING MY TIME!

Harvey “it’s hard to get people to work in my stables” Norman doesn’t fare that much better with its location finder. Though after interrogating you about which suburb you live in it has the gall to show you just how far you are away from its stores. I couldn’t pick a location in 4350 that put any of the other stores closer than 60km, that included Athol which was 12km away from the store. I mean seriously, IF NONE OF THE OPTIONS GO ANYWHERE ELSE WHY WASTE MY TIME? I live in Toowoomba, it’s a small town. You’ve got one near by store for 4350 how about you just get along with displaying me the information that I need? Otherwise you’re just making your service more irrelevant. I could have just as easily found “Toowoomba” on the list of “Queensland” stores instead of this arduous affair about selecting which one of 40 suburbs I live in after I told you my postcode.

Location services are supposed to make my life easier and quicker not make it so I have to jump through just as many hoops of pointlessness.

Once you’ve got ePrints set up with a new repository, check out the /archives/repositoryname/cfg/cfg.d directory and look for a “datasets.pl” file. This is the first step in the process and documents two samples: one using a set of fields and a second with a class that you can derive from. In practice neither example works properly which is a bit disappointing. Fortunately the second solution with the class works with one minor modification: you need to implement the “get_dataset_id” function for it to work properly. This is relatively trivial to complete and you’re off on your way. A copy of the datasets.pl is included in a ZIP package linked at the end of this blog post.

Once we’ve got our datasets we’re going to need to update the repository structure to add them to the database. ePrints makes this relatively trivial with the following command:

That will automatically connect to the relevant database, create tables and add fields where necessary. Very funky.

The next stop on the list is to create a workflow for it. If you’re not familiar with how ePrints workflows behave then check out the ePrints wiki on the Workflow Format for more details. The basics are that it describes how fields should work on a screen and some limited scripting capability. To begin with we need to head to our /archives/repositoryname/cfg/workflows directory and create a new one. My dataset was called “service” so I created a new “service” directory. Because I’m lazy, I copied the user/default.xml file into service/default.xml and edited it to meet my needs. Mostly deleted the items and added the fields that I cared about. Again, if in doubt check out the documentation on workflows.

Now ePrints also has citation support and you guessed it we need to provide a citation. The Citation Format is documented in the wiki as well though a bit simpler than the citation format. To create this we jump to /archives/repositoryname/cfg/citations, create a new directory for the dataset (“service” again) and then because I’m lazy copy the user/default.xml file to service/default.xml and customise to what you need. The format is again pretty simple so follow your nose from what is on offer and again if in doubt check out the wiki.

At this point if you restart your ePrints instance to ensure the configuration is fully reloaded you need to grant your users some roles before you can see the dataset. If you jump into your favourite user and administer their account you will need to add some roles. There are a few different actions that appear to make sense:

• view – required to see the dataset
• create – add a new item to the dataset
• edit – edit existing items in the dataset
• destroy – remove an item from the dataset
• details – display a details screen for an item in the dataset

Each of these can be turned into a role by adding “+datasetname/action” into the roles box in the administrate user screen. In my particular case I did “+service/view”, “+service/create”, etc for those roles. Once you give yourself all of these roles you will be able to see under the “Manage Records” option (to the right of “Manage deposits”).

At this point you’re looking at it and it looks ugly because the strings aren’t translated properly. ePrints makes this easy, just click on the “Edit page phrases” button. It’ll list the phrases displayed on the screen and give you the option to add new phrases for the ones you’re missing whilst highlighting it. Fill in the blanks as you go and once you’ve added phrases for everything you’re ready to deploy out your data set!

So whilst we’re done and we can edit it, we probably want to grant it to our users to have a look at these as well. Adding these roles individually to our users isn’t fun, however if we edit the /archives/repositoryname/cfg/cfg.d/user_roles.pl file and add the roles to the existing user role we gain what we need. In addition to the existing roles we added previously (e.g. service/view), a generic user account will need the “datasets” role to be able to see the datasets tab. This makes the user_roles.pl section for user relatively simple:

Pretty easy. At this point our users can now add, edit and view records in the service dataset. Mission accomplished? The next step is to clean up everything, add a brief citation file in addition to the default citation and for me work out how to have per user items so that users can only edit their own items. But I think that is enough for today. Hopefully I’ll be able to work out how to limit items to just their own users but I might need to build my own screens. I’ve done this before with USQ’s Author ID project so I know how to do it, I’d just rather not.

The long and the short is if you want to test out the latest Joomla! 1.6 beta release (beta 9 at the moment), all you have to do is add our own custom feed to the Web Platform Installer!

The first step is to get the Web Platform Installer. The easiest way of doing this is going to Microsoft’s Web Platform download page and then download and install the Web Platform Installer. From here it’ll load and have a list of tabs on the left and some content on the right. At the very bottom left of the screen will be an option button that you can select. This will open up a window which has a text box around the middle with a button that says “Add Feed”. This box allows you to add custom feeds to WebPI to enable my custom Joomla! 1.6 feed to work. In the box put in the following URL: “http://update.joomla.org/webpi/webpi.xml” and click “Add Feed”. Hit ok and the screen should refresh and a new “Joomla!” tab should appear. Under the tab there should be a few options, I’ve tagged Joomla! a few different times (in future I might add nightly builds for both 1.5 and 1.6 to aide testing) but there is only on 1.6 Beta package. From here you should be able to install Joomla! and in theory it should download and configure any dependencies (e.g. IIS, PHP on IIS and MySQL).

As a word of warning, if you’re already set up using XAMPP or similar this might interfere with those environments however if you’re new and wanting to get into testing Joomla! 1.6 Beta on Windows and IIS (we always need more IIS testers), this should make your life much easier!

Next battle was fighting MySQL Workbench to get it to create an ER diagram for me. Eventually I managed to work out how not to trigger its “pure virtual function call” – you need to ensure that you close the bottom panel before opening a new table. I’m sure they’ll get around to fixing it one day. MySQL’s GUI tools have historically had problems for me. MySQL Query Browser rarely manages to let me graphically create a table without crashing. Yet I still try. I think I’ve managed to work out most of the error conditions to avoid them but it is sad that such easy to trigger bugs exists. In any case I managed to get the diagram modelled correctly which will be useful for future reference as well as for the Faculty of Business who were chasing it. Hopefully it is of use to them.

The afternoon was spent fighting with our EQUELLA instance and trying to work out why it was misbehaving. I didn’t work it out and ended up purging the item in question instead and starting on a new item. I’m not sure what I did to get it so grumpy but the item is purged now so it isn’t going to cause me an issue. The institution in question is going to be wiped out again in a part of testing. I managed to create some cool tools as a part of working out how the system works which should make some future work easier. Also finished work on an interface as well. I went home perhaps a little bit too late but it gave me the chance to concentrate on getting things worked out in peace. I found a minor quirk in Joomla! that I’d found previously but now I have a reasonable solution to get around it.

More work, more juggling but the wheel slowly turns.

Sedona doesn’t appear to offer any form of integration services so the fact that they could integrate with ePrints via SWORD (deposits) or OAI-PMH (harvesting) is sounding impossible even though it would be a great standard way to send data around. The suggested synchronisation involved FTP but details are light on the ground.

Apparently the system doesn’t work properly with Chrome to the point that it won’t let you login claiming you need IE, Firefox or Safari. The system isn’t normalised as well with the author names fields (first name, middle name and last name) being stored as individual fields in the database (e.g. fname1, fname2) and limited to five authors. For overflow there is an “et al.” multiline textbox. The system does have a method of adding co-authors however this is distinct and is again a multiline select box. There is another free form box for citations which makes sense as they limit the list of periodicals (drop down select list) and lack enough information to build most citation types from the data that the system stores.

The system does appear to have given some thought to an export with an Access database being generated with appropriate tables. Access, whilst not perhaps my preferred database system, is perhaps one of the better choices for data export and certainly beats sending CSV or XLS files around. There is the proprietary aspect to Access but there could be no option at all and at least maintains type data better. It also appears that this form of export is the way of adding extra fields that might be required by the University to store extra information. It wouldn’t surprise me if an Access database isn’t the preferred synchronisation tool though that personally scares me. But at least they have thought about it which is a good start.

Overall I look at the system and think we could integrate it into ePrints and increase the overall value of our data anyway. We could handle the few extra fields that Sedona appears to add with the extra item types. The extra datasets are exactly that: datasets. ePrints has workflow to handle editing items to do what it needs. It would be less limited and give us more power to do what we need.

Reports appear to be the systems strong point. There is a prebuilt CV and various other rather trivial reports for a member. There is a set of aggregate reports that I haven’t looked into and there appears to be a rather picky way of formatting things for the AACSB requirements (including printed CV’s in a particular format – I hope AACSB doesn’t cover IT related areas given the backwards requirements).

The fun task I ended the day with was returning to work on the monstrous PHP application I’ve been fighting with all week. The particular point I’m at is adding plugin functionality to a section of the code base. Since I’ve been incorporating the Joomla! Framework into the system, doing this turned out to be relatively trivial. At the point where I wanted the plugin trigger I load a file called ‘notifier.php’. This loads in the Joomla! framework bootstrap file which defines a while bunch of things (JPATH_BASE, JPATH_ROOT, etc) that Joomla! uses and loads the relevant files plus a few other files to get the system started off the bat. The ‘notifier.php’ file also defines a class aptly named ‘Notifier’ which extends JDispatcher. JDispatcher is the work horse of the Joomla! event subsystem from which plugins derive. The Notifier constructor loads up a particular directory and opens all of the files in it and tries to create new classes in the format plgNotifierFilename similar to how Joomla!’s plugin infrastructure works. This then registers those classes with the notifier class for use. Then in the original file I use JDispatcher’s “trigger” method to send a message to all of the registered plugins and the integration is done. In the original file there is only 4 lines of code added to load the file, create the class and trigger the message – plus a comment summarising this as well. It is one of those things where the framework makes life really easy to extend systems in a flexible manner. Now if I want to attach multiple plugins to extend that particular bit of code it is easy to do and with the framework it is easy to define multiple plugin triggers.

After the demo one of the most painful parts was the fact that we rely upon the CMS for a significant amount of content. The main problem with the CMS is that it is horribly unfriendly for mobile devices. At one stage the home page had a Flash banner which didn’t degrade properly which left a frustrating image that said “click here” but wasn’t actually linked to anything. So I spent the middle of the day working out how to strip out the USQ home page template so that we can embed it else where. I managed to get it sorted mostly though as I’m abusing an XML parser it’ll barf on poorly formatted pages or with HTML entities not in XML. Most of the CMS pages are fine but some legacy ones aren’t so lucky. I’ve put in some minor checks to clean things up but hopefully it isn’t too much of a problem.

The afternoon rounded out with a meeting before leaving early so that I could return for my evening of debates. Life in the fast lane.